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Foreword 


TECHNOLOGY  AND  EDUCATION 

Karl  Benz’s  invention  of  an  automobile  with  a  built- 
in  internal  combustion  engine  in  1885  caused  a  world¬ 
wide  revolution,  not  only  in  the  technological  field  but 
also  in  all  segments  of  human  life,  such  as  economics, 
culture,  family  structure,  urbanization,  women  and 
youth  emancipation,  and  climate  change.  A  car,  how¬ 
ever,  seems  more  suited  to  solve  mobility  problems 
than  to  serve  educational  goals.  The  rise  of  personal 
computers  and  network  facilities  in  the  second  half  of 
the  20th  century  eventually  converged  in  a  digital 
World  Wide  Web  (WWW)  that  revolutionized  infor¬ 
mation  development  and  exchange.  Increasing  minia¬ 
turization,  integrated  functionalities,  and  wireless  use 
now  comprise  a  communication  hyperspace  in  a  global 
world.  In  contrast  to  the  gas-fueled  engine,  information 
and  communication  technologies  suggest  sensitivity 
toward  lifelong  human  learning  issues.  This  is  what 
the  Handbook  of  Research  on  Educational  Communi¬ 
cations  and  Technology  is  all  about. 

Because  education  is  a  critical  topic  in  all  societies 
and  cultures,  it  is  a  common  social  impulse  to  (try  to) 
use  available  technology  for  schooling  purposes. 
Though  linking  technology  and  education  seems  at 
first  glance  to  be  a  rather  natural  endeavor,  it  is  by  no 
means  a  smooth  and  progressive  enterprise.  Indeed, 
the  history  of  educational  technology  documents  a 
long  and  often  difficult  process  of  mutual  adaptation, 
hesitation,  and  integration.  Closer  scrutiny  of  this  his¬ 
tory  reveals  some  cracks  or  fissures  in  a  less  than 
smooth  process  of  progressive  development;  these 
breaks  are  successively  stimulated  by  promises  of  pro¬ 
ducers  of  new  media  and  technologies.  From  their 
backgrounds  in  business  or  technology,  these  self-pro¬ 
claimed  educational  reformers  make  a  linear  forecast 
of  efficient  and  effective  output  for  learning  and 
instruction  but  neglect  the  complexity  of  multidimen¬ 
sional  and  multilevel  educational  settings.  Moreover, 
these  overly  optimistic  promises  are  formulated  before 
any  valid  empirical  evidence  for  success  has  been 
found.  Entire  lists  of  sayings  about  that  promised 
techno-land  are  abundantly  available  on  the  Internet. 
Teachers,  parents,  and  school  directors,  either  genuine 
innovators  themselves  or  pressed  by  policymakers, 
hold  strong  beliefs  about  the  positive  impact  of  new 
devices.  They  are  sensible  to  the  societal  need  for 
implementing  technology  in  modern  schools  and  using 


new  technologies  as  external  means  for  serving  an 
internal  need  for  innovation  (the  Trojan  horse  phenom¬ 
enon).  In  most  cases,  however,  experience  with  tech¬ 
nologies  brings  more  difficulties  and  less  productivity 
than  expected  at  the  beginning,  an  observation  that 
often  causes  frustration  even  in  the  research  commu¬ 
nity.  One  waits,  then,  until  the  next,  more  powerful 
tool  is  available.  This  resembles  the  myth  of  Sisyphus, 
a  king  of  Corinth  who  was  condemned  forever  to  roll 
a  boulder  up  a  hill  in  Hades  only  to  have  it  roll  down 
again  as  he  neared  the  top. 

Indeed,  media  and  technologies  were  invented  and 
developed  for  out-of-school  purposes,  which  means 
that  they  are  initially  not  natural  but  foreign  organs  in 
the  education  organism;  therefore,  we  are  in  dire  need 
of  transplantation  knowledge  to  cope  with  the  school’s 
immune  system.  Building  this  transplantation  knowl¬ 
edge— to  stick  with  our  metaphor— is  not  a  mere  tech¬ 
nological  issue;  it  also  has  to  receive  an  empirical 
grounding  in  a  multidisciplinary,  broad-spectrum,  and 
multi-vocal  research  approach.  In  this  way,  research  on 
learning  and  instruction,  philosophy,  curriculum,  meth¬ 
ods,  school  organization,  technology,  and  user  charac¬ 
teristics  could  converge  in  educational  innovation  strat¬ 
egies  that  take  into  account  increasing  complexity. 

EVOLUTION  IN  MEDIA 
AND  TECHNOLOGY 

Depending  on  the  attributes  of  the  different  media  or 
technologies  and  the  educational  contexts  in  which 
they  are  embedded,  different  answers  are  given  or  need 
to  be  given  to  build  up  a  fruitful  interaction  between 
technology  and  education.  Some  speak  in  this  context 
of  a  marriage.  A  quick  scan  of  evolutions  in  media  and 
technology  reveals  various  stages  of  development; 
however,  rather  than  suggesting  a  definite  break 
between  each  developmental  step,  which  leads  inevi¬ 
tably  to  isolation  or  compartmentalization,  it  seems 
more  apt  to  use  the  concept  of  a  wave.  Educational 
technology  is  a  cumulative  endeavor  in  which  knowl¬ 
edge  from  earlier  phases  can  be  absorbed  smoothly  in 
more  recent  waves.  To  mention  only  a  few  examples, 
former  research  on  the  effects  of  media  attributes  or 
symbol  systems  can  easily  be  integrated  into  a  com¬ 
plementary  view  on  different  symbol  systems  in  mul¬ 
timedia  systems;  early  investigations  on  hypercard  and 
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hypertext  continue  to  yield  insight  into  the  character¬ 
istics  of  hypermedia;  research  on  mental  effort  can  be 
integrated  into  the  so-called  cognitive  load  theory;  and 
research  on  peer  learning  with  personal  computers  can 
shed  light  on  key  aspects  of  learning  communities  in 
an  Internet  environment. 

Evolutions  in  technology  can  be  summarized  in 
three  successive  developments.  After  World  War  II, 
information  representation  media  were  expected  to 
support  traditional  teaching  and  learning,  mostly  in 
classrooms.  Such  media  were  supposed  to  bring  the 
real  world  into  the  stuffy  classroom  with  the  use  of 
radio,  television,  slideshows,  film,  and  video.  This  kind 
of  technology  was  conceived  of  as  an  extension  of 
verbal  and  textual  symbol  systems  contained  in  teacher 
lectures  and  books;  however,  the  traditional  school 
format  hindered  the  flexible  adaptation  of  these  new 
media  due  to  locked  time  slots,  predefined  curricula, 
teachers’  resistance  toward  change,  and  limited  orga¬ 
nizational  flexibility,  infrastructure,  and  finances.  If 
used,  media  were  a  mere  add-on  that  brought  about  no 
fundamental  change  in  teaching  and  learning 
approaches. 

From  1970  on,  the  increasing  use  of  computers  in 
society  created  a  dual  argument  for  integrating  com¬ 
puters  into  educational  settings.  On  the  one  hand,  soci¬ 
ety  claimed  that  youths  had  to  be  prepared  to  live  in 
an  information  society,  equipped  with  the  computer 
skills  necessary  for  driving  on  the  information  high¬ 
way  (though  nowadays  youths  seem  to  outperform 
their  teachers  and  parents  in  their  use  of  digital  tech¬ 
nology).  On  the  other  hand,  the  attributes  of  computers 
allowed  for  interactive  data  processing,  symbol  trans¬ 
formation,  and  information  storage,  which  seemed 
compatible  with  human  information  processing  and 
knowledge  construction.  The  personal  computer  was 
loaded  with  software,  ranging  from  drill-and- practice 
programs  to  simulations  and  open  tools.  In  terms  of 
innovation  strategy,  most  governments  assumed  that 
filling  schools  with  computers  would  automatically 
enhance  learning  processes  and  higher-order  skill 
acquisition.  Research  has  shown,  however,  that  com¬ 
puters  only  create  learning  output  if  sufficient  support 
is  available  and  computer  use  is  an  integral  part  of  the 
curriculum  or  learning  environment. 

Since  the  last  decade  of  the  20th  century,  the  closed 
position  of  personal  computers  has  been  opened  by  the 
communication  facilities  of  networked  computers. 
Computers  are  no  longer  personal  or  even  computing 
machines  that  manipulate  digits.  They  are  tools  that 
activate  human  communication.  On  the  one  hand, 
these  tools  complement,  correct,  or  fine-tune  informa¬ 
tion  embedded  in  educational  software  or  available  on 
the  Internet;  on  the  other  hand,  they  can  be  used  to 


develop  new  information  and  shared  knowledge 
through  computer  interaction.  This  human  shield 
allows  for  many  interactions  among  experts,  coaches, 
teachers,  peers,  parents,  and  learners.  Computers  are 
communication  tools  in  an  information-rich  environ¬ 
ment  and  are  used  by  myriad  technology-enhanced 
communities  of  learners  and  communities  of  practice. 

INNOVATIVE  RESEARCH  ON 
EDUCATIONAL  TECHNOLOGY 

The  shared  feeling  that  recent  communication  and 
information  technologies  are  much  closer  to  education 
than  their  nondigital  predecessors  could  lead  to  a 
smoother  educational  use.  This  opinion  can  be  sus¬ 
tained,  at  least  from  a  logical  perspective;  however, 
innovations  in  instruction  and  learning  are  dependent 
not  only  on  logical  perspectives  but  also  on  psycho¬ 
logical  motives.  Surveys  on  computer  use  indicate  that 
only  a  fraction  of  the  time  spent  with  computers  is 
used  for  educational  or  learning  purposes.  Contrary  to 
the  simple  procedure  of  finding  useful  information  on 
the  Internet,  learning  and  knowledge  building  are 
developmental  processes  that  integrate  new  knowledge 
into  already  existing  prior  knowledge  and  lead  to 
increasing  abstraction,  comprehension,  and  conceptual 
change.  Learning  is  cumulative  and  an  object  of  mental 
effort,  motivation,  and  cognitive  skilfulness.  Learning 
is  not  so  much  defined  by  the  quantity  of  information 
available  but  by  the  gradual  transformation  of  relevant 
information  into  knowledge.  This  is  not  a  matter  of 
mere  information  delivery  in  an  information  society. 
It  is  essentially  the  way  novices  are  supported  in  build¬ 
ing  their  own  knowledge  out  of  available  information 
in  a  long-lasting  effort  and  a  long-term  perspective,  as 
well.  Learning  is  essentially  a  developmental  activity. 

Although  research  on  educational  technology  is 
often  tuned  toward  evolution  in  the  field,  it  started  with 
many  studies  at  a  microlevel  of  complexity:  assess¬ 
ment  of  the  effectiveness  and  efficiency  of  computer 
software,  detection  of  minimal  conditions  for  imple¬ 
mentation  of  technology  in  schools,  insight  into  the 
main  characteristics  of  learners  confronted  with  tech¬ 
nology,  and  so  on.  Later  on,  when  not  only  technolog¬ 
ical  tools  but  also  human  interactions  with  tools  and 
peers  became  dominant,  research  was  more  complex 
and  situated  at  the  mesolevel  of  complexity.  Integral 
school  innovation,  integration  of  computers  in  curric¬ 
ula,  and  interaction  between  offline  and  online  learning 
or  between  face-to-face  and  distant  learning  are  topics 
that  require  intensive  research.  The  rise  of  both  the 
complexity  and  the  range  of  technology  use  in  educa¬ 
tion  calls  for  adapted  research  approaches.  If  the  entire 
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educational  system  is  to  become  the  object  of  research, 
the  only  answer  is  a  systemic  approach.  Systemic 
means  that  both  the  conceptual  coherence  and  the 
empirical  method  contribute  to  a  better  understanding 
of  the  complexity,  leading  toward  better  designs. 
Instead  of  a  proliferation  of  terminology,  approaches, 
methods,  and  variables,  research  should  aim  at  accu¬ 
mulation.  The  centrifugal  trend  needs  to  be  trans¬ 
formed  into  a  centripetal  one:  bringing  researchers 
together  through  strong  research  programs. 

Complexity  cannot  be  solved  by  simple  or  mono- 
disciplinary  approaches.  Because  the  system  consists 
of  many  subcomponents  that  refer  to  different  back¬ 
grounds  and  interpretation  frameworks,  the  solution 
lies  in  an  interdisciplinary  research  endeavor.  Indeed, 
technology,  curriculum  development,  courseware 
development,  distance  teaching,  innovation  strate¬ 
gies,  philosophy,  social  psychology,  anthropology, 
learning  paradigms,  organizational  issues,  and  ethics 
all  define  the  value  of  modern  technology  use.  This 
is  simply  due  to  the  fact  that  at  the  macrolevel  of 
complexity  highly  organized  blended  learning 
includes  and  integrates  technology  and  human  inter¬ 
actions  among  the  individual,  group,  and  community 
levels  of  organization. 

The  research  question  is  no  longer  how  individual 
studies  contribute  to  evidence-based  uses  of  technol¬ 
ogy  but  how  these  studies  can  be  brought  together  in 
multidisciplinary  and  multilevel  research  programs. 
This  includes  an  emphasis  on  interactions  between 
variables  and  levels  rather  than  on  individual,  flat  stud¬ 
ies;  for  example,  because  open  technological  environ¬ 
ments  call  for  learner  control,  self-regulation,  and 
group  learning,  the  nature  of  technology  support  has 
to  be  defined  in  the  complex  relationship  between 


learner  control  and  environmental  (technological)  sup¬ 
port.  In  this  way,  research  programs  can  take  into 
account  long-term  goals  and  flexible  adaptation  toward 
changes  in  the  environment. 

Most  technology  research  is  focused  on  a  limited 
sample  (mostly  college  students)  with  a  restricted  set 
of  one-shot  research  questions.  We  know  little  about 
evolutions  in  technology  use  or  about  developmental 
characteristics  in  the  transition  from  novice  to  expert. 
There  is  a  need  for  more  longitudinal  studies  that  can 
shed  light  on  this  evolutionary  perspective. 

Since  the  early  times  of  design  and  development 
of  learning  software,  emphasis  has  been  placed  on  the 
designer  and  on  his  or  her  expertise.  If,  however,  learn¬ 
ers  are  to  codesign  their  learning  environment  as  part¬ 
ners  in  a  learning  community,  it  will  be  necessary  to 
conduct  research  on  the  users:  the  complex  mindset  of 
policymakers,  school  directors,  coaches,  teachers,  and 
students,  as  well  as  their  competencies  in  cobuilding 
environments.  This  includes  the  required  instructional 
knowledge  for  contributing  to  optimal  learning  envi¬ 
ronments. 

The  information  made  available  in  this  Handbook 
by  acknowledged  scholars  depicts  many  contributions 
to  the  foundation,  strategies,  technologies,  models, 
design  and  development,  and  methodology  of  educa¬ 
tional  communications  and  technology.  One  of  the 
expected  outcomes  of  this  immense  knowledge  base 
is  the  creation  of  a  new  generation  of  research  and 
development,  in  which  conceptual  alignment,  empiri¬ 
cal  richness ,  and  synergy  are  spearheads  of  educational 
innovation. 

Joost  Lowyck 

Professor  Emeritus,  Leuven  University,  Belgium 
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Preface 


HISTORY 

Early  in  2005,  Phil  Harris,  Executive  Director  of  the 
Association  of  Educational  Communications  and 
Technology,  approached  Mike  Spector  and  asked  if  he 
would  like  to  lead  the  development  of  the  third  edition 
of  the  Handbook  of  Research  on  Educational  Commu¬ 
nications  and  Technology.  Spector  consulted  with 
Dave  Jonassen,  editor  of  the  previous  two  editions,  and 
with  Lane  Akers ,  Senior  Education  Editor  at  Routledge 
Publishers.  A  strategy  was  then  developed  to  make  the 
third  edition  more  international  in  terms  of  scope  and 
contributions  and  to  have  an  editorial  team  rather  than 
an  individual  editor  lead  the  effort.  Spector,  Harris, 
Akers,  and  Jonassen  then  discussed  who  might  com¬ 
prise  that  editorial  team.  By  May  2005,  the  editorial 
team  of  Mike  Spector,  Dave  Merrill,  Jeroen  van  Mer- 
rienboer,  and  Marcy  Driscoll  was  formed. 

The  strategy  developed  to  determine  the  contents 
had  four  major  steps:  (1)  extended  conversations  with 
Jonassen  to  take  lessons  learned  from  the  first  two 
editions  into  this  edition;  (2)  a  survey  of  AECT  mem¬ 
bers  to  get  their  reactions  to  the  second  edition  and 
recommendations  for  the  third  edition;  (3)  presenta¬ 
tions  at  the  annual  meetings  of  AECT  in  2005  and 
again  in  2006  to  get  direct  feedback  from  AECT 
members  about  plans  for  the  third  edition;  and  (4) 
ongoing  discussions  among  the  editors  via  telecon¬ 
ference,  videoconference,  and  occasional  face-to-face 
meetings  about  the  Handbook. 

Nearly  200  persons  responded  to  the  Handbook 
survey,  which  asked  about  general  use,  individual 
chapters  in  the  second  edition,  and  their  desires  for  the 
third  edition.  Responses  indicated  that  the  Handbook 
is  used  primarily  by  doctoral  students  initiating  a 
research  review  for  their  dissertation  studies,  by  fac¬ 
ulty  as  an  additional  resource  for  teaching  courses  on 
related  topics,  and  by  researchers  seeking  a  quick 
review  on  a  specific  topic.  Recommendations  for  the 
third  edition  included  addressing  more  topics  and  hav¬ 
ing  shorter  chapters  with  more  references  to  recent 
research.  We  followed  these  recommendations  to  the 
best  of  our  ability. 

At  the  2005  AECT  meeting,  the  editorial  team 
developed  the  basic  organization  of  the  Handbook 
around  four  parts:  (1)  Strategies,  (2)  Technologies,  (3) 
Models,  and  (4)  Design  and  Development.  These  parts 
reflect  major  aspects  involved  in  deploying  informa¬ 


tion  and  communications  technologies  (ICT)  for  edu¬ 
cational  purposes;  they  are  preceded  by  a  Foundations 
part  and  followed  by  a  Research  Methodologies  part. 
Each  part  was  led  by  one  of  the  coeditors  with  assis¬ 
tance  from  a  second  coeditor  and  several  external 
reviewers. 

Also  at  that  meeting,  the  editorial  team  with  sig¬ 
nificant  input  from  Jonassen  developed  an  initial  list 
of  desired  contributors.  The  agreement  with  our  pub¬ 
lisher,  Lawrence  Erlbaum  Associates,  was  that  the 
second  edition  would  remain  available  online  through 
the  AECT  website  to  AECT  members ,  so  some  excel¬ 
lent  chapters  in  that  edition  were  not  included  in  this 
third  edition.  Author  guidelines  were  developed  that 
reflected  the  outcomes  of  the  survey  and  the  organi¬ 
zational  framework  developed  by  the  editorial  team. 
Specifically,  we  asked  for  shorter  chapters  with 
longer  bibliographies,  and  we  also  requested  key¬ 
words  with  definitions  to  be  included  in  a  glossary 
and  an  indication  (via  asterisks  in  the  reference  sec¬ 
tion)  of  core  references. 

A  general  call  for  contributions  was  issued  to  the 
AECT  membership.  The  lead  editor  for  each  part  of 
the  Handbook  was  responsible  for  determining  the 
contributing  authors  to  that  part  and  for  coordinating 
drafts  and  reviews  of  that  part  of  the  Handbook.  In 
short,  the  editorial  team  functioned  as  a  team,  with  co¬ 
editors  having  and  taking  responsibility  for  the  various 
parts  of  this  Handbook. 

We  made  a  conscious  effort  from  the  very  begin¬ 
ning  to  include  many  more  non-American  contribu¬ 
tors.  Rather  than  simply  ask  those  who  had  contributed 
to  the  first  two  editions  to  again  make  a  contribution, 
we  determined  that  we  ought  to  think  ahead  to  the 
fourth  edition.  We  also  knew  that  the  second  edition 
would  remain  available  for  the  indefinite  future  .  As  a 
consequence,  when  we  eventually  approached  authors, 
we  expressed  a  preference  for  pairing  a  highly  expe¬ 
rienced  author  with  a  promising  young  scholar  who 
conceivably  could  lead  a  contribution  in  the  next  edi¬ 
tion.  The  intent  in  selecting  authors  was  to  ensure  that 
chapters  would  be  broadly  representative  of  the  rele¬ 
vant  research  in  a  particular  area  rather  than  reflecting 
only  one  view  or  approach. 

The  Handbook  took  shape  in  late  2005.  At  that 
time,  we  had  as  many  as  100  chapters  targeted  for 
development.  The  editorial  team  narrowed  that  list  to 
about  65  in  early  2006  when  development  of  individual 
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chapters  took  place.  Along  the  way,  a  few  contributors 
decided  they  could  not  make  the  contributions  they 
had  wanted  to  make  and  withdrew  from  the  process. 
One  of  the  most  distinguished  contributors  who  had 
also  made  valuable  contributions  to  the  first  two  edi¬ 
tions,  Bill  Winn,  passed  away  unexpectedly.  Bill  can¬ 
not  be  replaced;  however,  his  chapter  on  computer- 
mediated  technologies  was  a  critical  one.  Art  Graesser 
agreed  to  take  on  the  formidable  task  of  leading  the 
development  of  that  chapter  in  a  short  period  of  only 
six  months.  In  the  end,  the  third  edition  of  this  Hand¬ 
book  consists  of  56  chapters  divided  into  six  parts. 


CONTENTS 

As  already  noted,  the  third  edition  is  divided  into  six 
parts.  Part  I,  Foundations,  led  by  Marcy  Driscoll,  is 
expected  to  remain  fairly  stable  over  the  years  and 
require  only  minor  updating  every  five  years  or  so.  The 
Foundations  part  includes  historical,  theoretical,  and 
methodological  foundations  and  perspectives.  This  ini¬ 
tial  part  of  the  Handbook  is  aimed  at  the  various  sets 
of  assumptions  that  underlie  research  in  educational 
communications  and  technology.  Some  of  these 
assumptions  are  based  on  what  has  gone  before.  Others 
are  based  on  developments  in  other  disciplines.  The 
goal  in  this  part  of  the  Handbook  is  to  make  these 
assumptions  explicit,  summarize  key  developments, 
and  provide  pointers  to  exemplary  work  that  has  impli¬ 
cations  for  research  in  educational  communications 
and  technology. 

Part  II,  led  by  Dave  Merrill,  is  focused  on  strat¬ 
egies.  The  various  chapters  in  the  Strategies  part 
cover  both  instructional  and  learning  strategies, 
although  the  emphasis  is  on  implications  for  design 
and  development.  These  various  strategies  can  be 
linked  with  subsequent  chapters  in  Parts  III,  IV,  and 
V  of  the  Handbook ,  in  accordance  with  our  organi¬ 
zational  framework. 

Part  III,  led  by  Mike  Spector,  is  focused  on  tech¬ 
nologies.  The  editorial  team  collectively  decided  that 
the  distinction  between  hard  and  soft  technologies  was 
too  facile  and  not  especially  meaningful,  and  it  was 
dropped  in  this  edition.  The  Technologies  part  of  the 
Handbook  consists  of  17  chapters  on  both  digital  and 
non-electronic  technologies,  intelligent  and  non-intel- 
ligent  technologies,  and  planning  and  evaluation  tech¬ 
nologies,  as  well  as  technologies  for  implementation. 

Part  IV,  led  by  Jeroen  van  Merrienboer,  focuses  on 
models.  Issues  concerned  with  various  types  of  and 
approaches  to  learning  are  discussed.  These  models 
clearly  inform  design  and  development  and  can  be 


linked  to  various  instructional  strategies  covered  in 
Part  II.  The  Models  part  includes  general  models 
directed  toward  learning  in  schools  as  well  as  outside 
schools  and  models  that  focus  on  learning  in  specific 
domains  such  as  medicine,  science,  and  reading. 

Part  V,  led  by  Dave  Merrill .  focuses  on  design  and 
development.  This  part  of  the  Handbook  discusses 
research  that  pertains  directly  to  professional  practice. 
Readers  will  find  chapters  on  familiar  topics  such  as 
competency  development,  task  analysis,  change 
agency,  and  performance  assessment.  The  Design  and 
Development  part  also  covers  innovative  treatments  of 
design  languages,  design  and  development  teams,  and 
user-centered  design  and  development. 

Part  VI,  led  by  Jeroen  van  Merrienboer,  focuses  on 
methodological  issues.  This  part  follows  the  empirical 
cycle  through  theory  development,  experimental 
design,  and  data  collection  and  analysis.  Design  sec¬ 
tions  adhere  to  the  main  parts  of  the  Handbook  and 
treat,  in  order,  research  on  strategies,  technologies, 
models,  and  design  and  development.  For  data  collec¬ 
tion,  special  requirements  for  (virtual)  laboratories  are 
discussed.  Analysis  methods  include  the  analysis  of 
learning  processes,  interactions,  and  complex  perfor¬ 
mances.  The  Methodological  Issues  part  ends  with  a 
discussion  by  the  editorial  team  of  a  research  agenda 
that  should  help  our  field  to  build  a  strong  scientific 
foundation  for  the  future. 


USE 

The  division  into  four  core  parts  that  represent  key 
aspects  of  using  information  and  communications 
technologies  to  support  learning  and  instruction,  pre¬ 
ceded  by  a  Foundations  section  and  followed  by  a 
Methodological  Issues  section,  is  intended  to  facilitate 
use  of  this  research  handbook.  We  have  intentionally 
kept  titles  brief  and  descriptive  to  help  facilitate  those 
who  wish  to  follow  a  thread  through  the  four  core  parts 
of  the  Handbook. 

An  example  of  a  thread  concerned  with  research 
on  instructional  modeling  and  representation  might 
involve  the  chapters  on  representation  strategies,  mod¬ 
eling  technologies,  model-facilitated  learning,  and 
design  languages.  Many  such  threads  are  possible. 
Individual  chapters  were  primarily  developed  to  rep¬ 
resent  stand-alone  treatments  of  specific  topics  within 
the  framework  and  guidelines  provided.  All  chapters 
have  extensive  lists  of  references  that  should  prove 
useful  to  researchers  new  to  a  particular  area  of 
research  and  to  doctoral  students  conducting  their 
background  research. 
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It  is  our  belief  that  professional  practitioners  and 
educational  researchers  will  benefit  from  the  chapters 
in  Parts  I  and  VI.  These  chapters,  as  well  as  many 
others  in  the  Handbook ,  are  likely  to  be  useful  for 
those  responsible  for  leading  graduate  seminars  in  the 
areas  of  educational  technology,  instructional  systems, 
or  learning  environment  design  and  development.  It  is 
our  hope  that  this  edition  of  the  Handbook  will  be  as 


useful  as  the  previous  two  editions.  Time  will  tell.  In 
any  case,  we  invite  you  to  give  us  your  feedback,  which 
we  will  pass  along  to  whoever  is  selected  to  lead  the 
development  of  the  fourth  edition. 

Learn,  educate  and  persevere. 

Mike  Spector,  Dave  Merrill, 
Jeroen  van  Merrienboer,  and  Marcy  Driscoll 
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Part  I 

Foundations 


This  Foundations  part  of  the  Handbook  was  led  by 
Marcy  P.  Driscoll  with  assistance  from  J.  Michael  Spec- 
tor.  The  Foundations  are  expected  to  remain  fairly  sta¬ 
ble  over  the  years.  The  relevant  Foundations  include 
historical,  theoretical,  and  methodological  develop¬ 
ments,  theories,  and  perspectives.  This  initial  part  of 
the  Handbook  is  aimed  at  the  various  sets  of  assump¬ 
tions  that  underlie  research  in  educational  communica¬ 
tions  and  technology.  Some  of  these  assumptions  are 
based  on  what  has  gone  before.  Others  are  based  on 


developments  in  other  disciplines.  The  goal  in  this  part 
of  the  Handbook  is  to  make  these  assumptions  explicit, 
summarize  key  developments,  and  provide  pointers  to 
exemplary  work  that  has  implications  for  research  in 
educational  communications  and  technology.  This  part 
of  the  Handbook  consists  of  seven  chapters  covering: 
(1)  historical  foundations,  (2)  theoretical  foundations, 
(3)  complexity  theory,  (4)  experiential  perspectives,  (5) 
empirical  perspectives,  (6)  contextualistic  perspectives, 
and  (7)  philosophical  perspectives. 
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ABSTRACT 

Research  and  practice  in  educational  technology  are 
rooted  in  a  primordial  human  drive  to  find  ways  of 
teaching  in  ways  that  are  more  efficient.  Every  civili¬ 
zation  has  developed  formal  methods  of  education 
more  efficacious  than  the  trial-and-error  of  everyday 
living.  In  the  first  decades  of  the  20th  century,  indi¬ 
viduals  and,  later,  groups  of  affiliated  professionals 
made  that  quest  a  central  focus,  thus  establishing  edu¬ 
cational  technology  as  a  field.  Their  first  activities 
aimed  at  enriching  the  learning  experience  with  visual 
and  later  audio-visual  resources.  As  radio  broadcasting 
grew  in  the  1930s  and  then  television  in  the  1950s, 
these  mass  media  were  accepted  as  ways  to  reach  even 
larger  audiences,  in  and  out  of  school,  with  educative 
audio-visual  programs.  In  the  1960s,  the  wave  of  inter¬ 
est  in  teaching  machines  incorporating  programmed 


instruction  based  on  behaviorist  psychology  engulfed 
the  field,  engendering  a  shift  in  identity.  The  proper 
study  of  the  field  expanded  from  audio-visual  technol¬ 
ogies  to  all  technologies,  including  psychological 
ones.  By  the  1980s,  the  center  of  gravity  had  shifted 
to  the  design  of  instructional  systems,  especially  the 
adroit  application  of  instructional  methods,  enlivened 
by  fresh  insights  from  cognitive  and  constructivist  per¬ 
spectives.  As  computers  became  ubiquitous  in  the 
1990s,  they  became  the  delivery  system  of  choice  due 
to  their  interactive  capabilities.  With  the  rapid  global 
spread  of  the  World  Wide  Web  after  1995,  networked 
computers  took  on  communication  functions  as  well 
as  storage  and  processing  functions.  The  21st  century 
began  with  educational  technology  increasingly 
focused  on  distance  education,  the  latest  paradigmatic 
framework  for  its  ageless  mission  to  help  more  people 
learn  faster,  better,  and  more  affordably. 
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KEYWORDS 

Constructivism:  In  learning  theory,  a  set  of  assump¬ 
tions  about  human  learning  emphasizing  the  cen¬ 
tral  role  of  the  mind’s  active  construction  of  new 
knowledge. 

Distance  education:  An  educational  program  charac¬ 
terized  by  the  separation,  in  time  or  place,  between 
instructor  and  student  and  in  which  communica¬ 
tions  media  are  used  to  allow  interchange. 
Technology:  The  application  of  scientific  or  other  orga¬ 
nized  knowledge  to  practical  tasks. 

INTRODUCTION 

The  area  of  research  surveyed  in  this  handbook— edu¬ 
cational  communications  and  technology  — is  broad 
and  complex.  The  constructs  on  which  the  individual 
chapters  focus  sometimes  have  vague  boundaries  and 
often  overlap  with  other  constructs.  The  research  sur¬ 
veyed  is  rooted  in  many  different  disciplines,  each  with 
its  own  history  and  subculture.  Any  attempt  to  impose 
a  coherent  story  line  on  such  a  variegated  drama  must 
necessarily  be  a  bit  complicated,  with  plot  lines  that 
crisscross  frequently.  This  brief  history  makes  no  claim 
to  originality  or  heterodoxy.  It  strives  for  the  opposite 
effect:  to  tell  the  story  of  the  evolution  of  educational 
communications  and  technology  as  it  is  understood  by 
mainstream  observers.  It  draws  heavily  on  well-known 
sources,  such  as  Saettler’s  (1990)  comprehensive  his¬ 
tory  and  the  most  recent  overview  of  the  main  con¬ 
structs  of  the  field  (Januszewski  and  Molenda,  2008). 
It  is  animated  by  the  editors’  goal  of  beginning  this 
handbook  by  making  explicit  the  assumptions  on  which 
research  has  been  based.  It  takes  the  vantage  point  of 
the  membership  and  readership  of  the  Association  for 
Educational  Communications  and  Technology  (AECT) 
and  its  predecessors  over  the  past  century.  For  simplic¬ 
ity’s  sake,  the  term  educational  technology  will  be  used 
as  the  name  of  the  field  whose  story  is  being  told. 

HISTORICAL  FOUNDATIONS 
OF  WHAT? 

The  Very  Beginning 

Humans  have  succeeded  as  a  species  largely  due  to 
their  ability  to  learn  from  their  experiences  and  to  pass 
along  their  wisdom  to  succeeding  generations.  Much 
learning  and  acculturation  happens  spontaneously, 
without  planning  or  structure.  Through  the  ages,  as 
human  society  has  become  increasingly  complex  and 


organized,  communities  have  consciously  set  up  par¬ 
ticular  arrangements,  such  as  apprenticeships,  schools, 
and  other  educational  institutions,  to  help  their  mem¬ 
bers  develop  the  cognitive  and  functional  skills  needed 
to  survive  and  flourish. 

The  history  of  organized  education  and  training 
can  be  viewed  as  a  long  struggle  to  extend  opportuni¬ 
ties  to  more  people  and  to  devise  means  of  helping 
those  people  learn  better  than  through  the  events  of 
everyday  life.  Institutions  established  for  education 
and  training  revolve  around  activities  intended  to  help 
people  learn  productively,  individually  or  in  groups,  in 
classrooms  or  at  a  distance.  Schools,  colleges,  corpo¬ 
rate  training  centers,  and  other  educational  institutions 
provide  many  sorts  of  facilities  to  facilitate  learning. 

Learning  goals  in  educational  settings  are  often 
complex,  difficult,  and  protracted.  Throughout  history, 
educators  have  devised  means  to  help  people  learn  that 
are  easier,  faster,  surer,  or  less  expensive  than  previous 
means.  Some  of  these  means  could  be  classified  as 
technological,  by  which  we  mean  applying  scientific 
or  other  organized  knowledge  to  the  attainment  of 
practical  ends,  a  definition  proposed  by  John  Kenneth 
Galbraith  (1967).  These  developments  may  take  the 
form  of  hard  technologies,  including  materials  and 
physical  inventions,  or  soft  technologies,  including 
special  work  processes  or  carefully  designed  instruc¬ 
tional  templates  that  are  applicable  beyond  a  single 
case.  This  chapter  aims  to  recount  some  of  the  mile¬ 
stones  in  the  history  of  these  developments. 

Precursors  of  the  Modern  Era 

The  ideas  that  have  propelled  educational  technology 
during  its  modem  history  have  their  roots  in  philosoph¬ 
ical,  pedagogical,  and  psychological  theories  stretch¬ 
ing  back  to  the  5th  century  B.C.,  when  Athenian  cul¬ 
ture  was  at  its  zenith  in  the  West  and  when  Confucius 
was  establishing  his  philosophy,  which  came  to  dom¬ 
inate  East  Asian  thinking.  (Confucian  thought,  how¬ 
ever,  was  not  known  in  the  Western  world  until  the 
translations  of  Italian  Jesuit  missionary  Matteo  Ricci 
around  1600.) 

In  classical  Athens,  the  Sophists  taught  provoca¬ 
tive,  often  relativistic,  notions  of  epistemology.  The 
works  of  Socrates,  Plato,  and  Aristotle  in  organizing 
philosophical  thought  can  be  seen  as  a  reaction  against 
the  Sophists’  position  that  a  good  argument  is  one  that 
prevails,  if  only  through  rhetorical  manipulation, 
regardless  of  truth  value.  Their  frameworks  for  discus¬ 
sion  of  cognition  and  knowledge  were  largely  lost 
during  the  Dark  Ages  in  Europe  but  were  gradually 
rediscovered  and  reexamined  as  medieval  scholars 
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gained  access  to  texts  saved  in  Arabic.  During  the  15th 
century,  Yi  T’oegye  in  Korea  was  developing  a  neo- 
Confucian  philosophy  that  focused  on  moral  principles 
but  also  treated  epistemology.  His  Steps  of  Practical 
Self-Cultivation,  procedures  for  thinking  through 
problems,  are  comparable  to  the  maieutic  method  of 
Socrates.  (Socrates  considered  his  educational  practice 
to  be  similar  to  midwifery  in  that  he  helped  individuals 
deliver  ideas;  see  Kim,  2003.) 

By  the  Renaissance  era,  European  philosophers  of 
education  such  as  Comenius  were  elaborating  peda¬ 
gogical  principles  and  practices  that  are  recognizable 
to  the  modem  educator— for  example,  arranging  the 
classroom  for  efficient  management,  systematically 
incorporating  visuals  into  text  presentations,  organiz¬ 
ing  the  curriculum  according  to  the  developmental 
stages  of  learners,  and  engaging  children  in  playful 
activities  instead  of  punishing  drills. 

Advances  in  communications  media  came  to  edu¬ 
cation  slowly  in  the  18th  and  19th  centuries.  Maps, 
globes,  and  scientific  apparatus  were  standard  equip¬ 
ment  in  the  better  schools  and  colleges  in  the  18th 
century,  but  it  was  not  until  early  in  the  19th  century 
that  a  new  general-purpose  media  format— the  black¬ 
board— came  into  widespread  use.  The  Scots  claim  that 
the  blackboard  was  invented  by  James  Pillans,  head¬ 
master  of  the  Old  High  School  in  Edinburgh  in  the  early 
1800s,  who  used  a  blackboard  and  colored  chalks  to 
teach  geography  (Scots  Community,  2007).  By  1830, 
the  blackboard,  usually  locally  made  by  painting  planks 
with  black  paint,  had  become  an  essential  part  of  class¬ 
room  furnishings.  Its  ability  to  make  teacher  or  student 
writing  or  drawings  visible  to  a  large  group  expanded 
the  teacher’s  capabilities  exponentially.  Bumstead 
(1841,  p.  viii)  proclaimed  that  the  “inventor  or  intro¬ 
ducer  of  the  blackboard  system  deserves  to  be  ranked 
among  the  best  contributors  to  learning  and  science,  if 
not  among  the  greatest  benefactors  of  mankind.” 

The  hand-held  stereoscope  became  popular  in  edu¬ 
cation  in  the  mid- 1850s,  promoted  by  Sir  David  Brew¬ 
ster  in  England,  who  carried  out  basic  research  in 
stereoscopy  and  became  a  firm  advocate  of  its  value 
in  visualizing  the  curriculum  (Anderson,  1962). 

EARLY  VISUAL  MEDIA 
Slide  Projection 

The  origins  of  the  modem  field  of  educational  technol¬ 
ogy  can  be  traced  to  the  efforts  of  practitioners  in  the 
late  19th  and  early  20th  century  to  use  projected  visual 
images  to  supplement  lectures.  Slide  projection  evolved 
from  17th-century  handpainted  slides  illuminated  by 


oil  lamps.  The  so-called  magic  lantern  provided  enter¬ 
tainment  for  paying  audiences  throughout  the  19th  cen¬ 
tury  (Petroski,  2006).  The  use  of  slide  projection  in 
education  was  restricted  by  the  high  cost  of  purchasing 
and  operating  these  early  devices.  They  ran  on  gas,  oil, 
or  hydrogen  combined  with  lime  (so-called  limelight, 
first  used  in  the  Covent  Garden  Theatre  in  London  in 
1837),  all  of  which  had  a  high  cost  per  hour  of  use. 
Edison’s  invention  in  the  1890s  of  incandescent  light¬ 
ing  powered  by  electricity  made  slide  projection  afford¬ 
able,  and  by  the  end  of  the  19th  century  lantern  slides 
were  in  common  use  in  education. 

Silent  Films  in  Education 

The  direct  ancestors  of  educational  films  were  the  non- 
theatrical  short  films  that  began  to  emerge  around  1910. 
British  and  French  cinematographers  exhibited  films 
showing  amazing  sights  such  as  microscopic  creatures, 
insects  in  flight,  and  underwater  seascapes.  Films  of 
news  events  and  travel  adventures  played  to  rapt  audi¬ 
ences.  Silent  films  began  to  be  used  in  schools  as  early 
as  1910  (Saettler,  1990).  In  1912,  the  Lycee  Hoche  de 
Versailles  in  France  had  gained  international  notice  for 
its  exemplary  incorporation  of  films  into  science  teach¬ 
ing.  By  the  1920s,  many  different  individuals,  compa¬ 
nies,  nonprofit  organizations,  and  government  agencies 
attempted  to  supplement  the  existing  supply  of  theatri¬ 
cal  films  and  newsreels.  Educators  could  find  many 
types  of  films  to  use:  theatrical  films  edited  for  special 
purposes,  industrial  films,  government  films,  and  a 
smaller  number  of  films  produced  specifically  for  the 
classroom.  Schools  that  wanted  to  be  viewed  as  pro¬ 
gressive  rushed  to  build  collections  of  films.  Despite 
the  marginal  value  of  many  of  the  available  films,  inter¬ 
est  and  usage  continued  to  grow,  and  by  the  end  of  the 
1920s  many  education  agencies  had  units  devoted  to 
film  or  visual  education,  and  thick  catalogs  documented 
the  thousands  of  films  available  to  educators. 


VISUAL  INSTRUCTION  MOVEMENT 

Enthusiasm  for  the  use  of  still  pictures  and  motion 
pictures  as  educational  resources  grew  to  become  the 
Visual  Instruction  movement,  an  increasingly  orga¬ 
nized  effort  by  enthusiasts  to  promote  wider  use  of 
these  new  technologies.  This  movement  is  regarded  as 
the  first  paradigm  in  which  the  field  found  its  identity. 
Under  this  paradigm,  advocates  sought  to  make  visual 
materials  widely  available  throughout  school  districts, 
postsecondary  institutions,  and  adult  education  institu¬ 
tions.  At  first  these  resources  were  included  in  the  col¬ 
lections  of  educational  museums;  the  first  in  the  United 
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States  was  established  in  St.  Louis,  Missouri,  in  1905, 
based  on  exhibits  saved  from  the  World’s  Fair  held  in 
that  city  in  1904  (Saettler,  1990).  Later,  collections  of 
visual  media  were  gathered  into  visual  resources  cen¬ 
ters  of  their  own,  and  the  leaders  of  the  emerging  held 
of  visual  instruction  were  the  directors  of  these  centers. 

The  earliest  formal  research  on  educational  appli¬ 
cations  of  media  was  Lashley  and  Watson’s  program 
of  studies  on  the  use  of  World  War  I  military  training 
films  on  the  prevention  of  venereal  disease  with  civil¬ 
ian  audiences  (Lashley  and  Watson,  1921).  An  early 
large-scale  effort  to  design  and  produce  a  set  of  films 
specifically  for  schools  was  the  Chronicles  of  America 
Photoplays,  produced  by  Yale  University  in  the  late 
1920s.  Knowlton  and  Tilton  (1929)  studied  the  use  of 
these  history  films  in  seventh-grade  classrooms.  One 
of  their  major  conclusions  was  that  the  educational 
value  of  such  films  lay  not  only  in  the  quality  of  the 
materials  but  also  in  how  well  teachers  used  them.  This 
finding,  that  the  instructional  value  of  any  media  prod¬ 
uct  is  determined  largely  by  how  it  is  used,  would  be 
rediscovered  by  each  succeeding  generation  with  its 
new  media:  radio,  then  television,  then  programmed 
instruction,  then  computer-based  instruction,  and  now 
Internet -based  learning  environments. 

The  making  of  films  for  educational  use  in  the 
early  years  was  not  explicitly  guided  by  pedagogical 
theories.  Producers  generally  chose  subjects  that  were 
visual  in  nature  then  applied  the  methodology  of  one 
of  the  existing  him  genres:  drama,  travelogue,  docu¬ 
mentary,  ethnography,  historical  reenactment,  nature 
study,  scientific  experiment  or  demonstration,  lecture, 
procedural  guide,  and  the  like. 

During  the  1920s,  visual  instruction  enthusiasts 
formed  a  number  of  organizations.  In  1923,  one  of 
them,  the  National  Education  Association’s  Depart¬ 
ment  of  Visual  Instruction  (DVI),  emerged  to  become 
the  preeminent  organization  of  professionals  con¬ 
cerned  with  the  use  of  visual  media  to  improve  instruc¬ 
tion.  The  name  changed  to  Department  of  Audio- 
Visual  Instruction  (DAVI)  in  1947  as  its  boundaries 
expanded  to  include  auditory  media:  sound  films  and 
various  forms  of  recorded  sound,  beginning  with  pho¬ 
nographs  and  later  including  radio  broadcasting,  sound 
filmstrips,  and  audiocassettes. 

AUDIO-VISUAL  INSTRUCTION 

The  phonograph  record,  introduced  in  1910,  was  the 
first  widely  available  format  for  recorded  sound  and 
was  used  almost  exclusively  for  music .  Although  mag¬ 
netic  tape  displaced  the  phonograph  for  recording  pur¬ 
poses  in  the  1950s,  vinyl  records  remain  in  use  into 


the  21st  century.  As  soon  as  the  phonograph  was 
invented,  him  producers  tried  various  methods  of  using 
this  new  technology  to  add  sound  to  motion  pictures, 
but  in  the  late  1920s  the  technique  of  adding  an  optical 
sound  track  to  the  him  itself  became  the  preferred 
format  for  sound  films.  Interestingly,  there  was  con¬ 
siderable  resistance  to  sound  films  in  the  education 
community.  Some  methodologists  felt  that  the  practice 
of  having  the  classroom  teacher  add  narration  to  silent 
films  added  a  level  of  customization  and  personaliza¬ 
tion  to  him  showings.  Administrators  worried  about 
their  installed  base,  the  large  investment  they  had  made 
in  silent  him  projectors.  As  late  as  1936,  a  survey 
showed  that  schools  owned  ten  times  more  silent  him 
projectors  than  sound  him  projectors  (Saettler,  1990). 
The  slide  format  had  become  standardized  at  the  2x2- 
inch  frame  size,  using  35-mm  him,  which  was  also 
used  for  the  hlmstrip,  which  later  became  the  most 
popular  format  for  commercially  produced  audio¬ 
visual  materials.  Audio  resources  were  added  to  the 
growing  base  of  visual  resources.  By  the  1930s, 
schools  maintained  equipment  pools  that  contained  (in 
order  of  frequency):  lantern  slide  projectors,  radio 
receivers,  16-mm  silent  him  projectors,  35-mm  silent 
him  projectors,  hlmstrip  projectors,  opaque  projectors, 
micro-slide  projectors,  16-mm  sound  him  projectors, 
and  35-mm  sound  him  projectors  (Saettler,  1990). 

EDUCATIONAL  RADIO 

Initiation  of  Radio  Services 

In  the  1920s  and  1930s,  broadcast  radio  became  the 
prime  mass  communication  medium  around  the  world. 
In  most  countries,  broadcasting  facilities  were  directly 
managed  by  the  government,  although  after  the  found¬ 
ing  of  the  British  Broadcasting  Corporation  (BBC)  in 
1927,  many  countries  (such  as  Japan’s  NHK  and  Can¬ 
ada’s  CBC)  followed  its  model  of  a  quasi-autonomous 
public  corporation.  Providing  cultural  and  educational 
programming  was  assumed  to  be  a  primary  responsi¬ 
bility  of  these  organizations;  such  programs  were  often 
among  the  hrst  to  be  broadcast.  The  hrst  school  pro¬ 
grams  in  Canada  began  in  1925,  in  England  in  1926. 
By  the  mid- 1930s,  there  were  school  broadcasting  ser¬ 
vices  in  virtually  every  European  country  as  well  as  in 
Australia,  Japan,  South  Africa,  and  India. 

Educational  Radio  in  Japan 

Japan’s  NHK  initiated  nationwide  school  broadcasts 
in  1935  and  soon  developed  a  policy  of  programs  to 
complement  the  school  curriculum,  “to  hll  in  areas 
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unreached  by  the  conventional  teaching”  (De  Vera, 
1967,  p.  23).  Following  reorganization  after  World 
War  II,  NHK  went  on  to  become  an  international 
exemplar  for  its  ambitious  and  high-quality  pro¬ 
gramming  in  radio  and  later  (beginning  in  1953)  in 
television. 

Educational  Radio  in  North  America 

In  Canada,  the  first  large-scale  school  broadcasts  were 
actually  offered  by  the  Canadian  National  Railways 
(CNR)  system.  This  radio  service  was  established  to 
entertain  rail  travelers,  but  it  also  reached  the  towns 
and  cities  along  its  route,  and  the  CNR  broadcasters 
were  quick  to  provide  programming  that  would  appeal 
to  school  audiences.  The  CNR  school  service  built  a 
loyal  audience  by  deliberately  building  participative 
activities  into  the  programs.  The  service  was  sub¬ 
sumed  into  the  CBC  in  1933.  In  the  early  1920s,  many 
American  universities  obtained  licenses  to  operate 
radio  stations ,  often  as  technical  experiments  in  elec¬ 
trical  engineering.  A  large  proportion  of  these  died 
out  in  competition  with  commercial  stations,  but  some 
put  down  roots.  The  operations  that  prospered  were 
the  ones  in  which  radio  played  an  integral  part  in  the 
university’s  mission— bringing  educational  opportu¬ 
nities  to  audiences  beyond  the  campus  (Wood  and 
Wylie,  1977). 

Educational  Radio  Programming 

By  the  mid-1930s,  many  Ameican  school  districts 
operated  radio  stations,  which  developed  sophisti¬ 
cated  educational  programming,  often  incorporating 
innovative  pedagogical  techniques.  At  the  Cleveland, 
Ohio,  Board  of  Education’s  radio  station,  WBOE,  in 
the  1930s  they  pretested  programs  by  creating  rough 
drafts  and  trying  them  out  with  student  audiences. 
This  practice  foreshadowed  the  later  notion  of  improv¬ 
ing  lessons  and  validating  their  worth  through  forma¬ 
tive  and  summative  evaluation  (Cambre,  1978).  Edu¬ 
cational  broadcasters  offered  programs  in  every 
conceivable  subject,  including  foreign  languages, 
health,  social  studies,  home  economics,  science, 
music,  art,  and  many  other  subjects.  BBC  program¬ 
mers  worked  closely  with  advisory  boards  of  teachers 
in  every  subject  area  to  find  niches  into  which  audio 
material  might  add  value  (Bailey,  1957).  They  reached 
thousands  of  schools  in  each  country;  for  example,  in 
1936  in  England  and  Wales  some  4600  schools  were 
registered  users  (Parker,  1939).  However,  in  the  Amer¬ 
icas  and  many  European  countries,  programming 
tended  to  be  what  Levenson  and  Stasheff  (1952) 
referred  to  as  “informally  educative”  rather  than 


directly  instructional.  Radio  services  had  difficulty 
playing  core  instructional  roles.  For  one  thing,  the 
advantage  of  broadcasting  is  its  coverage  of  a  broad 
area,  but  that  meant  crossing  school  district  and  even 
state  and  provincial  boundaries.  It  was  difficult  to 
create  any  lesson  that  would  meet  the  content,  scope, 
sequence,  and  timing  demands  of  multiple  schools 
across  multiple  jurisdictions .  For  another  thing ,  teach¬ 
ers,  the  gatekeepers  of  the  classroom,  were  reluctant 
to  turn  over  responsibility  for  core  subject  matter, 
sensing  that  it  would  threaten  their  authority.  This 
pattern  of  consigning  technology-based  programming 
to  a  supplementary  role  was  to  be  repeated  with  tele¬ 
vision,  programmed  instruction,  and  computer- 
assisted  instruction. 


EDUCATIONAL  MEDIA 
IN  WORLD  WAR  II 

During  World  War  II  both  the  Allies  and  Axis  powers 
used  motion  pictures  extensively  for  home-front  pro¬ 
paganda  purposes,  with  the  German  director  Leni  Rief- 
enstahl  setting  new  aesthetic  standards  with  psycho¬ 
logically  powerful  documentaries,  such  as  Triumph  of 
the  Will.  Such  films  provided  rich  material  for  a  gen¬ 
eration  of  researchers  in  psychology  and  media  studies 
in  the  United  States  and  Europe.  The  need  for  rapid 
mass  training  of  literally  millions  of  combatants  and 
industrial  workers  brought  films  to  the  forefront  of 
military  training.  The  British  and  American  armed 
forces  made  extensive  use  of  16-mm  films  for  training 
and  motivational  purposes,  but  the  U.S.  effort  was  the 
most  pervasive  of  any  of  the  combatant  nations. 
Between  1941  and  1945,  the  Division  of  Visual  Aids 
for  Military  Training— with  major  participation  by 
Hollywood  directors  and  actors  — produced  over  400 
sound  films  and  over  400  silent  filmstrips,  enabling  a 
military  mobilization  far  broader  and  faster  than  the 
Axis  strategists  had  expected  (Saettler,  1990). 

During  the  war,  as  films  were  being  produced  and 
used  in  training,  the  U.S.  Army  commissioned  a  series 
of  psychological  studies,  later  published  as  Experi¬ 
ments  on  Mass  Communication  (Hovland  et  al.,  1949), 
which  tested  hypotheses  about  various  filmic  tech¬ 
niques  and  their  instructional  effectiveness.  Because 
of  the  concentration  of  time,  money,  effort,  and 
research  expended  on  these  productions,  a  genre  of 
instructional  film  came  into  its  own.  New  filmic  con¬ 
ventions  were  established,  for  example,  showing  pro¬ 
cedural  tasks  from  the  performer’s  viewpoint  rather 
than  the  viewer’s  and  using  a  first-person  stream  of 
consciousness  narration  to  model  the  thought  process 
of  the  performer. 
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EDUCATIONAL  MEDIA 
IN  THE  POST-WAR  PERIOD 

Research  on  Media 

After  the  war,  instructional  film  research  continued 
under  U.S.  Navy  sponsorship  at  Pennsylvania  State 
University,  a  research  program  that  yielded  over  a  hun¬ 
dred  publications  (Hoban  and  Van  Ormer,  1970).  Some 
of  the  experiments  dealt  with  utilization  techniques, 
but  most  explored  presentation  variables,  such  as  cam¬ 
era  angles,  pacing,  narration,  music,  and  color  (Saet- 
tler,  1990).  The  U.S.  Air  Force  also  commissioned  a 
series  of  studies  in  the  early  1950s  that  explored  the 
possible  interactions  between  film  and  programmed 
instruction  techniques  and  broadly  examined  the  value 
of  learner  response  during  film  or  video  viewing. 
Research  within  the  field  received  a  major  stimulus 
with  the  founding  in  1953  of  the  journal  Audio-Visual 
Communication  Review  by  DAVI,  the  predecessor  of 
AECT.  The  National  Defense  Education  Act  passed  in 
1958  provided  a  flood  of  funding  in  the  United  States 
for  audio-visual  research  under  Title  VII. 

Basic  Research 

Most  of  the  basic  research  on  visual  and  auditory  per¬ 
ception  has  been  done  outside  the  field  of  educational 
technology.  The  most  relevant  strand  of  visual  learning 
research  began  in  Germany  with  Gestalt  psychology 
in  the  first  half  of  the  20th  century,  pioneered  by  Max 
Wertheimer  (1944)  and  elaborated  by  Kurt  Koffka  and 
Wolfgang  Kohler.  They  were  attempting  to  describe 
how  humans  and  other  primates  perceived  stimuli  and 
used  cognitive  processes  to  understand  and  solve  prob¬ 
lems.  Another  strand,  focusing  on  the  formation  of 
mental  models,  was  begun  by  Kenneth  Craik  (1943) 
in  England  and  elaborated  by  Johnson-Laird  (1983). 
Generalizations  gathered  from  these  sorts  of  basic 
research  were  compiled  by  Fleming  and  Levie  (1978) 
in  the  form  of  message  design  principles. 

Audio-Visual  Instruction  in  Practice 

The  period  between  World  War  II  and  the  advent  of 
personal  computers  in  1982  could  be  viewed  as  the 
audio-visual  instruction  period.  Immediately  after 
World  War  II,  educational  technology  practice 
revolved  around  the  media  formats  that  had  become 
widely  available  to  teachers  by  1946:  16-mm  films, 
35-mm  slide/filmstrip  projectors,  opaque  projectors, 
radio  receivers,  and  record  players.  These  formats  were 
owned  by  schools  at  the  rate  of  at  least  1  per  100 
teachers.  Television  receivers  reached  this  status  in 


1958  and  overhead  projectors  in  1960  (Finn  et  al., 
1962).  Magnetic  tape  recording  was  invented  in  Ger¬ 
many  in  1935  and  was  introduced  to  the  United  States 
by  servicemen  who  brought  back  recorders  after  the 
war.  By  1956,  reel-to-reel  tape  recorders  had  joined 
the  ranks  of  media  devices  found  in  mass  use  in 
schools  (Finn  et  al.,  1962).  Cassette  audio  recorders 
were  introduced  by  Philips  in  the  Netherlands  in  1962 
and  became  the  standard  audio  format  in  schools 
around  the  world  by  the  early  1970s. 

Meanwhile,  the  actual  rate  of  use  of  audio-visual 
media  by  K-12  teachers  during  this  era  would  have  to 
be  characterized  as  moderate.  Utilization  rates  were 
strongly  affected  by  accessibility.  Teachers  were  likely 
to  use  materials  that  were  stored  in  their  own  class¬ 
rooms,  somewhat  less  likely  to  use  those  housed  in  a 
center  in  their  building,  and  even  less  likely  to  use 
items,  such  as  16-mm  films,  that  had  to  be  delivered 
from  outside  the  building  on  a  scheduled  basis.  Sur¬ 
veys  in  the  1940s  and  1950s  in  the  United  States  indi¬ 
cated  that  about  40%  of  elementary  teachers  and  20% 
of  secondary  teachers  used  films  frequently.  Evidence 
from  various  sources  indicates  that  the  average  teacher 
used  about  one  film  per  month  (Cuban,  1986,  pp. 
14-18).  The  reasons  given  by  teachers  for  the  low  rate 
of  use  of  film  and  similar  media,  in  addition  to  acces¬ 
sibility,  included  lack  of  training  in  the  technology, 
unreliability  of  projection  equipment,  limited  school 
budgets  (for  rental  of  films  and  purchase  of  projectors), 
and  difficulty  integrating  the  material  into  the  curric¬ 
ulum.  Surveys  in  the  1990s  identified  the  identical 
barriers  to  teachers’  use  of  computers. 

The  animating  vision  of  the  audio-visual  paradigm, 
represented  by  Hoban  et  al.  (1937),  was  to  replace 
empty  verbalism  or  rote  memorization  with  meaning¬ 
ful  learning.  Dale  (1946),  an  early  advocate  of  rich 
learning  environments,  expanded  the  notion  of  visual 
instruction  by  proposing  in  his  Cone  of  Experience 
that  learning  experiences— including  direct  personal 
experiences,  field  trips,  and  dramatizations  as  well  as 
audio  and  visual  media— could  be  arrayed  in  a  spec¬ 
trum  from  concrete  to  abstract,  each  with  its  proper 
place  in  the  tool  kit. 


EDUCATIONAL  TELEVISION  (ETV) 

The  BBC  began  regular  television  broadcasts  in  1936, 
and  regular  programs  were  being  offered  in  the  United 
States,  Germany,  France,  and  the  Soviet  Union  before 
World  War  II,  which  brought  developments  to  a  stand¬ 
still.  After  the  war,  television  grew  rapidly;  for  exam¬ 
ple,  the  NBC  commercial  network  was  broadcasting 
by  1947,  and  NHK  began  regular  television  service  in 
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1953.  In  most  European  countries,  the  radio  formula 
was  carried  over  to  television,  with  the  state  broadcast¬ 
ing  agencies  expanding  into  this  new  medium,  funded 
by  license  fees,  and  continuing  their  tradition  of  bring¬ 
ing  cultural  and  educational  programming  to  the  pop¬ 
ulace. 

ETV  in  Europe 

The  BBC  began  school  broadcasts  in  1957;  by  1974, 
over  80%  of  all  schools  were  making  regular  use  of 
BBC  programs  (British  Information  Services,  1974), 
a  pattern  that  carried  on  into  the  21st  century.  Guided 
since  the  early  days  of  radio  by  a  School  Broadcasting 
Council  that  includes  strong  representation  of  teachers, 
programs  are  carefully  designed  to  be  integrated  into 
the  national  curriculum.  In  other  European  countries, 
the  general  pattern  is  for  the  state  television  corpora¬ 
tion  to  devote  a  small  percentage  of  its  broadcast  hours 
to  programming  aimed  at  in-school  audiences  and 
adult  education. 

ETV  in  North  America 

Like  the  United  Kingdom,  Canada  also  operates  a 
national  television  network,  the  CBC,  which  is  sup¬ 
ported  in  part  by  commercial  advertising.  The  CBC 
began  to  provide  school  broadcasts  when  television 
operations  commenced  in  1952.  They  continued  to 
offer  a  limited  schedule  of  in-school  programs 
throughout  the  1960s  and  1970s  as  the  various  prov¬ 
inces  gradually  undertook  their  own  program  produc¬ 
tion.  Just  as  in  the  United  States,  Canada’s  K-12  edu¬ 
cation  system  is  controlled  by  provincial  authorities 
rather  than  the  national  government,  and  by  the  mid- 
1960s  most  of  the  provinces  were  producing  in-school 
programs  tailored  to  their  specific  needs.  Beginning  in 
1970,  TVOntario,  a  public  noncommercial  network 
serving  Ontario,  offered  school-oriented  programs, 
some  of  which  attained  international  recognition  and 
distribution;  thus,  both  national  and  provincial  pro¬ 
grams  are  available  for  school  use. 

During  the  1950s,  dozens  of  noncommercial  tele¬ 
vision  licenses  were  granted  to  universities  and  com¬ 
munity  groups  in  the  United  States,  and  educational 
television  programs  began  to  be  beamed  to  school 
and  college  audiences  (the  first  being  KUHT  at  the 
University  of  Houston  in  1953)  and  adults  and  chil¬ 
dren  at  home.  Many  of  the  same  parties  that  had 
experimented  with  radio  also  did  so  with  television, 
essentially  replaying  the  radio  scenario.  Because  this 
was  a  period  of  rapid  school  population  growth  there 
was  a  general  shortage  of  qualified  teachers.  Televi¬ 
sion  was  seen  by  some  as  a  way  to  reduce  the  need 


for  additional  teachers  by  replacing  the  presentation 
function  with  broadcast  lessons. 

In  the  late  1950s  and  through  the  1960s  in  the 
United  States,  programs  were  distributed  on  a  regional 
basis,  such  as  by  the  Eastern  Educational  Network  and 
the  Midwest  Program  of  Airborne  Television  Instruc¬ 
tion  (MPATI,  a  precursor  of  satellite  broadcasting) ,  and 
a  few  on  a  national  basis,  such  as  Continental  Class¬ 
room.  During  this  period,  the  Ford  Foundation  and  the 
federal  government  were  subsidizing  the  expansion  of 
television  in  higher  education  through  grants  for 
closed-circuit  TV  construction  and  program  produc¬ 
tion.  By  the  end  of  the  decade  of  the  1960s,  tens  of 
millions  of  school  and  college  students  were  receiving 
televised  instruction  on  a  daily  basis.  After  the  popu¬ 
larization  of  videotape  recording  (later,  videocassette 
recording),  ETV  programming  was  increasingly  cre¬ 
ated  and  used  as  off-the-shelf  packaged  units  rather 
than  being  received  through  broadcasting. 

ETV  Programming 

As  with  educational  films,  ETV  programs  tended  to 
emulate  the  familiar  genres:  lecture,  demonstration, 
voice-over  visualization,  interview,  panel  discussion, 
dramatization,  field  trip,  or  documentary  (Wood  and 
Wylie,  1977).  The  production  processes  were  compa¬ 
rable  to  those  used  in  commercial  radio  and  television: 
“We  borrowed  from  commercial  television  certain 
ideas  about  what  constitutes  a  program,  and  we  have 
not  shaken  free  from  these  concepts”  (Suchman,  1966, 
p.  30).  American  ETV  presentations ,  particularly  those 
beamed  to  college  audiences,  tended  to  be  more  verbal 
(featuring  the  so-called  talking  heads)  than  European 
productions  (Tanner  and  Woerdehoff ,  1964).  European 
programs,  particularly  those  of  the  BBC,  were  notable 
for  their  emphasis  on  visualization.  The  BBC  collec¬ 
tion  became  a  major  international  archive  of  exemplary 
programming  that  was  drawn  upon  by  producers  from 
around  the  globe. 

A  break  from  this  expository  pattern  began  in  the 
1960s  under  the  influence  of  the  so-called  Cognitive 
Revolution,  which  suggested  that  television  should 
be  participative  rather  than  passive.  It  should  ask 
questions,  pose  challenging  problems,  and  spark  dis¬ 
cussion  and  search  for  answers.  In  short,  it  should 
trigger  inquiry  (McBride,  1966).  The  discovery 
learning  movement  eventually  led  to  the  production 
of  a  number  of  educational  television  series,  espe¬ 
cially  in  science  and  social  studies,  that  portrayed 
problematic  situations  and  invited  learners  to  discuss 
them.  The  Jasper  Woodbury  Problem  Solving  Series 
in  the  1990s  represented  the  culmination  of  this 
movement. 
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ETV  in  Developing  Countries 

As  television  was  later  in  coming  to  the  less  industri¬ 
alized  countries,  so  were  educational  applications.  In 
many  countries  educational  television  came  with  the 
financial  and  technical  support  of  industrialized  coun¬ 
tries,  intending  to  help  expand  educational  opportuni¬ 
ties  as  part  of  nation  building;  for  example,  in  1961 
UNESCO  and  the  Ford  Foundation  established  a  pilot 
project  in  Delhi,  India,  to  offer  televised  physics, 
chemistry,  and  English  lessons  to  secondary  students 
in  that  city  (Mohanty,  1984).  In  1966,  a  project  was 
initiated  for  communicating  agricultural  information 
to  farmers  in  some  80  villages  outside  of  Delhi;  the 
programs  were  viewed  communally  and  were  followed 
by  group  discussion. 

This  pattern  of  urban  in-school  programs  and  rural 
agricultural  development  support  was  followed  in 
many  other  developing  countries.  During  the  1960s 
and  1970s,  educational  television  projects  were  under¬ 
taken  in  more  than  a  dozen  countries  in  Fatin  America 
(e.g.,  El  Salvador  and  Colombia)  and  like  numbers  in 
Africa  (e.g..  Ivory  Coast  and  Niger)  and  Asia  (e.g.. 
South  Korea  and  India).  In  Oceania,  the  entire  educa¬ 
tional  system  of  American  Samoa  was  restructured 
around  television  in  the  1960s. 

In  many  cases,  these  projects  were  not  intended  to 
be  permanent;  in  any  event,  most  were  not  sustained. 
Tiffin’s  (1978)  system  analysis  of  ETV  projects  in 
Latin  America  indicated  that  although  the  lessons 
themselves  were  educationally  effective  the  overall 
projects  suffered  from  systemic  problems.  Clayton’s 
(1979)  analysis  echoed  these  findings,  noting  that  sys¬ 
tems  in  which  major  components  are  absent  or  dys¬ 
functional  tended  to  perish. 

In  the  post-colonial  era,  ETV  was  viewed  as  a 
means  of  expanding  the  reach  of  disadvantaged  edu¬ 
cation  systems  while  improving  the  quality  of  the  edu¬ 
cation  that  was  offered.  The  evidence  indicated  that 
from  a  strictly  economic  standpoint  these  early 
projects  were  difficult  to  justify.  Reform  based  on  tele¬ 
vision  may  be  a  faster  means  of  changing  the  curric¬ 
ulum  and  improving  teaching  methods  but  it  is  also 
more  expensive  in  these  settings  and  often  not  locally 
sustainable  (Camoy,  1975). 


THE  COMMUNICATION  PARADIGM 
Information  Theory 

During  the  later  days  of  educational  radio  and  the 
earlier  days  of  educational  television,  communication 
theory  became  a  dominant  paradigm  both  in  the  phys¬ 


ical  and  social  sciences.  Flowing  from  Shannon  and 
Weaver’s  (Shannon,  1949)  information  theory,  through 
Wiener’s  (1950)  cybernetics  and  Berio’s  (1960)  Pro¬ 
cess  of  Communication  model,  thinkers  in  educational 
technology  were  viewing  teaching/learning  problems 
as  communication  problems.  Improvement  of  commu¬ 
nication  depended  on  detecting  where  the  weak  points 
in  the  process  were  and  ameliorating  them:  choosing 
a  more  visual  medium,  building  more  redundancy  into 
the  message,  matching  the  receiver’s  language  capa¬ 
bility  better,  providing  the  sender  with  better  feedback 
about  the  receiver’s  response,  and  the  like. 

Semantics 

During  the  1940s  and  1950s,  theories  of  communica¬ 
tion  not  only  sparked  the  emergence  of  information 
science  but  also  attracted  attention  in  the  social  sci¬ 
ences.  General  Semantics,  conceived  by  Korzybski 
(1933)  and  interpreted  and  popularized  by  Hayakawa 
(1941),  offered  a  new  way  of  studying  the  meanings 
evoked  when  humans  communicated  through  various 
media.  It  added  the  human  dimension  to  the  technical 
process  of  communication  addressed  in  other  commu¬ 
nication  theories. 

A  New  Paradigm  for 
Audio-Visual  Education 

The  communication  perspective  became  a  new  para¬ 
digm  for  defining  the  audio-visual  instruction  field.  It 
was  embraced  wholeheartedly  by  a  segment  of  the 
field;  for  example  ,  the  name  of  the  academic  program 
at  Syracuse  University  changed  from  Audio-Visual 
Education  to  Educational  Media  to  Instructional  Com¬ 
munications  in  the  mid-1960s.  The  first  formal  defini¬ 
tion  of  the  field  in  1963  used  the  term  audiovisual 
communications  as  the  central  concept.  When  the  time 
came  to  change  the  name  of  DAVI  in  1971 ,  there  was 
nearly  equal  support  for  communications  and  technol¬ 
ogy  as  the  key  terms,  so  both  were  incorporated  into 
the  new  name:  Association  for  Educational  Commu¬ 
nications  and  Technology  (AECT). 

RADICAL  BEHAVIORISM 
Application  to  Instruction 

The  term  behaviorism  refers  collectively  to  several 
related  but  different  theories  in  psychology.  One  of 
them,  radical  behaviorism,  has  had  the  greatest  practical 
impact  on  educational  technology  due  to  the  application 
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of  its  primary  technique,  operant  conditioning,  to  teach¬ 
ing-learning  problems  (Burton  et  al.,  2004).  As  dis¬ 
cussed  in  the  chapter  by  Lockee  and  colleagues  in  the 
Technologies  part  of  this  handbook,  B.F.  Skinner’s  anal¬ 
ysis  of  the  problems  of  group-based  traditional  instruc¬ 
tion  led  him  to  the  invention  of  a  mechanical  device  for 
applying  operant  conditioning  to  academic  instruction 
(Skinner,  1954).  Referred  to  as  a  teaching  machine,  the 
device  gained  national  attention.  The  arrangement  of 
stimuli,  responses,  and  reinforcers  in  teaching  machines 
became  known  as  programmed  instruction,  and  pro¬ 
grammed  instruction  lessons  in  book  format  were  pub¬ 
lished  in  great  profusion  in  the  1960s. 

Impact  of  Teaching  Machines 

The  Department  of  Audiovisual  Instruction  (the 
hyphen  between  audio  and  visual  was  dropped  in 
1960)  joined  the  new  programmed  instruction  move¬ 
ment  by  publishing  Teaching  Machines  and  Pro¬ 
grammed  Learning:  A  Source  Book  (Lumsdaine  and 
Glaser,  1960).  The  1959  DAVI  convention  program 
had  a  single  paper  devoted  to  programmed  instruction, 
but  there  was  a  major  session  in  1960  entitled  “Pro¬ 
grammed  Instructional  Materials  for  Use  in  Teaching 
Machines.”  This  title  gives  a  clue  to  the  link  between 
audio-visual  administrators  and  programmed  instruc¬ 
tion:  the  machines  that  were  initially  used  to  deliver 
the  programmed  lessons.  When  schools  and  colleges 
acquired  teaching  machines  someone  had  to  take  care 
of  them:  the  audio-visual  coordinator!  The  primacy  of 
the  machine  was  indicated  by  the  name  that  marked 
this  special-interest  group  at  the  next  several  DAVI 
conventions:  the  Teaching  Machine  Group. 

Emergence  of  Educational 
Technology  Paradigm 

Gradually  the  emphasis  shifted  to  the  process  of 
designing  self-instructional  systems.  This  design  pro¬ 
cess  dovetailed  with  the  notion  promoted  earlier  by 
James  D.  Finn  that  instructional  technology  could  be 
viewed  as  a  way  of  thinking  about  instruction,  not  just 
a  conglomeration  of  devices.  Thereafter,  technology 
began  to  take  on  the  dual  meanings  of  application  of 
scientific  thinking  and  the  various  communications 
media  and  devices  (parallel  to  the  distinction  between 
hard  and  soft  technologies  found  in  the  second  edition 
of  this  handbook).  Further,  by  the  mid-1960s,  Skinner 
also  came  to  view  programmed  instruction  as  a  prac¬ 
tical  application  of  scientific  knowledge  to  education, 
and  he  referred  to  his  instructional  strategies  as  a  tech¬ 
nology  of  teaching  (Skinner,  1965,  1968).  Other 
authors  converted  this  term  to  educational  technology, 


an  early  example  is  Educational  Technology :  Readings 
in  Programmed  Instruction  (DeCecco,  1964). 

Between  1960  and  1970,  the  research  focus  of 
what  had  been  the  audio-visual  education  field  shifted 
sharply  toward  work  on  teaching  machines  and  pro¬ 
grammed  instruction,  prompting  the  change  of  the 
name  of  the  field  from  audio-visual  education  to  edu¬ 
cational  technology.  Torkelson  (1977)  examined  the 
contents  of  articles  published  in  AV  Communication 
Review  between  1953  and  1977  and  found  that  the 
topics  of  teaching  machines  and  programmed  instruc¬ 
tion  dominated  the  journal  in  the  1960s.  In  fact, 
between  1963  and  1967,  these  topics  represented  a 
plurality  of  all  articles  published  in  that  journal. 

This  reorientation  of  the  field  can  be  seen  as  a 
major  paradigm  shift,  from  the  creation  and  use  of 
audio-visual  media  or  the  communication  of  messages 
to  the  design  of  learning  environments  according  a 
specific  set  of  psychological  specifications.  The  dom¬ 
inant  psychological  theories  would  change  over  time, 
but  the  role  of  applied  psychologist  would  remain  at 
the  core. 

Behavioral  Technologies 

Research  and  development  in  behaviorism  led  to  other 
innovations  such  as  programmed  tutoring.  Direct 
Instruction,  and  Personalized  System  of  Instruction, 
which  are  discussed  at  greater  length  in  the  chapter  by 
Lockee  and  colleagues  in  Part  III.  These  technologies 
established  an  enviable  record  when  compared  with 
so-called  conventional  instruction  in  experiments  in 
which  paper-and-pencil  tests  were  used  as  the  measure 
of  learner  achievement  (Lockee  et  al.,  2004).  As  com¬ 
munication  technology  advanced,  these  frameworks 
were  incorporated  in  mechanical,  electromechanical, 
and  ultimately  digital  formats,  such  as  computer- 
assisted  instruction  and  online  distance  education. 


SYSTEMS  APPROACH  TO 
INSTRUCTIONAL  DESIGN 

Evolution  of  Systems  Approach 

The  essence  of  the  systems  approach  is  to  subdivide 
the  instructional  planning  process  into  steps,  to  arrange 
those  steps  in  logical  order,  then  to  use  the  output  of 
each  step  as  the  input  of  the  next.  The  systems 
approach  traces  its  origins  to  concepts  that  emerged 
from  military  research  during  World  War  II.  An  ana¬ 
lytical  technique  that  grew  out  of  submarine  hunting 
was  called  operations  research,  in  which  computers 
were  used  to  make  the  calculations  required.  After  the 
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war,  this  approach  to  analyzing,  creating,  and  manag¬ 
ing  man/machine  operations,  now  referred  to  as  the 
systems  approach ,  was  applied  to  the  development  of 
training  materials  and  programs. 

During  the  post-war  period,  each  of  the  U.S.  mili¬ 
tary  services  developed  its  own  model  for  training 
development,  and  all  of  them  were  based  on  the  systems 
approach,  a  soft  science  version  of  systems  analysis, 
itself  an  offshoot  of  operations  research.  The  systems 
approach  was  viewed  in  the  military  as  a  paradigm  for 
combining  the  human  element  with  the  machine  ele¬ 
ments  in  man-machine  systems,  an  antidote  to  purely 
mechanistic  thinking.  From  the  entry  of  the  systems 
approach  into  the  field  of  educational  technology,  it  was 
recognized  by  its  advocates  as  a  loose  set  of  guidelines 
that  were  applicable  to  the  complex  problems  of  human 
learning  only  by  analogy  and  not  the  sort  of  completely 
deterministic  and  tightly  controlled  methodology 
described  by  some  of  its  detractors. 

The  concept  of  systems  approach  probably  was 
introduced  to  educational  technology  leaders  in  the 
United  States  by  Charles  F.  Hoban  in  his  keynote 
address,  “A  Systems  Approach  to  Audio-Visual  Com¬ 
munication,”  presented  at  the  second  Lake  Okoboji 
leadership  conference  in  1956  (Noel  and  Noel,  1965). 
The  conference  spotlight  coincided  with  a  series  of 
articles  by  James  D.  Finn  published  around  the  same 
time.  Together,  they  helped  create  momentum  behind 
the  idea  of  the  systems  approach,  which  eventually 
became  a  hallmark  of  the  field. 

Instructional  Systems 
Development  Models 

During  the  1960s,  the  systems  approach  began  to  appear 
in  procedural  models  of  Instructional  Systems  Develop¬ 
ment  (ISD)  in  American  higher  education.  Barson’s 
( 1967)  Instructional  Systems  Development  project,  con¬ 
ducted  at  Michigan  State  University  and  three  other 
universities  between  1961  and  1965,  produced  an  influ¬ 
ential  model  and  a  set  of  heuristic  guidelines  for  devel¬ 
opers.  During  this  same  period,  Leonard  Silvern  at  the 
University  of  Southern  California  began  offering  the 
first  course  in  applying  the  systems  approach  to  instruc¬ 
tion  (Designing  Instructional  Systems),  which  was 
based  on  his  military  and  aerospace  experience  .  He  also 
produced  a  detailed  procedural  model  that  influenced 
later  model  builders  (Silvern,  1965). 

A  Model  for  Schools 

These  early  activities  in  the  consortium  that  included 
Syracuse,  Michigan  State,  U.S.  International  Univer¬ 
sity,  and  the  University  of  Southern  California  (later 


joined  by  Indiana  University)  culminated  in  a  joint 
project,  known  as  the  Instructional  Development  Insti¬ 
tute  (ID I).  The  IDI  was  a  packaged  training  program 
on  instructional  development  for  teachers,  and 
between  1971  and  1977  it  was  offered  to  hundreds  of 
groups  of  educators.  Because  it  was  usually  conducted 
by  faculty  and  graduate  students  from  nearby  univer¬ 
sities,  the  IDI  became  an  extremely  influential  vehicle 
for  disseminating  ideas  about  the  ISD  process  among 
educational  technology  faculty  and  students  across  the 
United  States. 

A  Model  for  the  Military  Services 

The  Center  for  Performance  Technology  at  Florida 
State  University  was  selected  in  1973  by  the  U.S. 
Department  of  Defense  to  develop  procedures  to  sub¬ 
stantially  improve  Army  training.  As  recounted  by 
Branson  (1978),  the  ISD  procedures  developed  for 
the  Army  evolved  into  a  model,  the  Interservice  Pro¬ 
cedures  for  Instructional  Systems  Development 
(IPISD),  that  was  adopted  by  the  Army,  Navy,  Air 
Force,  and  Marines.  The  detailed  procedures  clus¬ 
tered  around  five  major  functions:  analyze,  design, 
develop,  implement,  and  control.  The  IPISD  model 
eventually  had  enormous  influence  in  military  and 
industrial  training  because  its  use  was  mandated  not 
only  in  all  of  the  U.S.  armed  services  but  also  among 
all  defense  contractors. 

A  Generic  ISD  Model 

The  1980s  brought  a  proliferation  of  ISD  models  for 
education  and  training.  They  differed  in  details  but 
typically  adhered  to  the  common  conceptual  frame¬ 
work  of  analyze,  design,  develop,  implement,  and  eval¬ 
uate.  This  conceptual  framework  came  to  be  called  by 
its  acronym,  ADDIE.  During  the  1970s  and  1980s, 
advocates  for  the  systems  approach  attempted  to  pro¬ 
mote  its  use  in  K-12  and  higher  education.  These 
efforts  were  largely  unsuccessful,  possibly  for  reasons 
related  to  the  social  and  economic  dynamics  of  these 
institutions;  however,  ISD  was  welcomed  in  corporate 
and  military  training,  where  it  became  the  reigning 
paradigm  for  the  next  20  years  as  a  way  to  standardize 
design  practices  and  make  training  more  efficient  and 
effective. 

ISD  as  a  Paradigm  Shift 

The  ISD  movement  can  be  viewed  as  another  para¬ 
digm  shift  in  the  history  of  educational  technology. 
By  the  end  of  the  1980s,  skill  in  instructional  design 
was  viewed  as  the  core  competency  of  the  educational 
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technology  professional.  By  contrast,  the  develop¬ 
ment  and  production  of  audio-visual  materials  became 
a  niche  specialization,  one  that  was  often  outsourced. 

Critical  Questioning  of  ISD 

By  the  late  1990s,  however,  an  accumulation  of  pres¬ 
sures— new  digital  capabilities,  intense  cost  competi¬ 
tion  and  the  need  to  reduce  human  resources  costs,  and 
the  increasing  pace  of  change— led  to  increasingly  crit¬ 
ical  questioning  of  the  ISD  orthodoxy  (discussed  from 
an  international  perspective  in  Tennyson  et  al.,  1997). 
In  particular,  the  design  of  more  complex  computer- 
based  learning  environments  in  which  learners  are 
expected  to  take  the  initiative  in  pursuing  knowledge 
and  to  collaborate  with  others  in  doing  so  challenges 
conventional  ISD  procedures  (Hakkinen,  2002).  Low- 
yck  and  Poysa  (2001),  among  others,  have  called  for 
new  models  that  emphasize  co-construction  of  knowl¬ 
edge  and  indeed  co-design  of  the  learning  environment. 

ADVENT  OF  COMPUTERS 
IN  EDUCATION 

Mainframe  Era 

The  first  attempts  to  use  computers  to  present  and  con¬ 
trol  instruction  began  in  the  early  1960s  before  the 
microprocessor,  when  mainframe  computers  used 
punched  cards  for  input.  The  early  experiments  in  com¬ 
puter-assisted  instruction  (CAI)  began  just  at  the  time 
that  programmed  instruction  was  at  its  peak,  so  many 
of  the  early  CAI  programs  followed  a  drill-and-practice 
or  tutorial  format.  A  correct  response  was  confirmed, 
while  an  incorrect  response  might  branch  the  learner 
to  a  remedial  sequence  or  an  easier  question.  Beginning 
in  the  mid-1960s,  the  CAI  research  and  development 
program  at  Stanford  University,  later  the  Computer 
Curriculum  Corporation,  created  successful  drill-and- 
practice  materials  in  mathematics  and  reading,  later 
adding  foreign  languages  (Saettler,  1990). 

Minicomputer  Era 

More  innovative  and  more  learner-centered  programs 
were  developed  in  the  TICCIT  project  at  Brigham 
Young  University  in  the  1970s  after  the  introduction  of 
the  microprocessor  and  the  proliferation  of  minicom¬ 
puters.  These  sophisticated  programs  yielded  successful 
programs  in  mathematics  and  English  composition; 
however,  both  the  Stanford  and  TICCIT  programs  failed 
to  gain  major  adoption  in  their  intended  sectors:  K-12 
and  community  college  education  (Saettler,  1990). 


The  PLATO  project  at  the  University  of  Illinois 
began  in  1961  and  was  aimed  at  producing  cost-effi¬ 
cient  instruction  using  networked  inexpensive  termi¬ 
nals  and  a  simplified  programming  language  for 
instruction,  TUTOR  (Saettler,  1990).  Most  of  the  early 
programs  were  basically  drill-and-practice  with  some 
degree  of  branching,  but  a  wide  variety  of  subject 
matter  was  developed  at  the  college  level.  Over  time, 
terminals  at  outlying  universities  were  connected  to 
the  central  mainframe  in  a  timesharing  system,  grow¬ 
ing  to  hundreds  of  sites  and  thousands  of  hours  of 
material  available  across  the  college  curriculum.  As 
software  development  continued,  many  innovative  dis¬ 
play  systems  evolved,  including  a  graphical  Web 
browser.  With  experience  and  with  more  capable  hard¬ 
ware,  more  varied  sorts  of  instructional  strategies 
became  possible,  including  laboratory  and  discovery 
oriented  methods. 

The  PLATO  system  pioneered  online  forums  and 
message  boards,  e-mail,  chat  rooms,  instant  messag¬ 
ing,  remote  screen  sharing,  and  multiplayer  games, 
leading  to  the  emergence  of  what  was  perhaps  the 
world’s  first  online  community  (Woolley,  1994).  It 
continued  to  grow  and  evolve  right  through  the  early 
2000s,  sparking  the  expansion  of  local  CAI  develop¬ 
ment  and  finding  a  niche  in  military  and  vocational 
education. 


COGNITIVIST  AND 
CONSTRUCTIVIST  THEORIES 

Cognitivism 

Like  behaviorism,  cognitivism  is  a  label  for  a  variety 
of  diverse  theories  in  psychology  that  endeavor  to 
explain  internal  mental  functions  through  scientific 
methods.  Lrom  this  perspective,  learners  use  their 
memory  and  thought  processes  to  generate  strategies 
as  well  as  store  and  manipulate  mental  representations 
and  ideas.  The  Scottish  psychologist  Kenneth  Craik 
(1943)  theorized  that  thinking  and  reasoning  take  place 
through  the  internal  manipulation  of  mental  models. 
A  generation  later,  Johnson-Laird  (1983)  built  on 
Craik’s  foundation,  elaborating  a  theory  that  when 
people  participate  in  discourse  they  construct  a  mental 
model  of  the  situation  being  discussed.  Other  theories 
that  would  later  become  very  influential  were  being 
developed  in  the  1920s  and  1930s  by  Jean  Piaget  in 
Switzerland  and  Lev  Vygotsky  in  Russia,  but  these  did 
not  have  significant  impact  on  American  educational 
psychology  until  translations  were  widely  circulated 
in  the  1960s.  Cognitive  theories  gained  momentum  in 
the  United  States  with  the  publication  of  Jerome 
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Bruner’s  The  Process  of  Education  in  1960,  the  dis¬ 
semination  of  Piaget’s  and  Vygotsky’s  works,  and  the 
emergence  of  information-processing  theory  (leading 
to  cognitive  load  theory)  in  the  1960s.  By  1970,  when 
the  journal  Cognitive  Psychology  was  bom,  the  cog¬ 
nitive  perspective  had  gained  not  only  legitimacy  but 
also  dominance. 

Constructivism 

A  perspective  on  learning  and  instruction  known  as 
constructivism  entered  the  vocabulary  of  the  educa¬ 
tional  technology  field  in  North  America  by  means  of 
a  provocative  presentation  by  David  Jonassen  at  the 
1990  meeting  of  Professors  of  Instructional  Design  and 
Technology  (PIDT),  a  presentation  later  recorded  and 
amplified  in  Jonassen’s  column  (1990)  in  Educational 
Technology  and  in  a  lead  article  in  Educational  Tech¬ 
nology  Research  and  Development  (1991).  He  chal¬ 
lenged  instructional  design  and  technology  people  to 
question  the  “objeetivist  epistemology”  underlying 
practice  in  the  field.  He  attributed  the  field’s  failure  to 
change  in  a  revolutionary  way  to  its  acceptance  of  this 
epistemology,  which  he  claimed  undergirded  both 
behaviorist  learning  theories  and  cognitivist  learning 
theories. 

Early  advocates  of  constructivism  used  the  term  as 
an  umbrella  term  for  a  wide  range  of  innovative 
instructional  methods  drawn  primarily  from  recent 
developments  in  cognitive  psychology  (see  Bednar  et 
al.,  1991;  Duffy  and  Cunningham,  1996;  Duffy  and 
Jonassen,  1992).  Piaget  and  Vygotsky  were  frequently 
cited  as  foundational  influences  on  the  development  of 
this  perspective.  Because  of  the  importance  of  social 
and  cultural  influences  in  Vygotsky’s  theory,  it  is 
termed  a  sociocultural  approach  to  learning  and  the 
branch  that  follows  this  theory  is  often  termed  social 
constructivism . 

The  difficulty  of  defining  a  canonical  version  of 
constructivism  is  discussed  in  Robinson  et  al.  (2008). 
In  view  of  these  many  differing  streams  of  thought, 
Driscoll  (2005)  concludes  “there  is  no  single  construc¬ 
tivist  theory  of  instruction”  (p.  386),  and  she  cites  as 
constructivism’s  common  denominator  the  assumption 
"that  knowledge  is  constructed  by  learners  as  they 
attempt  to  make  sense  of  their  experiences”  (p.  387). 
This  assumption  actually  overlaps  with  those  of  earlier 
cognitivist  theories  of  learning.  Where  constructivists 
(some  of  them)  seem  to  differ  from  cognitivists,  accord¬ 
ing  to  Driscoll,  is  that  they  argue  that  “knowledge  con¬ 
structions  do  not  necessarily  bear  any  correspondence 
to  external  reality”  (Driscoll,  2005,  p.  388).  This  sub¬ 
jectivism  aligns  with  the  epistemology  of  von  Glasers- 
feld  (1984, 1992),  the  founder  of  radical  constructivism. 


Constructivist  Movement 

Irrespective  of  ambiguity  about  its  theoretical  basis, 
the  constructivist  message  struck  a  chord  among  many 
academics  in  educational  technology.  For  those 
attracted  to  postmodernism,  constructivism  shared  a 
foundation  in  a  subjectivist  epistemology.  For  those 
leery  of  behaviorism’s  reductionist  tendencies,  con¬ 
structivism  projected  a  vision  of  holism.  It  hearkened 
back  to  the  era  in  which  Bruner  called  for  a  learner- 
centered  approach  to  meaningful  learning.  Perhaps 
most  importantly  for  those  in  educational  technology, 
it  coincided  with  the  new  capabilities  of  digital  media. 

Constructivism  recommended  instructional  strate¬ 
gies  that  followed  several  broad  principles,  according 
to  Driscoll  (2005,  pp.  394-395): 

1.  Embed  learning  in  complex,  realistic,  and 
relevant  environments . 

2.  Provide  for  social  negotiation  as  an  integral 
part  of  learning. 

3 .  Support  multiple  perspectives  and  the  use  of 
multiple  modes  of  representation. 

4.  Encourage  ownership  in  learning. 

5.  Nurture  self-awareness  of  the  knowledge 
construction  process. 

At  least  the  first  three  of  these  principles  lend  them¬ 
selves  better  to  technology-based  delivery  than  face- 
to-face  conventional  instruction.  First,  complex,  real¬ 
istic  environments  (or  microworlds)  can  be  created 
using  simulation  software.  Second,  e-mail,  chat  rooms, 
and  threaded  discussions  can  facilitate  social  negotia¬ 
tion.  Third,  the  World  Wide  Web  platform  enables 
designers  to  link  pictures  and  moving  animation  clips 
to  verbal  presentations,  all  of  which  can  be  navigated 
according  to  individual  needs  and  interests. 

Many  of  the  instructional  strategies  promoted  by 
constructivists  were  mentioned  in  the  early  article  by 
Bednar  et  al.  (1991):  situated  cognition  (associated  with 
cognitive  apprenticeship),  anchored  instruction,  prob¬ 
lem-based  learning,  and  collaborative  learning.  These 
strategies  were  subsequently  supported  by  the  Ameri¬ 
can  Psychological  Association’s  Learner-Centered 
Psychological  Principles  (APA,  1995),  an  authoritative 
position  paper  on  approaches  to  teaching  and  learning. 

Constructivism  as  a  New  Paradigm 

Advocates  on  the  American  side,  such  as  Thomas 
Duffy  and  David  Jonassen,  and  on  the  European  side, 
such  as  Joost  Lowyck,  converged  on  several  joint 
projects  (see  Duffy  et  al.,  1993),  contributing  to  a 
growing  wave  of  interest  on  both  sides  of  the  Atlantic . 
Having  gained  momentum  by  the  mid-1990s  as  the 
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constructivist  movement,  this  could  be  viewed  as 
another  paradigm  shift  in  the  identity  of  the  field.  From 
that  time  to  the  present,  the  conversation  has  centered 
on  using  the  tools  of  educational  technology  to  create 
learning  environments  suited  to  experiential  learning: 
WebQuests,  problem-based  learning,  microworlds, 
simulations  and  games,  blogs,  and  wikis. 

Emerging  Syntheses 

In  recent  years,  theorists  have  been  seeking  a  resolu¬ 
tion  to  the  paradigm  wars  among  the  competing  learn¬ 
ing  theories.  A  synthesis  offered  by  M.  David  Merrill 
(2002)  suggests  a  lesson  design  framework  incorpo¬ 
rating  behaviorist,  cognitivist,  and  constructivist  con¬ 
ceptions.  Merrill’s  (2002)  framework,  which  he  refers 
to  as  first  principles  of  instruction ,  proposes  four 
phases  to  the  instructional  process:  (1)  activation  of 
prior  experience,  (2)  demonstration  of  skills,  (3)  appli¬ 
cation  of  skills,  and  (4)  integration  of  these  skills  into 
real-world  activities,  with  all  four  phases  revolving 
around  a  problem  or  realistic  task.  Diana  Laurillard 
(2002)  of  the  British  Open  University  characterizes 
academic  instruction  as  essentially  a  conversation,  an 
iterative  dialog  between  a  teacher  and  a  student  that  is 
focused  on  a  particular  topic  or  goal.  She  proposes  that 
media  may  play  four  roles  in  the  instructional  dialog: 
discursive,  adaptive,  interactive,  or  reflective.  These 
roles  reflect  the  emphasis  of  the  different  psychologi¬ 
cal  perspectives:  hence,  Laurillard’s  model  also  may 
be  viewed  as  an  attempt  at  a  theoretical  synthesis. 


THE  DIGITAL  AGE 

Microcomputers  and 
Personal  Computers 

The  role  of  computers  in  education  began  to  change 
dramatically  with  the  development  of  microcomputers 
in  France  and  in  the  United  States  in  the  1960s  and 
1970s.  Microcomputers  became  increasingly  commer¬ 
cially  successful  after  the  introduction  of  new  models 
in  1977  by  Apple  and  RadioShack  and  in  1981  by 
IBM.  By  the  end  of  1982,  the  rapid  proliferation  of 
the  personal  computer  was  acknowledged  by  Time 
magazine  by  being  named  Machine  of  the  Year,  a  break 
from  the  magazine’s  tradition  of  Man  of  the  Year. 
Apple’s  Macintosh  model  in  1984  enticed  even  more 
novices  to  venture  into  the  computer  world.  Previously, 
students  encountered  mainframe  or  minicomputers  in 
labs,  where  they  served  as  tutors  that  typically  con¬ 
trolled  drill-and-practice  exercises.  Now,  both  students 
and  teachers  could  have  access  to  user-friendly  desktop 


computers  in  the  classroom  and  at  home  as  well  as 
productivity  tools  such  as  word-processing  programs 
for  writing,  spreadsheets  for  organizing  quantitative 
data,  and  presentation  software  to  create  graphs  and 
slide  shows. 

School  Adoption  of  Computers 

An  international  survey  in  1989  revealed  that  in  most 
industrialized  countries  widespread  school  adoption  of 
computers  began  around  1983  and  increased  at  a  steady 
rate  each  year  thereafter.  By  1989,  several  countries 
(Luxembourg,  Switzerland,  the  Netherlands,  France, 
United  States,  and  British  Columbia  in  Canada)  had 
reached  the  plateau  of  having  approximately  one  com¬ 
puter  per  classroom:  however,  as  had  been  discovered 
earlier  in  the  audio-visual  era,  access  to  the  hardware 
does  not  equate  to  use.  This  survey  as  well  as  later 
surveys  indicated  that  only  a  small  percentage  of  teach¬ 
ers  who  had  access  to  computers  actually  integrated 
their  use  significantly  into  their  teaching  (Pelgrum  and 
Plomp,  1991:  Plomp  and  Pelgrum,  1993).  In  these 
early  years,  student  usage  was  primarily  to  learn  about 
computers  rather  than  to  learn  with  computers. 

As  access  to  computers  grew,  eventually  reaching 
recommended  levels  by  about  2005 ,  student  usage  con¬ 
tinued  to  lag  behind  availability.  In  the  mid-1990s, 
student  use  was  still  rather  mundane,  often  limited  to 
a  few  hours  per  week  of  drill  programs  or  routine 
word-processing  applications  (Anderson  and  Ronn- 
kvist,  1999).  By  the  mid-2000  decade,  it  appeared  that 
more  students  were  using  computers  for  many  more 
hours  (U.S.  Department  of  Education,  2005)  and  pos¬ 
sibly  for  uses  more  central  to  the  curriculum,  although 
this  is  debatable. 

From  the  1980s  through  the  1990s,  the  pendulum 
was  swinging  from  analog  media  to  digital  media  as  the 
primary  source  of  instructional  materials  in  schools, 
colleges,  and  corporate  training  centers:  however,  tradi¬ 
tional  media  formats  such  as  textbooks,  the  overhead 
projector,  and  videocassettes  have  continued  to  be  used 
heavily  by  teachers  at  all  levels  right  up  to  the  present. 
As  an  example,  three  fourths  of  all  corporate  trainers 
reported  that  they  use  manuals  and  textbooks  and  over 
one  half  use  videocassettes  (Dolezalek,  2004,  p.  34). 

Internet  and  World  Wide  Web 

As  profoundly  as  personal  computers  changed  the 
information  environment  in  the  1980s,  the  advent  of 
the  Internet  in  the  1990s  changed  it  even  more.  The 
rapid  increase  in  connections  to  the  Internet  in  the  early 
1990s  vastly  expanded  the  potential  for  sharing  infor¬ 
mation  at  a  distance.  The  invention  of  graphical  user 
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interfaces  allowed  the  World  Wide  Web  to  become  the 
most  popular  Internet  protocol  around  1993.  Because 
of  its  ubiquity  it  became  the  de  facto  standard  platform 
for  sharing  resources.  Being  structured  according  to 
hypermedia  principles  (links  and  nodes)  it  largely  dis¬ 
placed  the  earlier  concept  of  hypermedia  programs 
residing  in  a  local  computer  system.  With  programs 
residing  on  the  Web,  they  could  be  tapped  from  any 
place  in  the  world  that  could  access  the  Internet. 

Distance  Education 

What  is  now  known  as  distance  education  can  be  traced 
back  at  least  to  1840  in  England,  when  Isaac  Pitman 
began  offering  shorthand  lessons  through  the  medium 
of  mail.  Correspondence  study  was  well  established  in 
Germany,  as  well ,  before  the  first  American  correspon¬ 
dence  study  program  (Society  to  Encourage  Studies  at 
Home)  began  in  1873  in  Boston.  Correspondence 
study  attained  respectability  in  the  United  States  when 
major  programs  were  offered  at  the  University  of  Chi¬ 
cago  and  Columbia  University  early  in  the  20th  cen¬ 
tury.  In  1956,  Chicago  City  Junior  College  launched 
TV  College,  using  broadcast  television  to  offer  post¬ 
secondary  education  degrees  to  viewers  in  the  Chicago 
area.  It  was  immensely  popular  and  it  also  happened 
to  be  on  the  itinerary  of  a  visiting  group  of  professors 
from  the  United  Kingdom  in  1964,  a  group  known  as 
the  Brynmor  Jones  Committee.  The  1965  report  of  this 
committee  ( Audio-Visual  Aids  in  Higher  Scientific 
Education)  was  a  watershed  in  the  evolution  of  edu¬ 
cational  technology  in  the  United  Kingdom  (MacKen- 
zie,  2005),  and  it  inspired  the  vision  of  an  “open  uni¬ 
versity,”  which  became  part  of  the  Labour  Party’s 
platform  in  1963. 

The  British  Open  University 

The  establishment  in  1969  of  the  British  Open  Uni¬ 
versity  became  a  landmark  in  distance  education  in 
several  regards.  First,  it  was  designed  as  an  open- 
access,  degree-granting  institution.  Second,  it  was 
expected  to  scale  up  to  serve  tens  of  thousands  of 
students  at  a  time.  Third,  although  broadcast  television 
in  partnership  with  the  BBC  provided  the  most  visible 
part  of  its  instruction,  it  was  designed  to  integrate 
television,  other  audio-visual  media,  print,  telephone 
help  systems,  and  face-to-face  tuition  in  a  seamless 
whole.  The  British  Open  University  eventually  became 
the  model  for  most  of  the  distance  universities  that 
came  later.  In  particular,  the  broadcasting  partnership 
has  been  emulated  by  the  Open  University  of  Hong 
Kong,  the  Bangladesh  Open  University,  and  the 
Korean  National  Open  University  (Bates,  2005). 


Mega-Universities 

The  subsequent  years  have  seen  the  flowering  of  a 
series  of  large-scale  distance  universities,  referred  to 
as  mega-universities  by  Daniel  (1996),  such  as  the 
China  TV  University  System,  the  French  Centre 
National  d'Enseignement  a  Distance,  Indira  Gandhi 
National  Open  University,  and  Indonesia’s  Universitas 
Terbuka,  among  others.  Perhaps  the  largest  is  Anadolu 
University,  based  in  Turkey,  which  serves  over  500 ,000 
students  from  Germany  to  Cyprus . 

Web-Based  Courses 

The  impact  of  the  computer  on  distance  education  was 
first  manifested  through  courses  based  on  computer 
conferencing.  Boise  State  University  was  offering  a 
master’s  degree  program  in  educational  technology  via 
computer  conferencing  in  the  1990s.  More  recently, 
however,  it  is  the  mushrooming  of  the  World  Wide 
Web  that  has  fueled  growth  in  distance  education.  The 
Web  makes  it  feasible  for  higher  education  institutions 
and  corporate  training  operations  to  offer  their  courses 
at  a  distance  economically.  Instead  of  requiring  video 
studios  and  expensive  transmission  systems,  Web- 
based  courses  use  existing  computer  infrastructure  at 
no  extra  cost  to  deliver  their  courses  to  users.  Web- 
based  courses  began  to  appear  around  1995.  The  Uni¬ 
versity  of  British  Columbia  offered  its  first  entirely 
Web-based  credit  courses  in  1996  (Bates,  2005). 

These  demonstrated  successes  created  a  brief  land- 
rush  mentality  among  American  universities  in  the  late 
1990s  to  try  to  capture  a  commanding  share  of  the 
market  for  online  distance  education.  By  the  end  of  the 
1990s,  courses  delivered  via  Internet  were  offered  at 
60%  of  all  colleges,  compared  with  22%  only  5  years 
earlier  (U.S.  Department  of  Education,  1999).  Within 
a  few  years,  the  investment  fever  had  waned  and  the 
field  was  left  to  the  slow-but- steady  providers  of  reli¬ 
able,  good-quality  courses.  Over  30  states  formed  dis¬ 
tance  learning  consortia  to  pool  their  resources  and 
give  students  a  wider  menu  of  choices.  Consortia  such 
as  UMassOnline  and  Illinois  Virtual  Campus  experi¬ 
enced  double-digit  growth  in  enrollments  between 
2000  and  2006  (Bichelmeyer  and  Molenda,  2006).  In 
Europe,  the  Open  University  of  Catalonia,  which 
opened  in  1996  as  a  public,  fully  online  university,  was 
serving  25,000  students  by  2004  (Bates,  2005). 

Virtual  Schools 

During  the  past  decade  distance  education  also  became 
a  major  phenomenon  at  the  K-12  level  in  the  United 
States.  Many  school  districts ,  especially  those  in  urban 
centers  facing  competition  from  home  schooling,  char- 
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ter  schools,  and  private  alternatives,  started  virtual 
schools  to  try  to  retain  students  who  were  drifting 
outside  the  public  school  system.  A  major  national 
survey  in  2003  found  that  students  in  more  than  one 
third  of  public  school  districts  enrolled  in  distance 
education  courses:  76%  at  the  high-school  level  and 
another  15%  in  combined  or  ungraded  schools  (Setzer 
and  Lewis,  2005). 

Computer-Based  Residential  Courses 

The  use  of  the  Web  grew  not  only  for  off-campus 
distance  courses  but  also  for  on-campus  residential 
courses.  By  2004,  most  American  universities  had 
adopted  a  standard  course  management  system 
(CMS),  a  suite  of  applications,  tying  together  Web 
presentations,  e-mail,  discussion  forums,  and  other 
applications.  Blackboard.com,  introduced  its  first 
CMS,  Courselnfo,  in  1999.  By  2006,  Blackboard  had 
merged  with  its  largest  rival,  WebCT,  and  dominated 
the  field  of  college  and  university  CMSs,  although 
rival  open-source  software  CMSs  were  also  being 
developed,  such  as  Moodle  and  Sakai.  The  wide¬ 
spread  adoption  of  CMSs  blurred  the  line  between 
distance  and  residential  courses,  as  it  allowed  resi¬ 
dential  students  to  carry  out  more  of  their  course 
activities  on  a  computer,  making  distance  a  matter  of 
degree  rather  than  of  kind . 

Growth  in  distance  education  created  a  demand  for 
distance  course  designers  and  developers,  making  this 
one  of  the  key  growth  areas  for  educational  technolo¬ 
gists.  By  2006,  the  work  of  educational  technology 
had  shifted  heavily  into  the  digital  domain,  moving  the 
field  closer  to  the  field  of  informatics:  however,  the 
identity  forged  in  the  pre -computer  era— the  designer 
of  learning  environments— remains  central,  supple¬ 
mented  by  the  even  earlier  identity  as  promoter  of 
sensorially  rich  learning  experiences. 

CONCLUSION 

Since  its  inception,  the  field  of  educational  communi¬ 
cations  and  technology  has  been  characterized  by 
changes  in  technology  and  radical  shifts  in  its  under¬ 
lying  paradigms.  Beginning  as  visual  instruction,  then 
audio-visual  instruction,  adherents  were  attracted  by 
the  prospect  of  enriching  the  learning  experience 
through  involvement  of  the  senses.  Advances  in  broad¬ 
casting  technology  encouraged  another  vision:  to  bring 
educational  opportunities  to  audiences  hitherto  beyond 
the  reach  of  schooling.  New  psychological  theories 
came  along,  holding  out  the  promise  of  devising 
frameworks  for  lessons  that  would  facilitate  learning 
in  dramatically  improved  ways.  Then  digital  technol¬ 


ogies  captured  the  field,  enabling  designers  to  create 
learning  environments  in  which  verbal  and  visual 
media  could  be  combined  under  the  inspiration  of  var¬ 
ious  pedagogical  theories  into  expository  lessons, 
problem-solving  laboratories,  collaborative  work 
spaces,  or  hybrids  thereof— all  made  available  to 
almost  anyone,  anywhere. 

The  field  has  also  been  and  continues  to  be  thor¬ 
oughly  interdisciplinary.  Certain  individuals  and  insti¬ 
tutions  have  provided  a  measure  of  continuity  over  the 
years,  but  discontinuity  has  been  a  recurrent  problem. 
Each  media  revolution  and  each  paradigm  change 
bring  new  people  with  different  backgrounds  into  the 
field.  From  visual  instruction  to  radio,  from  audio¬ 
visual  media  to  programmed  instruction,  from  instruc¬ 
tional  design  to  distance  education— each  transition 
has  tended  to  mean  reinventing  the  wheel  in  terms  of 
the  questions  asked  in  research  and  in  terms  of  grand 
dreams  about  revolutionizing  education. 

Likewise,  the  field  has  been  and  continues  to  be 
international  in  scope.  Although  the  geographically 
diffuse  components  of  the  field  are  only  loosely  artic¬ 
ulated,  ideas  have  been  able  to  flow  sufficiently  to 
allow  rich  cross-fertilization  of  both  theories  and  prac¬ 
tices.  As  the  Internet  has  made  communication  and 
collaboration  an  order  of  magnitude  easier,  it  appears 
that  the  coming  years  will  knit  participants  even  more 
tightly  together. 
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ABSTRACT 

This  chapter  addresses  the  theoretical  foundations  for 
research  in  educational  communications  and  technol¬ 
ogy.  Four  relevant  areas  are  explored:  (1)  the  psychol¬ 
ogy  of  learning,  (2)  communications  theory,  (3) 
human-computer  interaction,  and  (4)  instructional 
design  and  development.  Past  work  in  these  four  areas 
can  be  viewed  as  providing  a  theoretical  foundation  for 
further  research  and  development  in  educational  com¬ 
munications  and  technology. 

KEYWORDS 

Educational  communications:  Forms,  means,  and  meth¬ 
ods  of  expressing  and  sharing  ideas,  information, 
and  knowledge  to  support  learning  and  instruction. 


Educational  technology:  The  disciplined  application  of 
scientific  principles  and  theoretical  knowledge  to 
support  and  enhance  human  learning  and  perfor¬ 
mance. 

Theoretical  foundation:  A  related  set  of  rules  and  prin¬ 
ciples  that  can  be  brought  to  bear  as  a  basis  for 
making  predictions  and  providing  explanations  for 
a  variety  of  phenomena. 

INTRODUCTION 

‘‘May  you  have  a  strong  foundation  when  the  winds  of 
changes  shift.”  (from  Bob  Dylan’s  “Forever  Young”) 

A  fundamental  aspect  of  educational  communications 
and  technology  is  change.  Television  was  a  new  com¬ 
munications  technology  that  influenced  (for  better  or 
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worse)  learning  and  instruction  50  years  ago.  Since 
then,  there  have  been  many  other  new  technologies 
and  innovations  in  educational  communications;  how¬ 
ever,  the  general  problems  for  educational  researchers 
have  remained  relatively  constant— for  example,  how 
to  make  effective  use  of  a  specific  technology  in  a 
particular  educational  context.  In  conducting  research 
for  such  a  purpose,  one  can  proceed  on  the  basis  of 
prior  work.  What,  then,  is  the  nature  of  theoretical 
foundations  in  educational  communications  and  tech¬ 
nology? 

The  Approach 

The  quick  answer  is  that  theoretical  foundations  are 
the  basis  for  conducting  research  in  an  area.  In  this 
chapter,  rather  than  review  specific  research  founda¬ 
tions  and  cite  appropriate  sources  along  the  way,  I 
describe  the  general  features  of  the  educational  com¬ 
munications  and  technology  research  landscape,  citing 
only  a  very  few  sources  in  the  main  body  of  this 
chapter.  I  have  included  an  extended  bibliography  at 
the  end  of  the  chapter  to  help  readers  extend  their 
investigations  in  directions  that  seem  useful  and  appro¬ 
priate  to  them  (an  asterisk  in  the  extended  bibliography 
indicates  a  critical  reference). 

There  are  two  reasons  for  selecting  this  approach: 
(1)  the  lay  of  the  land—  the  general  things  relevant  to 
most  educational  technology  research  are  worth  dis¬ 
cussing  as  part  of  our  intellectual  heritage  and  identity, 
and  (2)  my  lack  of  expertise— I  do  not  regard  myself 
as  an  expert  in  most  of  the  areas  to  be  mentioned,  so 
the  best  I  can  do  is  to  point  to  things  that  I  believe  are 
relevant.  I  certainly  do  not  mean  to  imply  that  the 
framework  presented  here  or  the  items  listed  in  the 
bibliography  are  exhaustive  or  even  the  most  salient 
aspects  of  educational  technology  research.  They  are 
simply  the  things  I  have  stumbled  across  in  my  wan¬ 
dering  around  this  intriguing  landscape  in  the  last  20 
years. 

Nature  of  Research  and  Theory 

The  basic  question  that  research  aims  to  answer  is  why 
things  happen  the  way  they  do.  Developing  an  answer 
to  such  a  question  often  involves  a  general  rule  or 
principle  that  has  explanatory  power  or  that  will  be 
predictive,  as  we  often  ask  about  future  events— what 
will  happen  if  we  manipulate  or  change  one  or  more 
things  in  the  situation?  A  related  set  of  rules  and  prin¬ 
ciples  that  has  been  shown  to  be  reliable  in  many 
situations  might  be  regarded  as  a  theory. 

Of  course,  there  are  different  kinds  of  research 
questions  and  a  variety  of  research  objects.  Appropri¬ 


ate  methods  depend  on  the  nature  of  the  questions  and 
the  objects  investigated.  Example  question  types  for 
educational  technology  researchers  include:  (1)  What 
will  help  these  students  leam  this  material?  (2)  Why 
do  those  students  have  difficulty  in  learning  that  mate¬ 
rial?  (3)  When  will  this  technology  and  that  form  of 
communication  be  effective  with  those  learners?  (4) 
How  can  we  explain  the  effects  of  that  change  in  the 
instruction?  Other  types  of  questions  can  be  framed, 
as  well. 

When  specific  questions  are  put  forth  in  concrete 
contexts,  one  can  identify  appropriate  methods  of 
investigation.  Oftentimes  in  exploratory  research 
involving  a  new  form  of  communications  or  a  new 
technology,  qualitative  methods  (e.g.,  action  research, 
case  studies,  ethnographic  research)  are  useful  in  gain¬ 
ing  an  understanding  of  and  interpreting  relevant 
aspects  and  factors  that  seem  to  influence  learning  and 
performance.  On  the  other  hand,  when  developing  a 
general  explanation  for  which  factors  systematically 
result  in  particular  outcomes,  one  might  use  quantita¬ 
tive  methods  (e.g.,  controlled  studies  with  randomized 
samples,  quasi-experimental  studies).  In  some  cases, 
a  study  might  involve  both  qualitative  and  quantitative 
methods  depending  on  the  questions  being  investigated 
and  the  research  objects  involved. 

What  confounds  this  already  complex  area  of 
research  and  theory  is  that  educational  technologists 
are  generally  trying  to  find  means  to  improve  learning 
and  performance.  When  a  general  instructional 
approach  is  devised,  initial  evidence  of  its  efficacy 
must  be  developed.  This  may  involve  a  formative  eval¬ 
uation  that  is  aimed  at  making  subsequent  modifica¬ 
tions  to  improve  outcomes.  Moreover,  in  the  area  of 
learning  and  instruction,  there  are  many  differences 
that  make  it  difficult  to  develop  general  explanations 
and  predictive  theories.  What  works  with  one  kind  of 
learning  task  and  a  particular  group  of  learners  may 
not  work  with  others.  The  circumstances  in  which 
learning  occurs  may  also  affect  outcomes. 

It  is  a  wonder  that  instructional  science  has  made 
such  progress  in  the  last  50  years  given  the  nature  of 
these  obstacles.  Some  researchers  are  inclined  to  aban¬ 
don  the  traditional  scientific  approach  altogether,  but 
doing  so  might  be  a  hasty  and  unwarranted  decision. 
A  more  modest  approach  is  to  use  traditional  research 
methods  and  make  adaptations  and  modifications  as 
they  seem  appropriate  for  particular  research  circum¬ 
stances.  The  starting  point  of  a  research  inquiry,  as 
suggested  earlier,  is  the  admission  that  one  lacks  a 
good  explanation  for  why  things  happen  the  way  they 
do.  This  point  of  departure  across  the  research  land¬ 
scape  involves  both  humility  (one  begins  an  inquiry 
not  knowing  the  outcome)  and  openness  (one  journeys 
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A  framework  for  a  discussion  about  foundations 


Figure  2.1  A  framework  for  discussing  theoretical  foundations. 


forth  open  to  a  variety  of  outcomes).  These  two  aspects 
of  traditional  research  and  theory  — humility  and  open¬ 
ness-ought  to  be  retained  as  the  core  of  our  theoret¬ 
ical  foundations. 

Four  Foundation  Areas 

The  notion  of  foundations  in  a  discipline  is  not  new. 
Having  foundations  implies  having  a  history.  More¬ 
over,  foundations  imply  that  there  is  a  solid  basis  for 
further  work— research  and  development,  in  this  case. 
In  other  words,  foundations  look  both  to  the  past  and 
to  the  future.  In  looking  to  the  past,  it  seems  to  me 
that  psychology  is  obviously  a  critical  foundation 
area— especially  the  psychology  of  learning.  Human 
behavior  and  the  development  of  knowledge  and  skills 
are  perhaps  the  bedrock  on  which  educational  com¬ 
munications  and  technology  rest.  This  idea  is  con¬ 
firmed  by  the  many  graduate  curricula  in  educational 
technology  and  instructional  systems  that  include  a 
core  course  requirement  in  the  psychology  of  learning 
(or  closely  related  subject  such  as  the  principles  of 
learning) .  One  section  of  this  chapter  briefly  introduces 
this  foundation  area. 

The  nature  of  communications  and  its  many  forms 
and  variations  also  seems  critical  to  educational  tech¬ 
nology.  Relevant  theories  exist  to  guide  both  the  rep¬ 
resentation  of  information  and  the  transmission  of 
information  from  one  place,  person,  or  system  to 
another  place,  person,  or  system. 

Thus  far,  I  have  identified  what  I  regard  as  two 
basic  foundation  areas:  what  people  do  (psychology) 
and  what  people  say  (communications).  The  other  two 
foundation  areas  to  be  discussed  below  are  more  spe¬ 


cific  and  applied  in  that  they  address  the  ways  and 
means  of  facilitating  what  people  do  and  say,  espe¬ 
cially  with  regard  to  their  interactions  with  others  (e.g., 
teachers  and  students)  and  with  learning  materials  and 
instructional  systems.  One  can  easily  imagine  basic 
values  and  scientific  attitudes  in  addition  to  the  two 
cited  here— humility  and  openness  (see  Figure  2.1). 
One  can  also  identify  other  basic  foundations  and 
building  blocks  to  support  and  motivate  research  in 
educational  communications  and  technology.  The 
framework  presented  here  (see  Figure  2.1)  is  intended 
only  for  purposes  of  discussion  and  as  an  arbitrary 
point  of  departure. 

In  each  of  these  four  areas,  I  identify  relevant  past 
research  and  developments  and  suggest  how  these 
might  influence  future  research  and  development  in 
educational  communications  and  technology. 


PSYCHOLOGY  OF  LEARNING 

In  How  We  Think ,  Dewey  (1910)  argued  that  we  need 
to  understand  the  nature  of  thought  to  be  able  to  devise 
appropriate  means  and  methods  to  train  thought.  Along 
the  way,  Dewey  distinguished  abstract  from  concrete 
thinking  and  indicated  that  many  training  regimens 
proceeded  from  the  concrete  to  the  abstract.  Piaget 
(1929)  identified  a  parallel  progress  in  the  develop¬ 
ment  of  reasoning  in  children.  Dewey  cited  implica¬ 
tions  for  his  theory  of  thought  for  instructional  plan¬ 
ning— namely,  he  argued  that  effective  training 
involves  a  balance  of  many  things,  including  especially 
information,  observation,  imagination,  reflection, 
playfulness,  and  seriousness. 
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Bruner  (1966)  made  explicit  the  close  relationship 
between  intellectual  development  as  investigated  by 
Piaget  (1929)  and  the  practical  task  of  devising  support 
for  learning  of  the  kind  suggested  by  Dewey  (1910). 
Since  that  time,  other  researchers  have  devised  instruc¬ 
tional  design  frameworks  based  on  new  psychological 
research  (see,  for  example,  Merrill  and  Twitchell, 
1993;  Reigeluth,  1983,  1999;  van  Merrienboer,  1997). 
Meanwhile,  the  psychology  of  learning  has  undergone 
basic  changes,  evolving  from  behaviorism  (see,  for 
example,  Skinner,  1938)  to  cognitivism  (e.g.,  Ander¬ 
son,  1983)  to  constructivism  (e.g..  Ford,  1987).  Asso¬ 
ciated  with  these  changes  have  been  changes  in  how 
researchers  and  developers  think  about  instruction 
(e.g.,  Driscoll,  2000;  Lesgold  et  al.,  1978;  Reigeluth, 
1983,  1999;  Spector  and  Anderson,  2000). 

What  seems  clear  is  that  there  is  much  more  to 
understand  about  human  psychology  and  learning.  As 
a  consequence,  models  of  instruction  are  likely  to  con¬ 
tinue  to  evolve  based  on  our  understanding  of  human 
behavior,  cognition,  and  emotion.  New  areas  of  explo¬ 
ration  in  psychology  that  are  likely  to  impact  instruc¬ 
tional  design  research  include  the  linkages  between 
neural  mechanisms  and  problem  solving  as  well  as  the 
structure  of  memory  and  its  role  in  both  learning  and 
unlearning. 


COMMUNICATIONS  THEORY 

The  second  basic  foundation  to  be  discussed  here  is 
communications  theory.  The  reason  for  this  is  that 
nearly  all  learning  involves  language  (Vygotsky,  1962, 
1978).  The  relationship  of  language,  thinking,  and 
learning  appears  in  many  philosophical  analyses;  see, 
for  example,  Dewey’s  (1910)  How  We  Think  and  Wit¬ 
tgenstein’s  (1953)  Philosophical  Investigations .  The 
notion  of  the  central  role  of  language  games  in  thinking 
and  problem  solving  was  elaborated  by  Wittgenstein 
(1953).  A  language  game  is  a  rule-governed  means  of 
communication  adopted  by  an  identifiable  community. 
In  the  case  of  learning  and  instruction,  one  might 
regard  the  professional  communities  of  practice  and 
the  language  games  they  have  adopted  as  fundamental 
units  of  analysis. 

One  can  find  connections  between  language  and 
learning  elaborated  in  the  ancient  works  of  Plato  as 
well  as  in  modern  philosophy  (Klein  et  al.,  2004;  Spec- 
tor,  1994).  In  short,  language  is  fundamental  to  think¬ 
ing  and  learning.  The  many  techniques  and  tools  for 
the  expression  and  sharing  of  language— communica¬ 
tions— are  then  fundamental  aspects  of  thinking  and 
learning.  Moreover,  methods  of  communications  are 


likely  to  influence  learning  outcomes  and  impact  the 
design,  deployment,  and  evaluation  of  instructional 
systems  and  performance  environments. 

The  notion  of  language  games  associated  with 
Wittgenstein  (1953)  places  particular  emphasis  on  the 
use  of  language.  The  underlying  notion  is  that  it  is  use 
and  context  that  give  meaning  to  words  and  statements, 
which,  without  a  context  and  accepted  or  expected  use, 
are  basically  lifeless.  Although  use  and  context  are 
certainly  fundamental  to  meaning,  communications 
theorists  have  examined  much  more  with  regard  to  the 
fundamental  aspects  of  language.  For  example, 
Charles  Sanders  Peirce,  Ferdinand  de  Saussure, 
Claude  Shannon,  and  others  developed  and  elaborated 
theories  of  signs  and  the  rules  that  govern  their  use  in 
various  contexts  (Hardwick,  1977;  Saussure,  1959; 
Shannon,  1948).  Saussure  (1959)  made  a  clear  distinc¬ 
tion  between  the  sign  and  that  which  is  signified. 
Peirce  added  a  third  category  — the  notion  of  an 
icon— and  argued  that  the  sign  by  itself  was  devoid  of 
meaning.  This  argument  was  based  on  a  pragmatic 
view  of  language  and  introduced  the  notion  of  lan¬ 
guage  in  use  that  permeates  Wittgenstein’s  Investiga¬ 
tions.  Shannon  extended  these  concepts  to  include  the 
computer  as  a  user  of  signs;  he  created  a  mathematical 
treatment  of  symbol  systems  that  has  been  widely 
influential  in  the  computer  and  information  science 
communities. 

As  with  the  other  pillars  of  educational  communi¬ 
cations  and  technology  mentioned  in  this  chapter, 
many  more  things  could  be  cited  as  fundamental  to 
inquiry  and  scholarship.  Before  turning  to  two  applied 
foundation  areas,  it  is  worth  mentioning  one  additional 
aspect  of  communications  theory  that  is  particularly 
pertinent  to  the  world  of  digital  media— namely,  the 
notion  of  visualization  as  a  form  of  communicating. 
Tufte  (1977)  argues  that  graphics  are  a  form  of  expres¬ 
sion— a  language— that  can  be  used  more  or  less  effec¬ 
tively  in  accordance  with  how  well  certain  rules  and 
guidelines  governing  their  use  are  followed.  Those 
who  are  interested  in  designing  effective  graphics  are 
well  advised  to  look  at  Tufte’s  (1977)  writings,  which 
might  be  considered  one  of  many  bridges  between  the 
basic  foundation  area  of  communications  theory  and 
the  more  applied  foundation  area  of  human-computer 
interaction. 


HUMAN-COMPUTER  INTERACTION 

How  one  person  represents  something  to  another  per¬ 
son  and  the  particular  form  of  expression  used  to  con¬ 
vey  that  representation  influence  what  is  likely  to  be 


24 


Theoretical  Foundations 


understood.  That  is  why  communications  theory  is  a 
fundamental  consideration  in  learning  and  why  so 
much  emphasis  is  placed  on  the  design  of  instructional 
messages.  Given  the  widespread  use  of  computers  to 
support  learning  and  the  growing  use  of  handheld 
devices,  it  seems  quite  natural  to  treat  the  exchange  of 
information  between  humans  and  the  computers  with 
which  they  interact  as  a  distinct  area.  There  are  cer¬ 
tainly  psychological  aspects  to  human-computer  inter¬ 
action,  as  well  as  communications  issues.  As  a  conse¬ 
quence,  in  this  simple  framework,  human-computer 
interaction  is  considered  an  applied  foundation  area  of 
educational  communications  and  technology  research. 

Many  aspects  of  human-computer  interaction  have 
been  investigated.  They  include  basic  human  factor 
issues  such  as  the  colors  and  fonts  that  are  easily 
discernible  on  a  computer  monitor  and  the  various 
types  of  control  devices  and  how  they  can  be  designed 
to  facilitate  human  use.  Other  human  use  issues 
include  when  and  how  systems  might  support  multiple 
learners  who  might  be  working  or  learning  in  different 
places  and  at  different  times.  The  types  of  computer¬ 
generated  messages  that  are  likely  to  be  supportive  of 
learning  and  performance  have  been  studied  in  many 
different  contexts  and  comprise  an  important  area  of 
human-computer  interaction  research. 

Just  as  different  disciplines  have  specific  research 
methods  that  are  considered  appropriate  for  particular 
problems,  these  foundation  areas  have  preferred  or 
commonly  used  research  methods.  One  of  the  widely 
used  methods  in  human-computer  interaction  studies 
involves  activity  theory  (Leont’ev,  1975;  Nardi,  1996). 
Activity  theory  is  a  framework  for  studying  humans 
and  their  use  of  artifacts.  Emphasis  is  placed  on  an 
object’s  purpose  and  how  it  is  used  by  an  individual 
often  working  with  others  to  achieve  a  particular  goal. 
Activity  theory  emphasizes  purposeful  social  interac¬ 
tions  and  might  well  be  considered  a  research  exten¬ 
sion  of  the  notion  of  a  language  game. 

Perhaps  the  most  well  known  instructional  theory 
that  involves  human-computer  interaction  is  Merrill’s 
(1980)  component  display  theory  (CDT).  An  interest¬ 
ing  aspect  of  CDT  is  that  it  introduced  the  issue  of 
learner  control  and  provided  guidelines  for  when  con¬ 
trol  should  pass  from  the  instructional  computing  sys¬ 
tem  to  the  learner  and  what  should  be  included  in  that 
control.  CDT  also  provided  an  early  version  of  what 
was  reasonable  to  display  on  the  computer  screen 
given  the  current  state  of  a  learner’s  progress  through 
a  set  of  learning  materials.  These  concepts  would  of 
course  evolve  and  have  given  rise  to  many  subsequent 
studies  and  instructional  design  frameworks,  which  are 
discussed  next. 


INSTRUCTIONAL  DESIGN 
AND  DEVELOPMENT 

The  final  foundation  area  to  be  briefly  discussed 
involves  what  might  be  regarded  as  the  core  area  of 
professional  practice  in  educational  communications 
and  technology— namely,  instructional  design  and 
development.  This  is  a  rich  area  of  empirical  research. 
Given  the  dynamic  nature  of  learning  and  instruction 
and  the  introduction  of  new  technologies  and  forms  of 
communications,  it  is  unlikely  that  this  research  area 
will  ever  be  exhausted.  In  closing  out  this  discussion 
of  theoretical  foundations,  a  model  to  guide  investiga¬ 
tions  of  instructional  design  and  development  issues 
is  presented. 

Instructional  design  and  development  are  human 
activities.  The  general  purpose  of  these  activities  is  to 
facilitate  and  support  human  learning  and  perfor¬ 
mance.  To  achieve  desired  outcomes,  instructional 
designers  have  developed  instructional  design  models 
and  principles,  based  in  large  part  on  the  psychology 
of  learning  and  what  is  known  about  effective  com¬ 
munications  (Gustafson  and  Branch,  2002).  These 
models  suggest  that  different  instructional  methods 
and  strategies  are  likely  to  be  effective  in  different 
circumstances  (Gagne,  1985;  Merrill  and  Twitchell, 
1993).  Instructional  design  principles  have  been  devel¬ 
oped  that  link  back  specifically  to  the  psychology  of 
learning  and  human  perception  (Gagne  et  al.,  1992). 
Specific  instructional  design  models  have  been  devel¬ 
oped  to  fit  particular  types  of  learning  outcomes. 

One  of  the  more  robust  and  well  elaborated  of  these 
models  in  the  four-component  instructional  design 
(4C/ID)  model  developed  by  van  Merrienboer  (1997). 
In  4C/ID  there  is  a  fundamental  distinction  between 
recurrent  and  nonrecurrent  tasks  — that  is  to  say,  those 
tasks  whose  performance  remains  relatively  constant 
in  spite  of  variations  or  changes  in  the  surrounding 
conditions  (recurrent  tasks)  and  those  requiring  signif¬ 
icant  changes  in  performance  due  to  changes  in  the 
surrounding  conditions  (nonrecurrent  tasks).  Instruc¬ 
tional  support  for  the  former  might  include  part-task 
training  aiming  for  automaticity  of  task  performance; 
instructional  support  for  the  latter  might  include 
whole-task  demonstrations  and  practice  in  a  variety  of 
circumstances  guided  by  heuristics  and  a  mentor  or 
coach. 

A  theoretical  framework  that  accommodates  all  of 
these  different  models  and  associated  research  can  be 
found  in  Reigeluth  (1983).  Reigeluth  argues  that  a 
basic  difference  between  psychological  research  on 
learning  and  instructional  design  research  is  that  the 
former  is  primarily  descriptive  (these  learners  under 
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those  conditions  achieved  particular  outcomes), 
whereas  the  latter  is  primarily  prescriptive  (to  achieve 
a  desired  outcome  given  certain  conditions,  one  ought 
to  use  that  instructional  strategy).  This  instructional 
design  research  framework  can  be  applied  at  the  lesson 
level  or  for  an  entire  course  or  program. 

Some  researchers  have  challenged  the  descriptive- 
prescriptive  distinction  as  being  a  naively  objectivist 
view  that  overlooks  the  fact  that  values  are  involved 
in  descriptive  research  just  as  they  are  obviously 
involved  in  prescriptive  research.  While  I  happen  to 
acknowledge  the  fundamental  nature  of  attitudes  and 
values  for  all  human  activity  (recall  Figure  2.1),  I 
believe  that  Reigeluth’s  framework  remains  a  valid 
guide  for  ongoing  research  in  our  discipline.  Indeed, 
I  believe  instructional  design  research  and  theory 
would  become  a  marginalized  craft  without  much 
influence  or  impact  on  education  without  such  a  sound 
theoretical  foundation. 

CONCLUSION 

It  is  my  hope  that  a  basis  for  ongoing  dialog  about 
theoretical  foundations  has  been  established  in  this 
chapter.  I  realize  that  I  have  traversed  much  landscape 
but  not  provided  very  much  detail.  I  acknowledge  hav¬ 
ing  ignored  much  that  is  relevant.  These  errors  of  com¬ 
mission  and  omission  could  form  the  basis  for  discus¬ 
sion  and  leave  the  author  of  the  Theoretical 
Foundations  chapter  in  the  next  edition  of  this  Hand¬ 
book  much  more  about  which  to  write. 

In  closing  I  would  like  to  return  briefly  to  the  two 
basic  values  that  provided  a  point  of  departure  :  humil¬ 
ity  and  openness.  It  is  clear  to  me  that  a  great  many 
have  contributed  to  our  knowledge  about  educational 
communications  and  technology.  I  have  chosen  to 
include  an  extended  bibliography  rather  than  a  simple 
list  of  references  so  as  to  emphasize  that  fact.  Indeed, 
we  stand  on  the  shoulders  of  giants.  The  problem  is 
deciding  whose  shoulders  to  use  for  a  meaningful 
boost  up  and  look  over  the  landscape.  Part  of  devel¬ 
oping  a  sense  of  humility  is  realizing  the  significance 
of  what  so  many  who  have  preceded  us  have  accom¬ 
plished. 

A  second  pathway  to  humility  is  realizing  how 
limited  our  own  understanding  is  on  any  particular 
issue  of  any  complexity  at  all.  I  have  adopted  the 
following  mantra  for  this  purpose— surely  it  would  be 
a  remarkable  coincidence  if  the  limits  of  my  imagina¬ 
tion  happened  to  coincide  with  the  limits  of  reality. 
Openness  follows  naturally  from  a  sense  of  humility 
and  the  realization  that  others  often  have  excellent 
ideas. 


As  educational  researchers,  we  might  wonder  what 
will  come  from  what  we  have  done,  are  now  doing, 
and  are  likely  to  do  in  the  future.  We  conduct  studies, 
we  write  articles  and  books,  we  pile  up  accomplish¬ 
ments,  and  yet  there  is  always  more  to  be  done,  new 
territory  to  be  explored,  alternative  explanations  to 
investigate,  new  methods  to  try,  and  so  on.  I  am 
reminded  of  a  sonnet  written  by  Percy  Bysshe  Shelly 
(1818)  with  which  I  close: 

Ozymandias 

I  met  a  traveler  from  an  antique  land 

Who  said:  Two  vast  and  trunkless  legs  of  stone 

Stand  in  the  desert  ...  Near  them,  on  the  sand, 

Half  sunk,  a  shattered  visage  lies,  whose  frown. 

And  wrinkled  lip,  and  sneer  of  cold  command. 

Tell  that  its  sculptor  well  those  passions  read 
Which  yet  survive,  stamped  on  these  lifeless  things. 

The  hand  that  mocked  them,  and  the  heart  that  fed; 

And  on  the  pedestal  these  words  appear: 

“My  name  is  Ozymandias,  king  of  kings: 

Look  on  my  works,  ye  Mighty,  and  despair!” 

Nothing  beside  remains.  Round  the  decay 
Of  that  colossal  wreck,  boundless  and  bare 
The  lone  and  level  sands  stretch  far  away. 
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ABSTRACT 

Complexity  is  a  common  phenomenon  existing  in  bio¬ 
logical  organisms,  geological  formations,  and  social 
constructions.  Educational  researchers  and  educational 
practitioners  routinely  encounter  complex  situations  as 
a  function  of  study  and  practice.  Managing  complex 
situations  has  become  a  common  necessity  for  educa¬ 
tional  technology  researchers  to  make  sense  of  compli¬ 
cated  situations;  however,  complexity  as  a  factor  in 
educational  technology  research  tends  to  be  mala- 
ligned,  oversimplified,  or  otherwise  insufficiently 
addressed,  thereby  rendering  the  results  of  many 
research  studies  about  educational  technology  lacking 
in  generalizability.  A  conceptual,  theoretical,  and  prac¬ 
tical  understanding  of  complexity  offers  a  framework 
for  research  on  educational  technology  that  addresses 
the  issues  associated  with  nonlinear  and  complicated 
relationships.  Complexity  theory  is  predicated  on  a 
phenomenon  possessing  five  attributes:  (1)  it  contains 
independent  complicated  entities,  (2)  multiple  entities 
are  contained  within,  (3)  the  entities  within  the  phe¬ 


nomenon  perform  interrelated  functions,  (4)  the  phe¬ 
nomenon  seeks  a  common  goal,  and  (5)  uncertainty  is 
generated  because  of  unpredictable  interactions  within 
itself  and  between  itself  and  the  environment.  A  sum¬ 
mary  of  the  main  conceptual,  theoretical,  and  practical 
aspects  of  complexity  is  presented  as  it  relates  to  edu¬ 
cational  communications  and  technology. 


KEYWORDS 

Complex:  Pertains  to  a  group  of  multiple  independent 
entities  with  interrelated  functions  seeking  a  com¬ 
mon  goal  through  adaptive  processes. 

Complicated:  Consisting  of  many  interconnecting  parts 
or  elements;  intricate;  involving  many  different  and 
confusing  aspects. 

System:  An  interdependent  entity  that  is  responsive, 
systematic,  and  bound  by  the  open  or  closed  nature 
of  the  context  in  which  it  is  located. 

Uncertainty:  Nonlinear  patterns  that  are  usually  the 
result  of  unpredictable  interactions. 
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INTRODUCTION 

Research  designs  in  the  field  of  educational  commu¬ 
nications  and  technology  feature  strategies  intended  to 
reduce  or  control  the  number  of  naturally  occurring 
relationships  during  an  intervention  to  isolate  a  subset 
of  relations  for  further  scrutiny.  Isolating  certain  vari¬ 
ables  and  controlling  selected  relationships  is  common 
practice  for  social  and  behavioral  scientists.  Managing 
the  number  of  variables  is  a  reasonable  practice  for 
educational  technology  researchers  seeking  to  make 
sense  of  complicated  situations.  Educational  technol¬ 
ogy  research  often  involves  the  study  of  complex  enti¬ 
ties  interacting  within  complicated  environments.  A 
complex  phenomenon  is  a  group  of  multiple  indepen¬ 
dent  entities  with  interrelated  functions  seeking  a  com¬ 
mon  goal  through  adaptive  processes. 

Complexity  is  a  common  phenomenon  existing  in 
biological  organisms,  geological  formations,  and 
social  constructions.  Although  a  complex  entity  is  a 
unique  phenomenon,  it  is  important  to  understand  the 
relationships  that  exist  within  it  and  the  relationships 
between  itself  and  the  environment  to  abstract  infor¬ 
mation  common  to  multiple  phenomena,  to  gain  mean¬ 
ing,  and  to  make  generalizations:  “Somewhere 
between  the  specific  that  has  no  meaning  and  the  gen¬ 
eral  that  has  no  content  there  must  be,  for  each  purpose 
and  at  each  level  of  abstraction,  an  optimum  degree  of 
generality”  (Banathy  and  Jenlink,  2003,  p.  38).  Com¬ 
plexity  as  a  factor  in  educational  technology  research 
tends  to  be  malaligned,  oversimplified,  or  otherwise 
insufficiently  addressed,  thereby  rendering  the  results 
of  many  research  studies  about  educational  technology 
lacking  in  generalizability  or  application.  A  concep¬ 
tual,  theoretical,  and  practical  understanding  of  com¬ 
plexity  can  offer  a  guiding  framework  for  research  on 
educational  technology  that  addresses  the  issues  asso¬ 
ciated  with  nonlinear  and  complicated  relationships. 

The  Concept  of  Complexity 

Complexity,  as  an  independent  research  concept, 
emerged  from  the  fields  of  mathematics  and  physics. 
Levy  (1992)  defined  a  complex  system  as  one  whose 
components  interact  with  sufficient  intricacy  such  that 
they  cannot  be  predicted  by  standard  linear  equations. 
Levy  added  that  multiple  variables  interact  nonlinearly 
such  that  the  overall  behavior  of  a  complex  entity  can 
only  be  understood  as  an  emergent  consequence  of  the 
holistic  sum  of  all  the  myriad  behaviors  embedded 
within.  Levy’s  definition  focused  on  dynamic  relation¬ 
ships  and  unpredictable  patterns  in  nonlinear  systems . 
Law  and  Mol  (2001)  claimed  there  is  complexity:  (1) 


if  things  relate  to  each  other  but  do  not  add  up,  (2)  if 
events  occur  but  not  within  the  processes  of  linear 
time,  and  (3)  if  phenomena  share  a  space  but  cannot 
be  mapped  in  terms  of  a  single  set  of  three-dimensional 
coordinates.  What  has  come  to  be  called  complexity 
science  first  arose  in  the  confluence  of  several 
fields  — including  cybernetics,  systems  theory,  artifi¬ 
cial  intelligence,  and  nonlinear  dynamics— many  of 
which  had  begun  to  appear  in  the  physical  sciences  in 
the  mid-20th  century.  Complexity  is  the  concept  used 
to  describe  phenomena  that  produce  increasing  amount 
of  information,  energy,  hierarchy,  variability,  relation¬ 
ship,  and  components,  which  in  turn  increase  the  pos¬ 
sible  outcomes  and  reduce  certainty  and  predictability. 
The  concept  of  complexity  applies  here  because  out¬ 
comes  associated  with  educational  communications 
and  technology  are  commonly  characterized  by 
increasing  amounts  of  information  and  an  increased 
number  of  outcomes. 

Complexity  Theory 

Complexity  implies  that  natural  and  social  systems  are 
nonlinear  and  dynamic.  Education  certainly  classifies 
as  a  nonlinear,  dynamic,  natural,  and  social  system: 
“Complexity  theories  have  come  to  be  taken  up  in  the 
social  sciences  in  many  and  various  ways  ranging  from 
the  highly  technical,  philosophical,  narrative  and  more 
recently  the  applied”  (Davis  et  al.,  2004,  p.  2).  Edu¬ 
cational  technologists  can  arrange  classrooms  in  vari¬ 
ous  configurations,  schedule  mutually  agreeable  meet¬ 
ing  times,  and  prepare  and  sequence  instructional 
materials  to  satisfy  a  linear  order  of  events,  but  even 
educational  technologists  cannot  guarantee  the  same 
linear  order  of  a  student’s  learning  experience  nor 
interaction  patterns  among  peers,  teachers,  media,  and 
context.  Complexity  theory  is  predicated  on  a  phenom¬ 
enon  possessing  five  attributes:  (1)  it  contains  indepen¬ 
dent  complicated  entities,  (2)  multiple  entities  are  con¬ 
tained  within,  (3)  the  entities  within  the  phenomenon 
perform  interrelated  functions,  (4)  the  phenomenon 
seeks  a  common  goal  through  a  process  of  adaptation, 
and  (5)  uncertainty  is  generated  because  of  unpredict¬ 
able  interactions  within  itself  and  between  itself  and 
the  environment.  Complex  phenomena  are  individual 
entities  that  can  be  divided  into  smaller  entities.  Each 
entity  has  its  own  characteristic  and  function  and  can 
be  divided  into  subentities.  The  human  body  is  a  per¬ 
fect  example  of  a  phenomenon  that  possesses  all  five 
attributes  and  is  divided  into  smaller  independent  com¬ 
plicated  entities  such  as  the  head,  trunk,  and  limbs. 
Each  of  the  human  body’s  entities  is  made  of  suben¬ 
tities,  such  as  bone,  tissue,  and  blood.  Likewise,  each 
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subentity  of  the  human  body  can  be  further  divided 
into  all  kinds  of  cells,  and  so  forth.  An  entity  can  be 
deconstructed  into  many  parts  (sometimes  beyond  our 
comprehension),  and  each  part  can  still  be  a  complex 
entity  but  at  a  micro  level.  A  drop  of  water  can  be  as 
complex  as  the  universe;  therefore,  theoretically, 
everything  is  complex  relative  to  the  space  in  which  it 
exists.  A  single  entity  may  be  able  to  fulfill  one  task 
independently  in  a  relatively  simple  context.  A  group 
of  entities  may  be  required  to  fulfill  multiple  tasks 
interdependently  within  a  complex  context. 

Complexity  also  means  that  a  phenomenon  pos¬ 
sesses  multiple  configurations  and  multiple  functions, 
which  is  consistent  with  systems  theory.  Each  part  of 
a  system  relies  on  other  parts  for  information  and 
production  and  therefore  interacts  with  each  other.  The 
human  system  relies  on  entities  such  as  muscle,  bone, 
nerve,  and  blood,  collaborating  with  each  other  to  per¬ 
form  bodily  functions.  A  system  is  synergistic,  where 
the  whole  is  greater  than  the  sum  of  its  parts.  Uncer¬ 
tainty  is  generated  because  of  unpredictable  interac¬ 
tions.  Many  natural  and  social  systems  are  compli¬ 
cated,  and  complexity  theory  can  be  used  as  a 
framework  for  understanding  these  systems.  Complex¬ 
ity  theory  also  provides  a  way  to  examine  complicated 
systems  such  as  educational  communications  and  tech¬ 
nology. 

Complexity  in  Practice 

Educational  technology  is  complex  because  it  is 
formed  by  multiple  interactions  within  itself  and 
between  itself  and  the  environment.  Educational  tech¬ 
nology  is  the  study  and  ethical  practice  of  facilitating 
learning  and  improving  performance  by  creating, 
using,  and  managing  appropriate  technological  pro¬ 
cesses  and  resources  (Association  for  Educational 
Communications  and  Technology,  2007).  Practice, 
within  a  research  context,  refers  to  “interventions  that 
are  designed  solely  to  enhance  the  well-being  of  an 
individual  patient  or  client  and  that  have  a  reasonable 
expectation  of  success”  (DHEW,  1979,  p.  3).  Educa¬ 
tional  technologists  develop  and  implement  products, 
processes,  and  programs  designed  to  promote  the  well 
being  of  students  with  expectations  of  success.  Gagne 
et  al.  (2005)  summarized  nine  instructional  events  to 
describe  optimum  instructional  arrangements  based  on 
a  cognitive  processing  model.  An  instructional  event 
is  a  relatively  small  unit  that  provides  the  external 
conditions  that  complement  a  learner’s  internal  condi¬ 
tion.  Branch  (1999)  defined  a  set  of  instructional 
events  that  form  a  discrete  teaching-learning  session 
as  an  instructional  episode.  An  instructional  episode  is 


an  entity  that  guides  a  learner  toward  defined  knowl¬ 
edge  and  skill.  A  complex  process  is  required  to 
accommodate  the  myriad  of  complicated  variables  and 
relationships  associated  with  educational  technology. 

Learning  refers  to  the  process  of  acquiring  knowl¬ 
edge  and  skill.  Intentional  learning  refers  to  learning 
that  happens  through  purposefully  arranged  informa¬ 
tion,  human  resources,  and  environments  to  achieve  a 
certain  purpose.  Intentional  learning  is  complex 
because  of  the  nature  of  knowledge  and  nonlinear 
interactions  among  multiple  entities.  According  to  You 
(1993),  learning  is  complex  because  knowledge  is  a 
dynamic  system  and  an  active  construction  of  dynamic 
reality  comprised  of  an  interconnected  Web  of  pat¬ 
terns.  Branch  (1999)  called  such  a  dynamic  system  as 
referred  to  by  You  an  intentional  learning  space. 
Branch  identified  eight  entities  that  are  always  present 
within  intentional  learning  space:  student,  content, 
media,  teacher,  peers,  time,  goal,  and  context.  Branch 
purported  that  each  of  these  eight  entities  is  inherently 
complex.  The  student  is  complex  because  of  the  phys¬ 
ical,  emotional,  social,  and  mental  development  of 
human  beings,  as  well  as  the  effect  of  intelligence, 
cognitive  styles,  motivation,  cultural  norms,  creativity, 
and  socioeconomic  status  on  behavior  patterns.  Con¬ 
tent  is  complex  because  it  is  a  collection  of  concepts, 
rules,  propositions,  procedures,  and  socially  con¬ 
structed  information.  Moreover,  the  information  types 
may  be  attribute,  categorical,  classification,  compo¬ 
nent  parts,  dimension,  elaboration,  goal,  hierarchical, 
kinds,  matrix,  prerequisite,  procedural,  rule,  skills,  and 
type.  A  peer  is  a  complex  entity  because  of  all  the 
social  negotiations  that  occur  among  persons  of  the 
same  age,  status,  or  ability.  Media  are  channels  of 
communication  that  come  in  a  multitude  of  forms.  The 
teacher  or  teacher  function  assumes  the  executive  deci¬ 
sion-making  role,  such  as  identifying  appropriate  goals 
and  expectations,  analyzing  learning  needs,  arranging 
content  information,  choosing  media,  selecting 
instructional  methods,  and  conducting  assessments  on 
instruction  and  students.  Time  is  a  complex  entity 
because  it  is  omnipresent  and  can  be  measured  by 
assigning  discrete  increments  and  intervals  but  not 
controlled.  Context  is  the  complex  entity  that  refers  to 
those  conditions  that  directly  and  indirectly  influence 
situations,  environments,  and  communities.  Contex¬ 
tual  conditions  are  formed  by  physical,  political,  eco¬ 
nomical,  and  cultural  settings— in  other  words,  human 
ecology.  Intentional  learning  space  is  the  place  where 
educational  entities  and  nonlinear  behavior  coexist. 
Educational  technologists  study,  experiment,  and  prac¬ 
tice  within  intentional  learning  spaces;  thus,  the  prac¬ 
tice  of  intentional  learning  is  complex. 
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CONCLUSION 

Complexity  is  a  basic  characteristic  of  educational 
technology;  however,  the  aspects  of  complexity  in 
educational  technology  research  tend  to  be  insuffi¬ 
ciently  addressed.  Although  existing  perspectives 
regarding  complexity  provide  foundational  informa¬ 
tion  for  instructional  designers  and  educational  tech¬ 
nologists,  more  study  is  needed  about  nonlinear  pat¬ 
terns  to  understand  education  as  a  complex 
phenomenon. 
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ABSTRACT 

A  review  of  modem  perspectives  on  learning  would  not 
be  complete  without  some  mention  of  the  increasing 
awareness  that  we  now  have  of  the  importance  of  expe¬ 
riential  learning  and  its  relationship  in  developing  not 
only  domain  skills  within  the  learner  but  also  an  under¬ 
standing  of  new  applications  for  such  domain  skills. 
Although  experiential  learning  was  first  proposed  by 
David  Kolb  in  his  1984  paper  titled  Experiential  Learn¬ 
ing:  Experience  as  the  Source  of  Learning  and  Devel¬ 
opment,  the  principles  outlined  in  this  work  can  be 
traced  back  to  the  concept  of  the  craft  apprenticeships 
in  the  Middle  Ages.  Kolb  also  referred  to  an  old  aph¬ 
orism  accredited  to  Confucius  around  450  B.C.:  “Tell 
me,  and  I  will  forget.  Show  me,  and  I  may  remember. 
Involve  me,  and  I  will  understand.”  This  chapter  dis¬ 
cusses  experiential  learning  in  a  modem  perspective 
and  provides  the  reader  with  a  deeper  understanding  of 


the  concepts  associated  with  these  ideas  and  the  likely 
future  directions  for  applying  experiential  leaning 
within  the  context  of  technology-enhanced  lifetime 
learning. 


KEYWORDS 

Active  learning:  Proponents  of  experiential  learning 
often  refer  to  active  learning,  and  by  this  phrase 
they  refer  to  the  participatory  nature  of  experiential 
learning.  Learners  take  an  active  role  in  their  own 
learning,  and  this  often  also  implies  that  learners 
have  to  take  some  responsibility  for  their  own  learn¬ 
ing  and  advancement  of  understanding. 

Apprenticeship:  An  apprenticeship  is  a  traditional  learn¬ 
ing  paradigm  in  which  a  student  learns  by  working 
with  a  master;  this  method  of  education  was  prac¬ 
ticed  widely  in  the  Middle  Ages.  Many  crafts  and 
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skills  involved  a  7-year  period  where  a  student  was 
regarded  as  an  apprentice;  only  after  some  formal 
test,  which  often  involved  the  unsupervised  cre¬ 
ation  of  an  object  or  completion  of  a  task,  was  the 
student  graduated.  A  longer  process  was  required 
before  graduates  could  attain  the  additional  practi¬ 
cal  experience  required  for  recognition  as  a  master 
of  their  crafts  or  trades.  The  concept  of  a  lengthy 
apprenticeship  fell  from  favor  with  the  establish¬ 
ment  of  mass  education  in  the  20th  century. 

Learning  style:  A  learning  style  involves  a  theoretical 
model  that  proposes  that  individuals  have  preferred 
ways  in  which  they  learn.  These  learning  styles 
typically  involve  visual  representation,  vocal 
explanations,  and  practical  examples.  Experiential 
learning  proposes  several  ways  in  which  an  educa¬ 
tor  can  accommodate  these  learning  styles, 
although  all  such  styles  have  a  bias  toward  practical 
expression  and  less  emphasis  on  theoretical  expres¬ 
sions  of  learning. 

Reflection:  Reflection  is  proposed  to  be  a  key  element 
in  the  model  of  experiential  learning  where  the 
learner  has  some  inner  mental  consideration  of  the 
consequences  of  the  actions  they  have  taken  in  prac¬ 
tical  learning.  Some  mental  models  are  implied  that 
are  necessary  for  reflection  to  be  an  effective  ele¬ 
ment  within  learning  in  the  experiential  approach. 


INTRODUCTION 

For  the  sake  of  the  discussion  in  this  chapter  on  expe¬ 
riential  learning,  the  following  terms  are  being  used 
as  indicated  in  the  keyword  definitions  and  below: 

•  Active  learning— Proponents  of  experiential 
learning  often  refer  to  active  learning,  and 
by  this  phrase  they  refer  to  the  participatory 
nature  of  experiential  learning.  Learners  take 
an  active  role  in  their  own  learning,  and  this 
often  also  implies  that  learners  have  to  take 
some  responsibility  for  their  own  learning 
and  advancement  of  understanding. 

•  Apprenticeship— An  apprenticeship  is  a  tra¬ 
ditional  learning  paradigm  in  which  a  student 
learns  by  working  with  a  master.  This  method 
of  education  was  practiced  widely  in  the 
Middle  Ages.  Many  crafts  and  skills  involved 
a  7-year  period  where  a  student  was  regarded 
as  an  apprentice.  Only  after  some  formal  test, 
which  often  involved  the  unsupervised  cre¬ 
ation  of  an  object  or  completion  of  a  task, 
was  the  student  graduated.  A  longer  process 
was  required  before  graduates  could  attain 


the  additional  practical  experience  required 
for  recognition  as  a  master  of  their  crafts  or 
trades.  The  concept  of  a  lengthy  apprentice¬ 
ship  fell  from  favor  with  the  establishment 
of  mass  education  in  the  20th  century. 

•  Learning  style— A  learning  style  involves  a 
theoretical  model  that  proposes  that  individ¬ 
uals  have  preferred  ways  in  which  they  learn. 
These  learning  styles  typically  involve  visual 
representation,  vocal  explanations,  and  prac¬ 
tical  examples.  Experiential  learning  pro¬ 
poses  several  ways  in  which  an  educator  can 
accommodate  these  learning  styles,  although 
all  such  styles  have  a  bias  toward  practical 
expression  and  less  emphasis  on  theoretical 
expressions  of  learning. 

•  Rejection— Reflection  is  proposed  to  be  a 
key  element  in  the  model  of  experiential 
learning  where  the  learner  has  some  inner 
mental  consideration  of  the  consequences  of 
the  actions  they  have  taken  in  practical  learn¬ 
ing.  Some  mental  models  are  implied  that 
are  necessary  for  reflection  to  be  an  effective 
element  within  learning  in  the  experiential 
approach. 

The  concept  of  practice-based  education  had  a  long 
history  before  the  inspirational  paper  by  Kolb  and  Fry 
( 1 975 ) .  In  fact ,  before  the  20th  century  when  educ  ation 
had  a  less  mass  market  approach,  one-to-one  appren¬ 
ticeship  training  was  the  main  method  of  learning  in 
the  arts  or  sciences. 

The  major  change  that  started  in  the  late  nineteenth 
century  was  that  mass  education  had  become  the  major 
method  of  education  with  a  focus  on  the  theoretical  and 
rote  learning  of  set  items  of  information  that  were  pre¬ 
sented  as  facts  (Dewey,  1933).  Lessons  and  assessment 
then  focused  on  the  student  learning  one  single  set  of 
view  or  facts,  and  the  student  was  expected  to  extrap¬ 
olate  these  facts  into  a  systematic  approach  to  a  topic. 
This  learning  of  the  theoretical  aspects  of  a  domain 
made  great  economic  sense  but  tended  to  produce  stu¬ 
dents  who  had  less  practical  experience  and  who  often 
could  not  extrapolate  the  set  knowledge  they  had  been 
taught  into  the  working  world  after  they  had  graduated. 

This  type  of  theoretical  and  rote  learning  of  the 
factual  elements  of  a  topic  was  especially  prevalent  in 
the  early  educational  technology  systems.  Even  the 
early  models  of  simulation-based  learning  environ¬ 
ments  had  weaknesses  in  that  they  did  not  permit  stu¬ 
dents  to  learn  by  practical  experimentation.  In  the  best 
students  this  was  not  a  problem  because  such  gifted 
individuals  were  able  to  learn  a  topic  in  spite  of  the 
way  in  which  the  material  was  presented;  however,  for 
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the  majority  of  students  it  is  more  effective  for  them 
to  be  able  to  experience  the  practical  effects  of  different 
behaviors  and  strategies  in  a  learning  situation  than  to 
passively  observe  them.  This  was  why  the  ideas  pre¬ 
sented  in  David  Kolb’s  papers  (1976, 1981,  1984)  and 
by  his  associate  Roger  Fry  (Kolb  and  Fry,  1975)  were 
a  revolution  in  educational  theory  and  practice. 

The  1975  paper  by  Kolb  and  Fry  stated  a  central 
hypothesis  that  learning  was  made  more  effective  by 
allowing  the  learner  to  experience  the  real-world  situ¬ 
ation  involved  in  the  target  discipline  (Smith,  2001). 
Kolb  and  Fry  (1975)  and  later  Kolb  together  with  other 
colleagues  (Kolb,  1976,  1981,  1984;  Kolb  et  al.,  1995; 
Osland  et  al.,  2007)  argued  that  experiential  learning 
allowed  the  learner  to  make  sense  of  concrete  experi¬ 
ence  and  that  this  concrete  experience  made  enormous 
differences  to  the  effectiveness  and  application  of 
teaching. 

Kolb  and  Fry  (1975)  made  many  references  to 
various  sources  for  their  ideas.  One  of  the  most  con¬ 
vincing  arguments  in  support  of  experiential  learning 
appears  in  Kolb's  influential  paper  Experiential  Learn¬ 
ing:  Experience  as  the  Source  of  Learning  and  Devel¬ 
opment  ( 1 984),  which  made  reference  to  a  quote  attrib¬ 
uted  to  the  Chinese  philosopher  Confucius  (450  B.C.): 

“Tell  me,  and  I  will  forget.  Show  me,  and  I  may  remem¬ 
ber.  Involve  me,  and  I  will  understand.” 

This  attribution,  however,  may  be  apocryphal  as  there 
was  already  a  well-established  history  of  praxis-based 
learning  in  most  traditional  arts,  and  in  many  disci¬ 
plines  (such  as  medicine  and  law)  the  concept  of  a 
lengthy  apprenticeship  has  remained  an  established 
part  of  their  profession. 

Concepts  of  Experiential  Learning 

Having  examined  the  basis  and  historical  perspective 
for  the  idea,  we  can  now  discuss  the  elements  of  expe¬ 
riential  learning  (Borzak,  1981;  Brookfield,  1983; 
Houle,  1980).  Kolb’s  model  has  four  key  elements  in 
any  effective  experiential  learning  situation:  (1)  con¬ 
crete  experience,  (2)  observation  and  reflection,  (3) 
formation  of  abstract  concepts,  and  (4)  testing  in  new 
situations  (Kolb,  1984).  Kolb’s  work  has  referred  to 
the  elements  of  this  learning  cycle  as  activists,  reflec¬ 
tors,  theorists,  and  pragmatists,  respectively. 

There  is  some  discussion  within  the  literature  as 
to  whether  there  is  an  optimum  starting  point  in  the 
proposed  cycle  of  these  stages  of  learning  (see  Figure 
4.1);  however,  it  is  likely  that  many  of  these  stages 
are  not  discrete;  given  the  nature  of  neural  parallelism 
in  the  human  brain,  these  stages  probably  occur 


Concrete  Experience  (1) 


Figure  4.1  Phases  of  experiential  learning. 

simultaneously  while  a  student  is  involved  in  experi¬ 
ential  learning. 

It  is  interesting  that  this  model  involves  several 
stages  that  are  purely  internal  and  conceptual.  A  lot  of 
the  focus  in  experiential  learning  has  been  based  on 
the  objective  physical  stages  of  enabling  practical 
active  experience  with  less  emphasis  on  the  internal 
mental  manipulations  that  Kolb’s  model  proposes. 

It  is  also  appropriate  to  mention  that,  however 
appealing  this  model  appears,  in  fact  very  little  empir¬ 
ical  evidence  supports  or  contests  that  these  phases  of 
learning  are  involved  in  any  actual  learning  situations. 
The  reader  must  therefore  realize  when  dealing  with 
the  topic  of  experiential  learning  that  it  is  more  akin 
to  a  teaching  philosophy  than  a  science  of  education. 

One  of  the  key  claims  for  experiential  learning  is 
how  it  effectively  allows  the  student  to  learn  the  key 
aspects  of  a  subject  and  then  extrapolate  those  key 
concepts  to  different  situations  and  act  independently 
(Mezirow,  1997;  Tennant,  1997).  Again  the  reader 
should  recognize  that  experiential  learning  theories  do 
not  provide  detailed  advice  about  how  an  educator 
should  organize  experiential-based  educational  activi¬ 
ties  to  anticipate  the  best  way  or  approach  to  teach  any 
specific  type  of  topic  or  subject. 

Because  experiential  learning  is  more  of  a  teaching 
philosophy  than  a  scientific  theory,  it  is  important  for 
the  reader  to  recognize  that  there  is  very  little  evidence 
from  cognitive  science  that  provides  theoretical  or 
objective  confirmation  or  denial  of  the  key  tenets  of 
experiential  learning.  It  is  clear,  however,  that  the 
actual  experience  of  participation  has  a  very  strong 
role  within  experiential  learning.  Although  the  tenets 
of  experiential  learning  are  not  necessarily  supported 
within  any  cognitive  or  neural  science  explanations  of 
learning,  this  may  be  a  fertile  area  of  further  investi¬ 
gation  for  researchers  interested  in  exploring  links 
between  neuroscience  and  learning. 
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TABLE  4.1 

Experiential  Learner  Types 

Converger 

Abstract  conceptualization  and 
active  experimentation 

Strong  in  practical  application  of 
ideas;  deductive  reasoning; 
unemotional;  high  levels  of 
concentration 


Diverger 

Concrete  experience  and  active 
experimentation 

Strong  imaginative  ability;  good 
at  new  ideas  and  different 
perspectives;  social  and  group 
orientation 


Assimulator 

Abstract  concepts  and  reflective 
observation 

Theoretical  models;  inductive 
reasoning;  abstract  concepts;  not 
thought  to  be  a  social  person 


Accommodator 

Concrete  experience  and  active 
experimentation 
Doing;  risk  taking;  immediate 
response;  intuitive  problem 
solving 


Source:  Tennant,  M.,  Psychology  and  Adult  Learning ,  2nd  ed.,  Routledge,  London,  1997.  With  permission. 


LEARNING  STYLES  AND 
EXPERIENTIAL  LEARNING 

The  original  papers  that  form  the  basis  for  experiential 
learning  make  clear  the  importance  the  influence  of 
Piaget  (Jean  Piaget  Archives  Foundation,  1989).  It 
should  not  be  surprising  to  find  that  a  major  aspect  of 
experiential  learning  is  the  idea  of  learning  styles: 
converger,  diverger,  assimilator,  and  accommodator 
(Table  4.1)  (Kolb,  1984).  These  styles  provide  some 
practical  guidance  for  how  to  structure  learning  activ¬ 
ities  so  as  to  match  the  preferred  learning  style  of  the 
students.  Unfortunately,  no  definitive  guide  exists  for 
how  to  determine  the  learning  style  for  a  student,  and 
again  it  must  be  recognized  that  relatively  little  empir¬ 
ical  evidence  exists  to  support  these  proposed  learning 
styles,  nor  is  it  recognized  that  educators  themselves 
will  have  preferred  teaching  styles,  which  must  also 
be  taken  into  account  for  truly  effective  learning  to 
take  place.  On  the  positive  side,  however,  the  concept 
of  these  learning  styles  does  remove  a  single  dimen¬ 
sion  of  learning  such  as  social  skills  or  intelligence 
and  also  shows  the  strengths  and  weaknesses  of  any 
single  learning  style.  Tennant  (1997)  elaborated  basic 
learning  styles  within  experiential  learning  as  shown 
in  Table  4.1 . 


DIFFERENT  USES  OF 
EXPERIENTIAL  LEARNING 

The  researcher  will  often  find  that  there  are  two  ways 
in  which  the  term  experiential  learning  may  be  encoun¬ 
tered  in  the  literature  related  to  technology-enhanced 
learning  (TEL).  The  first  is  as  a  way  of  learning  that  is 
a  direct  experience  of  real  life  (Borzak,  1981;  Brook¬ 
field,  1983).  This  would  be  in  contrast  to  the  second 
and  more  common  use  of  the  term  within  TEL  as  the 


chance  to  experience  a  simulation  of  the  various  tasks, 
communications,  and  activities  that  are  associated  with 
a  particular  topic  of  learning  (Houle,  1980).  Both  of 
these  uses  of  the  term  are  different  from  the  study  of 
the  theoretical  aspects  of  a  topic  or  its  history  or  a 
highly  formalized  and  abstract  series  of  steps  as  the 
more  traditional  ways  in  which  a  topic  is  taught. 

Further  Developments 
in  Experiential  Learning 

Weil  and  McGill  (1989)  proposed  four  interpretations 
or  villages  associated  with  experiential  learning  within 
the  literature: 

•  Village  One  assesses  and  accredits  learning 
from  life  and  work  experience. 

•  Village  Two  focuses  on  experiential  learning 
as  the  basis  for  bringing  change  into  the 
structures  of  post  school  education. 

•  Village  Three  emphasizes  experiential  learn¬ 
ing  as  a  basis  for  group  consciousness  rais¬ 
ing. 

•  Village  Four  is  concerned  about  personal 
growth  and  self  awareness. 

Although  these  definitions  are  useful  it  is  important  to 
realize  that  the  major  use  of  experiential  learning 
within  TEL  will  be  for  practical  work-related  simula¬ 
tions  that  are  closely  linked  to  real-world  practice. 

These  uses  of  the  term  do  not  make  problematic 
the  notion  of  experience  itself.  Jarvis  (1994,  1995) 
makes  a  case  for  how  experiential  learning  theories 
can  cope  with  learning  through  secondary  or  indirect 
experience,  such  as  through  communication.  For 
example,  it  is  obvious  that  learners  can  develop  con¬ 
siderable  knowledge  and  expertise  simply  by  reading, 
and  little  mention  is  made  of  this  within  the  experien¬ 
tial  learning  literature. 
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CRITICISMS 

As  already  mentioned,  one  of  the  more  major  criti¬ 
cisms  of  experiential  learning  is  the  lack  of  empirical 
validity  for  the  theory  or  the  learning  styles  that  have 
been  proposed.  Further,  strong  evidence  from  neuro¬ 
science  suggests  that  any  stages  of  learning  would  be 
largely  simultaneous ,  so  the  ideas  of  fixed  stages  must 
be  regarded  as  being  for  the  ease  of  designing  learn¬ 
ing  situations  or  for  explaining  the  central  concepts 
behind  the  theory.  There  has  also  been  some  criticism 
that  the  experiential  learning  literature  pays  insuffi¬ 
cient  attention  to  the  process  of  reflection  (Boud  and 
Miller,  1983;  Schon,  1983).  In  addition,  little  consid¬ 
eration  is  given  to  how  social  or  cultural  issues  can 
be  included  within  the  experiential  learning  model 
(Anderson,  1988;  Brookfield,  1983;  Fraser,  1995; 
Johnson  and  Johnson,  2002).  These  critiques  of  expe¬ 
riential  learning  may  serve  as  the  basis  for  further 
research  in  this  area. 


CONCLUDING  REMARKS 

In  spite  of  the  criticisms,  based  on  this  review  of  the 
literature  it  is  clear  that  the  central  concepts  within 
experiential  learning  contain  some  extremely  valuable 
principles  for  simulation-based  learning  in  TEF.  Fur¬ 
ther  research  should  take  the  concepts  and  begin  to 
address  how  the  various  ideas  within  experiential 
learning  could  be  empirically  tested  to  allow  deeper 
insights  into  how  the  findings  from  neuroscience  could 
be  applied  to  technology-enhanced  learning  so  as  to 
maximize  effective  learning  in  digital  environments. 
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ABSTRACT 

This  chapter  is  concerned  with  empirical  research  on 
memory  and  motivation  within  the  realm  of  educational 
communication  and  technology.  A  variety  of  research 
methods  and  methodologies  are  described,  with  empha¬ 
sis  on  the  accessibility  of  theoretical  constructs  such  as 
memory  and  motivation  and  the  validity  and  reliability 
of  assessments.  The  abundance  of  issues  surrounding 


the  topics  of  memory  and  motivation  is  narrowed  to  a 
few  which  appear  repeatedly  in  the  literature  and  are 
dominant  in  the  domains  under  discussion  in  current 
instructional  psychology.  Several  theoretical 
approaches  to  the  architecture  of  cognition  and  models 
of  human  memory  are  described  and  discussed  with 
regard  to  the  question  of  how  experiences  with  media 
and  human  memory  interact.  Accordingly,  cognitive 
load  theory  and  dual-code  processing,  as  well  as  the 
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theories  of  schemata  and  mental  models,  are  reviewed 
critically  and  evaluated  with  regard  to  their  advantages 
and  drawbacks  for  learning  with  media.  This  chapter 
furthermore  introduces  relevant  motivational  research 
issues  concerning  technology-enhanced  learning. 
After  a  description  of  several  theoretical  approaches 
concerning  relevant  motivational  factors  (such  as  flow 
and  self-efficacy),  the  empirical  research  dealing  with 
the  question  of  how  media  and  motivation  interact  are 
critically  reviewed  and  evaluated. 


KEYWORDS 

Information  processing:  The  modeling  of  sensory 
input  and  cognitive  transformations  as  a  series  of 
processing  stages. 

Memory:  The  mental  faculty  of  retaining  and  recalling 
past  experiences. 

Mental  model:  A  mental  representation  that  people  use 
to  organize  their  experience  about  themselves,  oth¬ 
ers,  the  environment,  and  the  things  with  which 
they  interact;  its  functional  role  is  to  provide  pre¬ 
dictive  and  explanatory  power  for  understanding 
these  phenomena. 

Schemata:  Data  structures  for  representing  both 
generic  and  specific  knowledge. 


INTRODUCTION 

We  do  not  leam  only  through  experience;  rather,  media 
and  technological  artifacts  also  play  a  central  role  in 
the  formation  of  our  knowledge  of  the  world.  Accord¬ 
ingly,  learning  with  media  has  been  at  the  core  of 
instructional  research  for  more  than  six  decades,  and 
there  is  not  any  medium  or  feature  of  a  medium  that 
has  not  been  investigated  with  regard  to  its  effective¬ 
ness  on  learning.  The  use  of  media  may  change  the 
characteristic  features  of  learning  environments,  bring¬ 
ing  about  effects  on  cognitive  operations,  representa¬ 
tional  formats,  interactivity,  visualization  of  semantic 
structures,  and  feedback.  More  specifically,  the  per¬ 
ceptual  organization  of  messages  affects  how  learners 
encode  information  because  it  is  responsible  for  the 
nature  of  the  mental  representations  that  learners  con¬ 
struct  as  a  result  of  interaction  with  media  of  commu¬ 
nication  (Seel  and  Winn,  1997). 

Media  that  combines  visual  and  auditory  presen¬ 
tation  modes  is  called  multimedia ,  and  much  research 
has  been  done  on  how  people  process  audio-visual 
information.  This  research  on  audio-video  redundancy 
has  led  to  the  development  of  several  theoretical 
approaches  over  the  past  decades.  Although  Severin’s 


(1967)  cue-summation  theory  was  a  promising 
approach  in  the  1960s,  Paivio’s  (1971)  dual-code  the¬ 
ory  emerged  in  the  1970s  and  was  then  combined  with 
Sweller’s  (1988)  cognitive  load  theory  (CLT)  in  the 
1980s.  Both  dual-code  theory  and  CLT  operate  with 
the  stage  model  of  memory,  which  presupposes 
sequential  information  processing.  In  contrast,  cogni¬ 
tive  psychologists  emphasize  parallel  information  pro¬ 
cessing  and  functional  capabilities  that  a  memory  sys¬ 
tem  requires  to  support  performance  in  a  broad  range 
of  cognitive  tasks.  Important  movements  are  the  levels 
of  processing  approach  (Craik  and  Lockhart,  1972), 
parallel  distributed  processing  (PDP)  models,  as  well 
as  connectionist  models  of  information  processing 
(McClelland  et  al„  1986).  Finally,  dual-process  mem¬ 
ory  models  have  influenced  the  understanding  of  rec¬ 
ognition  memory  over  the  past  30  years.  Dual-process 
memory  models  distinguish  between  a  recall-like  pro¬ 
cess  in  which  episodic  information  is  retrieved  at  the 
time  of  recognition  and  a  fluency-based  process  in 
which  general  familiarity  is  used  as  a  basis  for  recog¬ 
nition  (Kelley  and  Jacoby,  2000).  Dual-process  models 
focus  on  the  relationships  between  semantic  and  epi¬ 
sodic  memory,  for  which  research  shows  the  impor¬ 
tance  of  subjectivity  in  remembering.  Indeed,  memo¬ 
ries  are  constructions  made  in  accordance  with  present 
needs  and  desires,  and  they  are  often  accompanied  by 
feelings  and  emotions  (Schacter,  1996).  This  corre¬ 
sponds  with  studies  that  show  that  multimedia  learning 
can  enhance  the  motivation  of  students  (Cheung  et  al., 
2003).  Consequently,  more  researchers  are  proposing 
connecting  motivational  and  cognitive  features  of 
interaction  with  multimedia  (Hede,  2002). 

This  chapter  describes  the  major  lines  of  research 
on  memory  and  motivation  within  the  realm  of  multi- 
media  learning.  It  follows  the  current  discussion  about 
the  need  for  sound  models  of  multimedia  learning  in 
which  cognitive  and  motivational  factors  of  informa¬ 
tion  processing  are  integrated  to  achieve  a  better  and 
instructionally  relevant  understanding  of  the  effective¬ 
ness  of  learning. 


METHODOLOGICAL  ISSUES 

Theoretical  constructs  such  as  memory,  learning,  and 
motivation  are  scientific  inventions  that  serve  to 
describe  phenomena  that  cannot  be  observed  and 
therefore  must  be  inferred  from  observable  data.  Cog¬ 
nitive  psychology  has  at  its  disposal  a  great  variety  of 
methods  for  assessing  cognition,  ranging  from  natu¬ 
ralistic  observation  to  computer  simulation,  from 
experimental  methods  to  verbal  protocols,  from 
recording  electrical  impulses  of  the  brain  to  collecting 
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reaction  times  (Simon  and  Kaplan,  1989).  Corre¬ 
spondingly,  we  can  find  two  major  methodologies  of 
memory  research  based  on  these  methods.  The  first 
methodology  corresponds  to  a  physical  approach  and 
applies  psychophysical  methods  (such  as  EEG  or 
PET)  to  assess  mental  states.  The  other  methodology 
corresponds  with  functional  aspects  of  cognition  and 
infers  mental  operations  from  observable  behavior. 
The  functional  methodology  is  at  the  core  of  experi¬ 
mental  methods.  It  contains  traditional  tests,  question¬ 
naires,  the  time  needed  for  learning,  the  frequency 
and  type  of  errors  in  performing  tasks,  drawings,  eye 
fixations  during  task  accomplishment,  and  so  on.  In 
natural  settings,  verbal  communication  is  convenient 
for  mediating  ideas,  thoughts,  and  feelings.  Taking 
into  account  the  prime  importance  of  language  for 
communication,  it  is  evident  that  different  methods  of 
verbalization  play  a  central  role  in  the  diagnosis  of 
cognition.  Psychologists  often  consider  think-aloud 
protocols  as  an  adequate  method  for  verbalizing 
thoughts,  explanations,  inferences,  speculations,  and 
justifications.  From  a  methodological  perspective,  an 
important  effect  of  the  infusion  of  information  tech¬ 
nologies  into  education  consists  in  the  potential  of  the 
computer  to  assess  knowledge  and  cognitive  skills. 
Computer-based  approaches  to  cognitive  modeling  are 
considered  effective  methods  for  assessing  concepts, 
ideas,  and  thoughts  concerning  a  particular  subject 
matter  domain;  however,  these  methods  presuppose 
special  tools  (e.g.,  concept  maps)  for  the  assessment 
of  knowledge. 


LEARNING  AND  MEMORY 
Theories  of  Human  Memory 

Theories  are  constructed  to  explain  complex  concepts 
and  ideas.  They  are  often  difficult  to  talk  about  because 
they  are  not  clear  cut  or  concrete.  Memory,  for 
instance,  is  a  fairly  abstract  concept,  and  a  number  of 
theories  have  been  developed  to  explain  how  people 
store  information  in  such  a  way  that  it  can  be  retrieved 
when  needed.  Although  all  of  these  theories  describe 
the  basic  function  of  memory  as  enabling  the  retention 
of  information  and  personal  experiences  and  the  recall 
of  that  information  and  those  experiences,  there  is  no 
universally  accepted  model  of  human  memory.  Often, 
models  of  human  memory  are  based  on  common-sense 
assumptions  about  how  information  is  processed  and 
stored. 

A  popular  model  of  memory  is  the  stage  model  of 
information  processing  (Atkinson  and  Shiffrin,  1968). 
This  model  proposes  three  stages  of  information  pro¬ 


cessing:  sensory  memory,  short-term  memory,  and 
long-term  memory.  Information  is  thought  to  be  pro¬ 
cessed  in  a  serial,  discontinuous  manner  as  it  moves 
from  one  stage  to  the  next.  A  central  component  of  a 
modified  version  of  the  stage  model  is  working  mem¬ 
ory  with  a  limited  processing  capacity  (Baddeley  and 
Hitch,  1974).  This  idea  goes  back  to  Ebbinghaus 
(1885)  and  Miller  (1956),  who  argued  that  humans  can 
deal  with  about  7  ±  2  elements  of  information  at  a  time 
without  overloading  working  memory. 

Several  alternative  theories  of  information  pro¬ 
cessing,  however,  do  not  correspond  with  the  stage 
model;  for  example,  the  Soar  architecture  is  a  theory 
of  cognition  that  also  presupposes  the  notion  of  a 
working  memory  but  in  a  different  sense  than  the 
stage  model  (Newell,  1990).  In  Soar,  the  working 
memory  is  a  more  elaborate  temporary  memory  struc¬ 
ture  that  holds  information  pertaining  to  the  state  of 
the  current  problem-solving  context.  The  functions 
of  working  memory  are  distributed  across  multiple 
components  of  the  architecture,  including  long-term 
production  memory,  which  enables  the  cognitive  sys¬ 
tem  to  handle  complex  tasks  that  require  large  quan¬ 
tities  of  information  by  relying  on  recognition-based 
long-term  working  memory. 

The  levels-of-processing  (LOP)  approach,  dual¬ 
process  memory  models,  and  connectionist  models  do 
not  include  the  conception  of  working  memory;  rather, 
the  LOP  approach  argues  that  all  stimuli  that  activate 
a  receptor  cell  are  permanently  stored  in  memory  but 
different  levels  of  processing  contribute  to  the  ability 
to  access  or  retrieve  information  from  memory.  Con¬ 
nectionist  models,  which  are  currently  at  the  core  of 
cognitive  psychology  (McLeod  et  al„  1998),  and  PDP 
models  suggest  that  information  is  stored  in  multiple 
locations  throughout  the  brain  in  the  form  of  networks 
of  connections.  At  a  molar  level,  however,  PDP  models 
operate  with  the  theoretical  constructs  of  schemata  and 
mental  models  to  explain  human  information  process¬ 
ing  (Rumelhart  et  al.,  1986). 

The  fact  that  memory  is  studied  in  a  range  of 
disciplines  often  leads  to  academic  suspicion.  Educa¬ 
tional  psychologists  focus  on  the  stage  model  of  mem¬ 
ory,  whereas  cognitive  psychologists  focus  on  parallel 
associative  memory.  The  question  comes  up  as  to 
whether  educational  psychologists  and  cognitive  sci¬ 
entists  are  really  addressing  the  same  phenomena  when 
they  focus  on  human  memory. 

Cognitive  Multimedia  Learning  and  Memory 

An  analysis  of  the  literature  on  multimedia  learning 
shows  that  mainstream  educational  psychology 
agrees  on  a  cognitive  architecture  that  corresponds  to 
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Multimedia  Sensory  Working  Long-Term 

Presentation  Memory  Memory  Memory 


Figure  5.1  The  cognitive  theory  of  multimedia  learning.  (From  Mayer,  R.E.,  Multimedia  Learning,  Cambridge  University  Press, 
Cambridge,  U.K.,  2002.  With  permission.) 


the  stage  model  of  information  processing  (Reed, 
2006).  In  this  model,  restrictions  in  the  processing  of 
information  occur  at  specific  points,  especially  within 
the  working  memory  with  its  limited  capacity.  In 
accordance  with  this  assumption,  Mayer  (2002)  has 
presented  a  cognitive  theory  of  multimedia  learning 
in  which  he  also  refers  to  Paivio’s  (1971)  dual-code 
theory  of  information  processing.  Paivio  suggests  that 
there  are  two  cognitive  channels,  one  specialized  in 
processing  nonverbal  objects  by  means  of  imagery 
and  the  other  specialized  in  dealing  with  language. 
Learning  is  better  when  information  is  processed 
through  both  channels  instead  of  only  one,  but  the 
working  memory  limits  the  amount  of  information 
that  can  be  processed  in  each  channel  at  one  time. 
The  capacity  may  be  increased  through  the  simulta¬ 
neous  use  of  both  channels  when  the  corresponding 
visual  and  verbal  representations  are  contiguously 
present  in  working  memory.  In  addition,  by  creating 
connections  between  corresponding  verbal  and  visual 
information  along  with  relevant  prior  knowledge,  the 
learner  organizes  this  information  into  a  coherent  rep¬ 
resentation  both  verbally  and  visually.  The  complete 
model  is  described  in  Figure  5.1. 

Because  working  memory  can  only  process  lim¬ 
ited  amounts  of  new  information,  an  overload  can 
result  if  the  learning  task  is  too  complex.  This  is  the 
central  assumption  of  the  cognitive  load  approach 


(Sweller,  1988).  Cognitive  load  may  be  affected  by 
the  mental  effort  necessary  to  process  new  informa¬ 
tion  ( intrinsic  cognitive  load),  the  manner  in  which 
the  material  is  presented  ( extraneous  cognitive  load), 
or  the  effort  required  for  activating  schemata  ( ger¬ 
mane  cognitive  load).  Earlier,  Berlyne  (1971)  had 
already  introduced  the  simple  idea  of  information 
overload  and  suggested  that  the  more  elements  a 
pattern  contained,  the  more  complex  that  pattern 
would  be.  The  cognitive  load  approach  deals  with 
complexity  using  a  single  construct:  element  inter¬ 
activity.  If  many  elements  interact,  element  interac¬ 
tivity  is  high;  if  few  interact,  element  interactivity  is 
low.  To  bypass  the  limitations  of  working  memory, 
cognitive  load  theory  stresses  the  activation  of  sche¬ 
mata  that  may  encapsulate  numerous  elements  of 
information  in  a  single  chunk.  The  activation  of 
schemata  allows  automatic  processing  and  thus  min¬ 
imizes  the  load  of  working  memory.  Skilled  perfor¬ 
mance  develops  through  the  construction  of  an 
increasing  number  of  ever  more  complex  and 
abstract  schemata  (Sweller,  1994). 

Cognitive  load  theory  is  an  instructional  design 
theory  that  aims  to  reduce  the  cognitive  load  caused 
by  poorly  designed  learning  tasks  (Sweller  et  al., 
1998),  whereas  Mayer’s  cognitive  theory  of  multime¬ 
dia  learning  can  be  easily  integrated  into  the  held  of 
research  on  audio-video  redundancy. 
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Results  of  Research 
on  Audio-Video  Redundancy 

Many  attempts  employing  the  concept  of  audio-video 
redundancy  have  been  made  to  understand  how  people 
integrate  and  learn  information  presented  verbally  and 
visually— for  example,  through  a  text  enriched  with 
static  pictures  or  through  dynamic  pictures  (animation 
or  video)  accompanied  by  narration.  This  research  on 
how  people  process  audiovisual  information  has  high¬ 
lighted  many  complexities  and  inconsistencies  (Lang, 
1995);  for  example,  people  have  better  short-term 
recall  of  auditory  than  of  visual  information  (Penney, 
1989),  and  they  need  narration  to  receive  effective 
instruction  from  animation  (Mayer  and  Anderson, 
1991).  Furthermore,  Nugent  (1992)  found  increased 
achievement  for  combinations  of  audio  and  images  but 
not  for  audio  and  text,  whereas  other  studies  (Mayer 
and  Gallini,  1990)  revealed  that  visual  cues  amplify 
and  explain  text  and  facilitate  the  recall  of  new  knowl¬ 
edge  because  visual  cues  create  imagery  during  learn¬ 
ing  that  is  critical  to  memory  processes.  Obviously, 
information  across  two  channels  can  serve  as  recipro¬ 
cal  reinforcement  and  enhance  both  recall  and  com¬ 
prehension  (Levie  and  Lentz,  1982). 

These  observations  correspond  with  the  cue-sum¬ 
mation  theory ,  which  states  that  learning  is  increased 
as  the  number  of  available  stimuli  are  increased  (Sev- 
erin,  1967).  Multiple-channel  communications  are 
superior  to  single-channel  communications  when  rel¬ 
evant  cues  are  summated  across  channels,  are  neither 
superior  nor  inferior  when  the  cues  provided  on  the 
two  channels  are  redundant,  and  are  inferior  when 
irrelevant  cues  are  combined,  presumably  because 
irrelevant  cues  cause  interference  between  the  two 
channels.  If  the  stimuli  provided  on  different  channels 
are  not  reciprocally  relevant,  the  distraction  causes  a 
decrease  rather  than  an  increase  in  learning  and  reten¬ 
tion  (Brashears  et  al.,  2005). 

Another  issue  is  the  perception  of  complex  visual¬ 
izations,  which  can  be  difficult  to  perceive  and  interpret. 
Several  studies  (Lowe,  2003;  Tversky  et  al.,  2002)  have 
shown  that  perceptual  features  of  visualizations  can 
interfere  with  successful  comprehension.  Although 
Lowe  found  an  advantage  for  predictions  drawn  from 
animation,  this  advantage  was  limited  to  perceptually 
salient  features.  However,  novice  learners  often  were 
distracted  by  perceptually  salient  features  of  an  anima¬ 
tion  at  the  expense  of  more  important  content  informa¬ 
tion.  Thus,  the  interpretation  and  use  of  visualizations 
may  be  affected  greatly  by  perceptual  qualities  of  the 
visualization.  In  a  comprehensive  study  on  the  effects 
of  static  and  dynamic  complexity  on  children’s  attention 
to  and  recall  of  televised  instruction.  Watt  and  Welch 


( 1983)  showed  that  increasing  static  detail  on  the  screen 
by  using  complex  sets,  elaborate  graphics,  and  long 
shots  decreases  recall  while  leaving  recognition  unaf¬ 
fected,  whereas  increases  in  visual  dynamic  complexity 
produce  decreases  in  recall  coupled  with  increases  in 
recognition.  An  analysis  of  the  entire  viewing  process 
revealed  that  visual  attention  was  only  of  minor  impor¬ 
tance  in  the  learning  process,  whereas  static  and 
dynamic  complexities  were  found  to  have  different 
effects  on  learning  involving  a  verbal  recall  process  than 
learning  requiring  only  visual  recognition.  Although  the 
visual  attention  was  not  significant  in  this  study  on  tele¬ 
vision  viewing,  Mayer  and  Moreno  (1998)  found  that 
presenting  instruction  in  both  auditory  and  visual  modes 
can  cause  a  split-attention  effect  in  which  students  have 
to  divide  their  attention  across  multiple  inputs, resulting 
in  reduced  processing  of  information.  These  findings 
were  explained  from  the  perspective  of  cognitive  load 
theory  (CLT),  which  is  at  the  core  of  current  educa¬ 
tional  psychology  (Paas  and  van  Gog,  2006) .  Although 
the  results  of  CLT  research  are  important  for  instruc¬ 
tional  design,  there  are  some  limitations  from  both  a 
methodological  and  a  theoretical  perspective.  Advo¬ 
cates  of  the  cognitive  load  approach,  such  as  Mayer 
and  Moreno  (1998),  have  pointed  out  that  the  limita¬ 
tions  of  CLT  studies  include  the  fact  that  the  partici¬ 
pants  were  usually  college  students;  however,  numer¬ 
ous  studies  indicate  that  the  capacity  of  memory  and 
the  application  of  memory  search  strategies  increase 
with  development  (Harris,  1978).  Evidently,  adults  can 
refer  to  more  perceptual  experiences  and  can  use  more 
effective  strategies  for  the  retrieval  of  knowledge  rep¬ 
resentations  than  children  (Lindberg,  1980;  Seel,  1984). 
Another  limitation  of  CLT  studies  is  that  the  instruc¬ 
tional  episodes  have  been  short  and  only  how-it-works 
material  has  been  used  (Mayer  and  Moreno,  1998). 

Alternative  Theories  and  Predictions 

Cognitive  load  theory  and  the  dual-code  theory  pre¬ 
suppose  a  working  memory  with  a  limited  capacity. 
The  notion  of  a  working  memory  refers  to  computa¬ 
tional  mechanisms  that  maintain  and  provide  access  to 
information  to  be  retrieved  during  the  performance  of 
a  task.  Any  computational  system  must  support  such 
functionality  because  computation  is  inherently  a  pro¬ 
cess  that  requires  the  temporary  storage  of  informa¬ 
tion;  however,  in  cognitive  science  reaction  times 
longer  than  6500  msec  are  treated  as  outliers  of  the 
working  memory  (e.g.,  Reijnen  et  al.,  2005). 

The  question  comes  up  whether  theoretical 
approaches  of  educational  psychology,  such  as  CLT  or 
dual-code  theory,  are  compatible  with  theories  on  infor¬ 
mation  processing  within  the  realm  of  cognitive  science. 
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Actually,  Young  and  Lewis  (1999)  argue  that  complex 
cognitive  tasks  simply  cannot  be  performed  with  a  stor¬ 
age  capacity  of  7  ±  2  items.  A  problem-solving  task 
involving  inner  speech,  for  example,  may  occur  over 
tens  of  seconds  or  even  minutes  according  to  thought¬ 
monitoring  studies  (Franklin  et  al.,  2005).  To  capture 
the  involvement  of  memory  in  dealing  with  complexity, 
Ericsson  and  Kintsch  (1995)  introduced  the  notion  of  a 
long-term  working  memory ,  and  Logie  (1995)  identified 
several  conceptions  of  working  memory,  including  the 
idea  of  working  memory  as  controlled  attention.  This 
corresponds  with  the  argumentation  of  Shiffrin  and 
Schneider  (1977)  that  individual  differences  on  mea¬ 
sures  of  cognitive  limitations  primarily  reflect  differ¬ 
ences  in  the  ability  of  controlled  processing.  It  seems 
that  memory  theories  of  cognitive  science  are,  to  a  large 
extent,  incompatible  with  the  stage  model  and  related 
theories;  however,  if  alternative  theories  exist  and  make 
differing  predictions,  we  can  test  a  prediction  of  the 
theory  and  a  prediction  of  a  competing  theory  at  the 
same  time.  When  the  two  predictions  are  incompatible, 
a  strong  inference  (Platt,  1964)  results.  In  contrast  to 
CLT,  dual-process  models,  for  example,  argue  that  con¬ 
scious  recollection  and  familiarity  contribute  to  mem¬ 
ory  performance.  Memory  judgments  are  made  by  set¬ 
ting  some  level  of  familiarity  as  a  response  criterion  and 
accepting  items  that  exceed  this  criterion.  A  comparison 
of  each  information  unit  to  long-term  memory  produces 
a  continuous  familiarity  value  that  is  used  to  make  rec¬ 
ognition  judgments.  This  process  can  be  appropriately 
explained  by  signal-detection  theory. 

Another  example  is  schema  construction,  which 
plays  an  important  role  in  CLT.  Bransford  (1984)  has 
pointed  out  that  schema  activation  and  schema  con¬ 
struction  are  two  different  problems.  Although  it  is 
possible  to  activate  existing  schemata  with  a  given 
topic,  it  does  not  necessarily  follow  that  a  learner  can 
use  this  activated  knowledge  to  develop  new  knowledge 
and  skills.  Furthermore,  a  major  criticism  of  schema 
theories  is  that  they  are  basically  assimilation  models 
that  fail  to  answer  questions  on  how  existing  concep¬ 
tions  are  modified  in  the  face  of  inconsistent  input  and 
how  such  theories  deal  with  novelty  (Brown,  1979). 

Learning  and  Remembering 
in  Cognitive  Psychology 

Whereas  educational  psychologists  (Clark,  2006) 
emphasize  the  importance  of  automated  knowledge  for 
learning  and  memory,  cognitive  psychologists  focus 
on  the  role  of  consciousness  in  information  processing 
because  much  of  human  cognition  functions  by  means 
of  continuous  interactions  between  contents  of  infor¬ 
mation  and  various  memory  systems.  Franklin  and 


colleagues  (2005)  call  these  interactions  cognitive 
cycles.  Human  computational  ability  has  been  esti¬ 
mated  at  10  cognitive  cycles  per  second,  where  each 
cycle  is  able  to  recognize  or  select  information. 
Although  these  cycles  can  overlap,  producing  parallel 
actions,  they  must  preserve  consciousness.  This  corre¬ 
sponds  with  the  basic  assumption  of  cognitive  psychol¬ 
ogy  that  humans  store  information  in  memory  by  relat¬ 
ing  new  information  to  what  is  already  known.  This 
process  is  semantic  in  nature  because  information  is 
stored  in  terms  of  its  meaning,  as  defined  by  its  asso¬ 
ciations  to  other  information  in  the  memory.  The 
capacity  for  such  storage  is  essentially  unlimited,  and 
the  cognitive  system  can  handle  complex  tasks  that 
require  large  quantities  of  information  by  relying  on  a 
recognition-based  long-term  memory,  which  works  in 
concert  with  the  external  environment. 

Humans  have  some  basic  abilities  that  are  essential 
for  processing  information  and  acting  successfully  in 
different  environments.  According  to  Rumelhart  and 
colleagues  (1986),  one  of  these  abilities  is  that  humans 
are  very  good  at  pattern  matching  to  quickly  settle  on 
an  interpretation  of  any  input  pattern.  Times  for  object 
recognition,  counting,  and  selection  tasks  vary  from  25 
to  170  msec  (with  70  msec  as  the  average).  Potter  and 
Levy  (1969)  cited  ranges  from  50  msec  to  300  msec  for 
the  processing  of  single  pictures  with  low  to  high  levels 
of  visual  noise,  and  their  studies  demonstrate  that  the 
accuracy  of  visual  recognition  improves  as  display  times 
increase  from  125  msec  (16%  accuracy)  to  1000  msec 
(80%  accuracy).  A  second  basic  ability  of  humans  is 
that  they  are  very  good  at  modeling  their  worlds;  that 
is,  they  can  anticipate  the  new  state  of  affairs  resulting 
from  actions  or  from  an  event  they  observe  .  This  ability 
is  based  on  building  up  expectations  and  assumptions 
and  is  crucial  for  inferential  learning  (Seel  ,1991).  Third , 
humans  are  good  at  manipulating  their  environments. 
This  can  be  considered  as  a  version  of  “man  the  tool 
user”  and  is  perhaps  the  crucial  skill  in  forming  a  cul¬ 
ture.  Especially  important  here  is  the  ability  to  manip¬ 
ulate  the  environment  so  it  comes  to  represent  some¬ 
thing.  Rumelhart  and  colleagues  (1986)  argue  that  these 
three  basic  cognitive  abilities  involve  the  interplay 
between  two  units  of  memory:  an  interpretation  net¬ 
work ,  which  corresponds  to  the  activation  of  schemata , 
and  the  construction  of  a  model  of  the  world. 

Parallel  Distributed  Processing, 

Schemata,  and  Mental  Models 

In  cognitive  psychology,  schemata  are  defined  as  large- 
scale  slot-filler  structures  that  play  critical  roles  in  the 
interpretation  of  input  data,  the  guiding  of  action,  and 
the  storage  of  knowledge  in  memory  (Anderson, 


44 


Empirical  Perspectives  on  Memory  and  Motivation 


Figure  5.2  Schemata  and  mental  models  as  means  of  assimilation  and  accommodation.  (From  Seel,  N.M.  et  al.,  Technol.  Instruct. 
Cognit.  Learn.,  4(4),  273-299,  2006.  With  permission.) 


1984).  PDP  models,  however,  do  not  work  with  con¬ 
cepts  such  as  schemata  but  rather  conceptualize,  con¬ 
sistent  with  recent  brain  research,  memory  by  means 
of  constraint  networks  resulting  from  the  sum  of  exci¬ 
tatory  and  inhibitory  influences  on  a  unit.  Input  comes 
into  the  cognitive  system  and  activates  a  set  of  units 
that  tend  to  act  together  as  response  to  certain  patterns 
of  the  input.  These  tightly  interconnected  units  corre¬ 
spond  to  what  has  been  called  a  schema  in  cognitive 
psychology.  A  schema  is  the  state  that  maximizes  the 
particular  set  of  constraints  acting  at  a  given  moment. 
This  conception,  of  course,  contradicts  the  conven¬ 
tional  story  that  schemata  are  stored  in  memory.  In 
PDP  models,  they  are  the  major  content  of  memory, 
but  nothing  stored  corresponds  to  a  schema.  What  is 
stored  is  a  set  of  connection  strengths  which,  when 
activated,  contains  the  ability  to  generate  mental  states 
that  correspond  to  an  instantiated  schemata. 

Similarly,  mental  models  consist  of  relaxation  net¬ 
works ,  which  take  as  input  some  specification  of  the 
actions  we  intend  to  carry  out  and  produce  an  inter¬ 
pretation  of  what  would  happen  if  we  did  that.  It  is 
not  necessary  for  world  events  to  have  really  happened. 
In  the  case  that  they  have  not,  the  cognitive  system 
replaces  the  inputs  from  the  world  with  inputs  from  a 
model  of  the  world.  In  PDP  models,  schemata  and 
mental  models  are  defined  at  a  molar  level  of  cogni¬ 
tion,  at  which  individuals  organize  symbols  of  experi¬ 
ence  or  thought  in  such  a  way  that  they  form  a  sys¬ 
tematic  representation  of  this  experience  or  thought  as 
a  means  of  understanding. 

Mental  models  and  schemata  are  basic  formats  of 
mental  representation  that  fulfill,  in  terms  of  Piaget’s 
epistemology,  the  functions  of  assimilation  and  accom¬ 
modation.  Schemata  regulate  the  assimilation  of  new 
information  into  cognitive  structures.  If  a  schema  does 


not  fit  with  a  new  task  it  can  be  adjusted  to  meet  the 
task  requirements  by  means  of  accretion,  tuning,  or 
reorganization.  If  this  is  not  successful,  accommoda¬ 
tion  must  take  place  for  the  individual  to  reorganize 
the  available  knowledge  to  construct  a  mental  model 
(Seel,  1991).  This  can  be  illustrated  as  in  Figure  5.2. 

Mental  models  are  dynamic  representations  con¬ 
structed  to  serve  several  functions,  such  as  (1)  the 
simplification  of  an  investigation  to  particular  and  rel¬ 
evant  phenomena  in  a  domain,  (2)  the  envisioning  of 
complex  structures  (to  make  visible  the  invisible),  (3) 
analogical  reasoning,  and  (4)  mental  simulation  in  the 
sense  of  thought  experiments  (Seel,  1991).  There  are 
two  procedures  for  coming  to  a  mental  model:  either 
by  means  of  constructing  an  analogy  with  the  known 
or,  if  this  is  not  possible,  by  integrating  relevant  bits 
of  generic  knowledge  into  a  coherent  structure  to  meet 
the  requirements  of  a  phenomenon  to  be  explained. 
Both  procedures  require  quite  a  lot  of  time  and  mental 
effort.  A  central  component  of  modeling  consists  in 
the  process  of  fleshing  out  (Johnson-Laird,  1983), 
which  can  be  understood  as  the  step-by-step  comple¬ 
tion  of  an  initial  model  to  a  more  complete  model. 
While  a  schema  is  a  slot-filler  structure,  a  mental 
model  contains  assumptions  that  must  be  justified  by 
observations.  The  justification  of  the  assumptions  is 
connected  with  a  reduction  to  absurdity  (Seel,  1991). 
This  is  a  process  of  testing  continuously  whether  a 
model  can  be  replaced  with  a  better  model.  As  long  as 
this  is  not  possible,  the  model  is  considered  as  suitable. 

Research  on  Schemata  and  Mental  Models 

The  schema  concept  has  been  an  influential  construct 
of  cognitive  psychology  since  the  early  1970s  and  has 
inspired  an  abundance  of  studies  in  the  field  of  text  and 
picture  processing  as  well  as  television  viewing.  In 
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many  studies,  the  role  of  human  memory  has  been 
ignored,  but  some  researchers  (e.g..  Watt  and  Welch, 
1983)  have  referred  explicitly  to  the  interplay  among 
perception,  attention,  and  memory.  In  these  studies,  it 
became  evident  that  both  the  context  and  complexity 
of,  for  example,  pictorial  information  can  play  an 
important  role  for  learning  and  remembering.  Friedman 
(1979)  found  that  unexpected  (i.e.,  schema-dissonant) 
objects  embedded  in  a  complex  scenario  can  be  recalled 
better  than  expected  objects  because  dissonant  infor¬ 
mation  evokes  a  focus  of  attention  that  is  not  necessary 
in  the  case  of  schema-congruent  information. 

The  research  on  memory  for  schema-relevant  infor¬ 
mation  has  produced  contradictory  results.  Some  stud¬ 
ies  demonstrate  memory  selectivity  for  schema-consist¬ 
ent  information,  whereas  others  demonstrate  memory 
selectivity  for  schema-inconsistent  information.  A 
meta-analysis  by  Rojahn  and  Pettigrew  (1992)  on  a 
sample  of  60  independent  studies  with  165  comparative 
tests  suggested  a  slight  overall  memory  advantage  for 
schema- inconsistent  information.  In  addition,  a  study 
by  Seel  (1984)  demonstrated  that  learners’  attention  to 
local  information  within  a  complex  scenario  as  well  as 
the  processing  of  this  information  and  its  retrieval  from 
memory  are  dependent  on  age-related  development. 
First-graders  could  encode  the  global  and  local  infor¬ 
mation  of  complex  scenarios  by  referring  to  scene  sche¬ 
mata,  but  they  could  not  recall  the  pictorial  details  with 
a  quantity  and  quality  comparable  to  older  children  and 
adults.  Visual  attention  runs  from  a  small  area  with  high 
resolution  to  a  broad  area  with  a  loss  of  details,  and 
focusing  attention  on  a  specific  object  can  lead  to  a 
neglect  of  other  objects  (Shaw,  1982).  Obviously,  the 
salience  of  specific  local  information  plays  a  central 
role  for  comprehension  and  remembering.  Finally,  the 
study  by  Seel  (1984)  demonstrated  that  the  content  of 
complex  scenarios  can  evoke  emotional  reactions  with 
a  resulting  focal  attention  on  salient  features  of  the 
scenarios  due  to  a  novelty  effect. 

Text  and  picture  processing  has  also  been  a  central 
issue  of  mental  model  research  since  its  beginning  in 
the  1980s.  The  effects  of  graphical  diagrams  or  even 
helpful  video  on  the  construction  of  mental  models 
have  been  investigated  (Hegarty  and  Just,  1993;  Sharp 
et  al.,  1995).  Evidently,  diagrams  of  complex  technical 
systems,  for  example,  proved  to  be  effective  not  only 
for  the  representation  of  the  complex  real  scenarios 
but  also  for  the  simulation  of  their  processes;  however, 
the  success  of  such  media  presupposes  that  the  learner 
is  semantically  sensitive  to  their  model-relevant  char¬ 
acteristics.  The  concept  of  semantic  sensitivity  was 
introduced  by  Anzai  and  Yokoyama  (1984),  who 
argued  that  individuals  spontaneously  respond  to  the 
information  from  the  environment  to  generate  a  basic 


understanding  of  it.  Numerous  studies  demonstrate  the 
semantic  sensitivity  of  learners  acting  in  learning  envi¬ 
ronments  that  demand  the  construction  of  a  mental 
model  (Anzai  and  Yokoyama,  1984;  Seel,  1995). 

Despite  these  promising  results,  several  authors 
(Snow,  1990)  have  criticized  the  early  research  on 
mental  models  because  it  was  regularly  done  piece¬ 
meal  in  small-scale,  specialized  contexts.  In  many  of 
these  early  studies  (Kieras  and  Bovair,  1984),  the  fol¬ 
lowing  method  was  applied.  After  an  initial  training  in 
which  the  subjects  had  to  generate  an  initial  mental 
model  and  again  after  an  experimentally  varied  learn¬ 
ing  phase,  the  subjects  performed  specific  tasks  con¬ 
sidered  indicative  of  successful  model  construction. 
Since  the  1990s,  however,  alternative  approaches  have 
emerged  that  focus  on  instructional  research  on  con¬ 
structing  and  revising  mental  models.  These 
approaches  of  model-oriented  learning  and  instruction 
follow  an  understanding  of  learning  as  an  active,  con¬ 
structive,  and  cumulative  process,  which  has  been 
described  by  Norman  (1981,  p.  284)  as  follows: 

I  have  estimated  that  experts  at  a  task  may  spend  5000 
hours  acquiring  their  skills:  that  is  not  such  a  long  time; 
it  is  2-1/2  years  of  full-time  study,  40  hours  a  week,  50 
weeks  a  year.  Not  much  time  to  become  a  professional 
tennis  player,  or  computer  programmer,  or  linguist.  What 
goes  on  during  that  time?  Whatever  it  is,  it  is  slow, 
continuous.  No  magic  dose  of  knowledge  in  form  of  pill 
or  lecture.  Just  a  lot  of  slow,  continual  exposure  to  the 
topic,  probably  accompanied  by  several  bouts  of  restruc¬ 
turing  of  the  underlying  mental  representations,  recon¬ 
ceptualizations  of  the  concepts,  plus  many  hours  of  accu¬ 
mulation  of  large  quantities  of  facts. 

Accordingly,  approaches  to  model-oriented  learning 
and  teaching  emphasize  long-time  learning  and  the 
accumulation  of  knowledge.  Model-oriented  teaching 
focuses  on  the  patterns  of  participation  and  discourse 
in  the  classroom,  where  learners  construct  their  under¬ 
standing  of  some  phenomenon  (Buckley  and  Boulter, 
1999).  This  is  accomplished  mainly  with  the  help  of 
external  representations,  provided  and  guided  by  the 
teacher  or  a  multimedia  program.  Model-based  learn¬ 
ing,  on  the  other  hand,  focuses  on  the  construction  of 
mental  models  of  the  phenomena  under  study.  When 
the  model  is  used  successfully,  it  is  reinforced  and  may 
eventually  become  a  precompiled,  stable  model  or 
even,  after  many  repetitions,  a  schema.  If  the  model  is 
not  satisfactory,  it  may  be  revised  or  rejected  in  a 
progression  of  mental  models. 

At  the  moment,  several  research  groups  are  con¬ 
cerned  with  model-oriented  learning  and  teaching 
(Lesh  and  Doerr,  2003;  Penner,  2001).  This  line  of 
research  is  characterized  by  references  to  the  traditional 
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use  of  models  in  math  and  science  instruction,  but  it 
does  not  refer  to  theories  and  research  on  human 
memory. 

Alternatively,  my  own  studies  (Seel  et  al.,  2000) 
correspond  with  PDP  models  of  information  process¬ 
ing  and  are  concerned  with  complex  problem  solving 
and  the  learning-dependent  progression  of  mental 
models.  The  main  results  of  different  replication  stud¬ 
ies  (in  total  with  more  than  600  students  of  various 
ages  and  backgrounds)  can  be  summarized  as  follows: 

•  Mental  models  are  not  fixed  structures  that 
can  be  retrieved  from  memory  but  are  con¬ 
structed  when  needed  to  master  the  specific 
demands  of  a  new  learning  situation. 

•  Students  dynamically  modify  and  restruc¬ 
ture  their  initial  mental  models  when  they 
evaluate  externally  provided  information  as 
being  more  plausible  and  convincing  than 
their  prior  knowledge.  This  can  be  inter¬ 
preted  as  an  indicator  of  the  learners’ 
semantic  sensitivity  with  regard  to  relevant 
information  from  the  environment.  The 
learning  environment  serves  as  an  informa¬ 
tion  resource  from  which  the  learners 
extract  the  information  they  need  to  con¬ 
struct  an  explanatory  model. 

•  With  regard  to  the  stability  and  change  of 
mental  models,  the  various  studies  agreed  on 
the  observation  that  mental  models  are 
highly  situation  dependent.  Although  they 
were  not  constructed  independently  from 
each  other  at  various  points  of  measurement, 
their  structures  were  obviously  different. 
Naturally,  it  was  cognitively  less  demanding 
for  the  students  to  construct  new  models  at 
each  point  of  measurement  than  to  remem¬ 
ber  previously  constructed  models. 

•  Mental  models  proved  to  be  effective  for 
mentally  simulating  the  relevant  properties 
of  complex  systems  to  be  cognitively  mas¬ 
tered  (Seel  et  al.,  2006). 

The  results  of  these  studies  indicate  that  the  time 
needed  for  successful  model-based  learning  was  not 
sufficient  with  regard  to  the  emergence  of  a  schema, 
defined  here  as  an  automated  routine  that  can  be  used 
to  access  and  apply  stored  knowledge  structures  with¬ 
out  mental  effort. 

The  instructional  research  on  mental  models  has 
highlighted  several  complexities  and  consistencies. 
One  consistency  is  concerned  with  the  development  of 
a  new  methodology  for  assessing  the  learning-depen- 
dent  progression  of  mental  models.  The  principles  of 


this  methodology  include  embedding  the  diagnosis  of 
mental  models  in  a  complex  problem  situation,  col¬ 
lecting  data  in  a  longitudinal  design,  providing  valid 
and  reliable  quantitative  data,  and  enabling  a  method¬ 
ologically  straightforward  analysis  and  interpretation 
of  the  data  collected  (Seel  et  al.,  2007).  At  the  core  of 
this  methodology  are  the  SMD  (Surface,  Matching, 
Deep  Structure)  Technology  and  MIToCaR  (Model 
Inspection  Trace  of  Concepts  and  Relations).  These 
can  be  combined  with  alternative  methods  for  assess¬ 
ing  mental  models,  such  as  think- aloud  protocols, 
structure-formation  techniques,  concept-mapping 
tools,  and  causal  diagrams,  all  of  which  have  been 
applied  successfully  in  mental  model  research  (Seel, 
1999).  Another  consistency  of  mental  model  research 
consists  in  new  instructional  approaches  for  improving 
the  construction  and  refinement  of  effective  mental 
models  (Seel,  2003). 

Summary  and  Criticism 

Modem  psychology  regards  the  human  being  as  play¬ 
ing  an  active  and  selective  role  in  approaching  each 
new  environment.  Cognitive  theorists  place  much 
greater  emphasis  on  the  fact  that  individuals  bring  to 
each  environment  preestablished  schemata  based  on 
previous  experiences.  These  schemata  have  been  built 
up  in  the  course  of  many  previous  interactions  with 
the  environment  as  well  as  further  reflections  upon 
these  experiences  during  mental  rehearsal.  Some  of 
these  schemata  are  more  complex,  integrated,  orga¬ 
nized,  or  differentiated  than  others;  however,  although 
the  activation  of  schemata  plays  a  central  role  in  dif¬ 
ferent  psychological  approaches,  such  as  the  cognitive 
load  theory,  we  still  do  not  know  enough  about  the 
construction  of  schemata.  For  example,  it  is  not  clear 
under  which  conditions  mental  models  merge  to  sche¬ 
mata.  Recent  research  (Seel  et  al.,  2006)  has  demon¬ 
strated  that  it  takes  a  lot  of  time  and  repetitions  of 
successful  construction  of  mental  models  within  a 
domain  until  they  become  precompiled  and  stable  and 
merge  to  a  schema. 

All  of  the  theoretical  conceptions  and  related 
research  that  have  been  considered  in  the  foregoing 
sections  are  concerned  with  structures  and  functions 
of  the  semantic  memory;  however,  in  cognitive  psy¬ 
chology  a  distinction  has  been  made  between  semantic 
and  episodic  memory  as  specializations  of  the  declar¬ 
ative  memory. 

Episodic  Memory 

A  simple  definition  of  episodic  memory  is  that  it 
encompasses  everything  we  remember  as  opposed  to 
everything  we  know.  The  latter  is  usually  called 
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semantic  memory ,  upon  which  episodic  memory  is 
believed  to  be  built.  A  memory  of  the  last  family 
vacation  is  an  episodic  memory;  however,  if  someone 
asks  what  color  a  ripe  banana  is,  semantic  memory 
is  used  to  answer.  The  notion  of  episodic  memory 
was  first  proposed  some  30  years  ago  by  Tulving  and 
Thomson  (1973).  At  that  time  episodic  memory  was 
defined  in  terms  of  materials  and  tasks.  Today,  epi¬ 
sodic  memory  is  understood  as  a  neurocognitive  sys¬ 
tem,  uniquely  different  from  other  memory  systems 
that  enable  human  beings  to  remember  past  experi¬ 
ences  (Tulving,  2002).  It  includes  time,  place,  and 
associated  emotions  that  affect  the  quality  of  the 
memorization.  Interestingly,  an  autobiographical 
memory  has  been  introduced  as  a  specialization  of 
the  episodic  memory.  It  is  related  with  the  investiga¬ 
tion  everyday  memory,  which  refers  to  memory  oper¬ 
ations  that  routinely  occur  in  one’s  daily  environment, 
and  eyewitness  testimony,  which  has  inspired  a  lot  of 
research  into  its  accuracy  (Loftus,  1975).  Evidently, 
human  memory  for  events  is  fragile  and  susceptible 
to  distortion. 

Although  the  notion  of  episodic  memory  plays  a 
central  role  in  research  on  human  memory,  it  does  not 
play  this  role  in  the  cognitive  architectures  for  multi- 
media  learning.  Indeed,  in  the  most  recent  overview 
by  Reed  (2006),  neither  episodic  memory  nor  autobio¬ 
graphical  memory  is  mentioned.  Nevertheless,  we  can 
learn  from  the  theoretical  conceptions  and  research  on 
this  topic  that  memories  are  constructions  made  in 
accordance  with  present  needs,  desires,  social  influ¬ 
ences,  etc.  Accordingly,  memories  are  often  accompa¬ 
nied  by  strong  feelings  and  emotions  that  increase  the 
recall  of  the  stored  events  from  episodic  memory. 
Finally,  memory  usually  involves  awareness  of  the 
memory  (Schacter,  1996). 

Memory  is  the  retention  of,  and  ability  to  recall, 
information,  personal  experiences,  and  procedures.  A 
memory  system  is  fairly  universally  agreed  to  have 
three  high-level  activities:  encoding,  storage,  and 
retrieval.  Most  lost  memories  are  lost  because  they 
were  never  elaborately  encoded.  Perception  is  mostly 
a  filtering  and  fragmenting  process .  Our  interests  and 
needs  affect  perception,  but  most  of  what  is  available 
to  us  as  potential  sense  data  will  never  be  processed, 
and  most  of  what  is  processed  will  be  forgotten.  We 
do  not  forget  simply  to  avoid  being  reminded  of 
unpleasant  things.  We  forget  because  we  either  did  not 
perceive  closely  in  the  first  place  or  did  not  encode  the 
experience  (Schacter,  1996);  thus,  a  central  feature  of 
human  memory  is  the  subjectivity  of  remembering. 
This  leads  to  the  next  section  of  this  chapter,  where 
motivational  aspects  of  multimedia  learning  are 
emphasized. 


LEARNING  WITH  MEDIA 
AND  MOTIVATION 

Motivation  is  a  fairly  difficult  area,  and  a  number  of 
theories  have  been  developed  to  try  to  explain  why 
people  behave  in  the  ways  that  they  do  and  predict  or 
guess  what  people  actually  will  do  on  the  basis  of  these 
theories.  Although  motivation  is  generally  considered 
a  key  variable  in  learning,  it  is  surprising  that  so  little 
attention  is  apparently  paid  to  studies  on  motivation  in 
multimedia  learning  as  well  as,  vice  versa ,  to  learning 
and  cognition  in  the  motivation  literature.  Indeed,  an 
analysis  by  Elliott  and  Dweck  (2005)  shows  that, 
among  more  than  5000  citations,  no  reference  is  made 
to  cognitive  load  theory  and  the  stage  model  of  infor¬ 
mation  processing.  On  the  other  side,  Lowe  (2003), 
for  instance,  divorced  the  motivational  aspects  of  ani¬ 
mation  from  its  instructive  power.  It  seems  apparent 
that  there  are  two  large  groups  of  researchers  who  are 
not  making  much  use  of  each  other’s  insights. 

As  a  consequence,  we  can  find  only  a  few  theoret¬ 
ical  approaches  that  aim  at  integrating  motivation  into 
comprehensive  models  of  multimedia  learning  (Astle- 
itner  and  Wiesner,  2004;  Samaras  et  al.,  2006);  how¬ 
ever,  these  models  lack  substantial  empirical  evidence, 
and  future  research  is  necessary  to  analyze  the  inter¬ 
relations  of  the  motivational  and  cognitive  variables. 
The  infusion  of  motivation  in  multimedia  learning  is 
a  complex  challenge.  Motivation  might  be  intrinsic, 
extrinsic,  mixed,  or  absent  altogether.  There  is  some 
evidence  that  intrinsic  motivation  encourages  the 
learner  to  become  cognitively  engaged  with  the  mul¬ 
timedia  material  and  improves  learning. 

Nevertheless,  extrinsic  motivational  factors  such  as 
the  design  features  of  a  multimedia  package  are  thought 
to  provide  some  initial  incentive  for  learners  to  access 
the  material,  but  sustained  effort  occurs  only  when  they 
encounter  intrinsic  motivational  factors  provided  by 
interesting  and  challenging  content  (Najjar,  1998).  This 
may  lead  to  cognitive  engagement,  which  is  the  process 
whereby  learners  become  motivated  to  take  full  control 
of  their  own  learning.  Generally,  it  has  been  argued  that 
multimedia  and  computers  have  the  capacity  to  allow 
for  external  regulation  and  autonomy  support  (Stefanou 
et  al.,  2004).  Technology  may  also  provide  for  context 
and  variety  in  learning  tasks  that  theoretically  could  be 
exploited  to  situate  motivation.  Hede  (2002)  sees  the 
various  motivational  factors  as  impacting  learner  con¬ 
trol  or,  more  specifically,  the  time  and  effort  learners 
devote  to  engaging  with  multimedia.  Beyond  Hede’s 
model,  there  are  three  approaches  to  integrating  moti¬ 
vation  into  multimedia  learning:  (1)  schema-based 
approaches,  (2)  flow-based  approaches,  and  (3)  the 
uses-and-gratifications  approach. 
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Schema-Based  Approaches  to  Motivation 
in  Multimedia  Learning 

In  correspondence  with  Piaget’s  (1952)  theory  of  dise¬ 
quilibrium,  a  class  of  theoretical  models  postulates 
strong  relationships  between  cognition  and  emotions. 
Eckblad  (1981),  for  example,  argued  that  each  resis¬ 
tance  for  assimilating  new  information  into  existing 
schemata  evokes  affects  and  emotions  of  a  specific 
nature.  Similarly,  Stein  and  Levine  (1991)  argued  that 
attempts  to  assimilate  new  information  into  schemata 
are  associated  with  emotional  experiences.  When  the 
incoming  information  is  novel,  it  causes  a  mismatch 
with  existing  schemata  and  results  in  arousal  of  the 
autonomic  nervous  system.  When  this  arousal  occurs  in 
conjunction  with  a  cognitive  appraisal  of  the  situation, 
a  more  or  less  specific  emotional  reaction  occurs.  Plans 
or  anticipatory  schemata  are  not  only  specific  to  situa¬ 
tions;  they  also  involve  search  and  selection  strategies 
related  to  the  kind  of  information  to  be  processed.  When 
the  strategies  for  processing  new  materials  and  specific 
anticipations  match  well  with  the  information  we  actu¬ 
ally  confront  in  a  new  setting,  we  generally  experience 
positive  affects,  joy,  or  a  general  sense  of  well-being. 
If,  however,  our  anticipatory  schemata  or  plans  for  pro¬ 
cessing  information  are  inadequate,  we  may  experience 
negative  emotions  (Singer  and  Singer,  1983). 

The  basic  assumption  of  schema-based  research  on 
motivation  is  that  deep  comprehension  occurs  when 
learners  confront  contradictions,  anomalous  events, 
obstacles  to  goals,  salient  contrasts,  perturbations,  sur¬ 
prises,  equivalent  alternatives,  and  other  stimuli  or 
experiences  that  fail  to  match  schema-based  expecta¬ 
tions  (Mandler,  1999).  Cognitive  disequilibrium  has  a 
high  likelihood  of  activating  conscious  and  effortful 
cognitive  deliberation,  questions,  and  inquiry  that  aim 
to  restore  cognitive  equilibrium.  The  affective  states 
of  confusion  and  perhaps  frustration  are  likely  to  occur 
during  cognitive  disequilibrium.  Recent  research 
(Rozin  and  Cohen,  2003)  has  indeed  pointed  to  con¬ 
fusion  as  an  important  affective  state  for  learning.  Con¬ 
fusion  indicates  an  uncertainty  about  what  to  do  next 
or  how  to  act;  thus,  confusion  often  accompanies  cog¬ 
nitive  disequilibrium.  Similarly,  states  of  perturbation 
and  hesitation  often  indicate  the  need  for  clarification 
or  more  information. 

In  four  experiments,  Schutzwohl  (1998)  investi¬ 
gated  the  effects  of  stimuli  discrepant  with  schemata  of 
varying  strength  on  three  components  of  surprise:  (1) 
the  interruption  of  ongoing  activities  (indexed  by 
response  time  increase),  (2)  the  focusing  of  attention  on 
the  schema-discrepant  event  (indexed  by  memory  per¬ 
formance),  and  (3)  the  feeling  of  surprise  (indexed  by 
self-reports).  Response  times  were  consistently  found 


to  increase  with  schema  strength.  This  effect  was  attrib¬ 
uted  to  the  increasing  difficulty  of  schema  revision.  In 
contrast,  memory  for  the  schema-discrepant  event  was 
not  affected  by  schema  strength,  supporting  the  hypoth¬ 
esis  that  schema-discrepant  stimuli  are  stored  in  mem¬ 
ory  with  a  distinct  tag.  Finally,  self-reports  of  surprise 
intensity  varied  with  schema  strength  only  if  they  were 
made  immediately  after  the  surprising  event  without  any 
intervening  questions,  suggesting  that  self-reports  of 
surprise  are  highly  susceptible  to  memory  distortions. 

Emotional  and  motivational  aspects  of  learning 
and  remembering  are  also  discussed  in  connection  with 
cognitive  load  theory;  for  example.  Early  (2006) 
argued  that  it  may  be  useful  to  separate  factors  that 
affect  learner  motivation  into  distinct  groups:  the  vari¬ 
ables  or  values  of  variables  that  increase  learner  moti¬ 
vation  and  the  variables  or  values  that  decrease  or  fail 
to  influence  learner  motivation.  Custers  and  Aarts 
(2005)  have  identified  positive  affect  as  an  example  of 
a  variable  value  that  increases  future  goal  pursuit  via 
choice.  The  memory  of  the  pleasurable  sensation  asso¬ 
ciated  with  prior  goal  pursuit  may,  over  time,  become 
automated  and  lead  to  an  active  choice  of  novel  goals; 
however,  excess  burden  on  working  memory  may 
serve  as  an  example  of  a  variable  value  that  decreases 
goal  pursuit  by  decreasing  effort  or  persistence. 

This  corresponds  with  Berlyne’s  (1971)  precept 
that  another  process  at  the  other  extreme  of  information 
overload  is  boredom.  Overly  simple  visual  fields,  for 
example,  may  lead  to  the  uncomfortable  feeling  of 
boredom.  One  response  to  boredom  with  the  external 
environment  is  to  switch  to  internal  information  pro¬ 
cessing  (for  example ,  to  daydream) .  Another  interesting 
experiment  is  that  of  Ashcraft  and  Kirk  (2001),  who 
investigated  the  impact  of  anxiety  on  working  memory. 
Individuals  with  high  math  anxiety  demonstrated  a 
reduced  working  memory  capacity,  which  led  to  a  pro¬ 
nounced  increase  in  reaction  time  and  errors  when  men¬ 
tal  addition  was  performed  concurrently  with  a  memory 
load  task.  The  effects  of  the  reduction  also  generalized 
to  a  working-memory-intensive  transformation  task. 
Overall,  the  results  demonstrated  that  anxiety  affects 
performance  in  tasks  and  that  this  effect  can  be  inter¬ 
preted  as  a  transitory  disruption  of  working  memory. 

Flow,  Engagement,  and  Self-Efficacy 

According  to  the  constructivist  theoretical  frame¬ 
works,  a  person’s  affective  states  are  expected  to  sys¬ 
tematically  influence  how  they  process  new  material. 
The  intrinsic  motivation  literature  has  identified  affec¬ 
tive  states  such  as  curiosity  as  indicators  of  motivation 
level  and  learning.  Learners  with  more  intrinsic  moti¬ 
vation  and  interest  display  greater  levels  of  pleasure. 
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more  active  involvement  in  tasks  (Tobias,  1994),  more 
task  persistence  and  lower  levels  of  boredom  (Miser- 
andino,  1996),  and  less  anxiety  and  anger  (Patrick  et 
al.,  1993).  Furthermore,  because  a  person’s  affective 
state  is  linked  to  their  motivation  level,  intrinsically 
motivated  learners  who  are  engaged  should  demon¬ 
strate  more  engagement  and  persistence  in  performing 
tasks.  A  deeper  understanding  of  the  materials  should 
be  one  important  consequence. 

In  an  exploratory  study,  Craig  and  colleagues 
(2004)  investigated  the  role  of  several  affective  states 
(frustration,  boredom,  flow,  confusion,  eureka,  and 
neutral)  that  potentially  occur  during  learning  with  an 
intelligent  tutoring  system  for  dialogs  in  natural  lan¬ 
guage.  Observational  analyses  revealed  significant 
relationships  between  learning  and  the  affective  states 
of  boredom,  flow,  and  confusion.  Other  research  sug¬ 
gests  that  interest  increases  as  anxiety  decreases  and 
attitude  improves  and  that  such  motivation  assists  the 
achievement  of  self-efficacy  in  virtual  learning  envi¬ 
ronments  (Dyck  and  Smither,  1994). 

These  findings  indicating  that  learning  correlates 
negatively  with  boredom  and  positively  with  engage¬ 
ment  are  consistent  with  predictions  from  Csikszent- 
mihalyi’s  (1975)  anaysis  of  flow  experiences.  Flow 
describes  a  state  of  complete  absorption  or  engagement 
in  an  activity  and  refers  to  the  optimal  experience. 
During  optimal  experiences,  individuals  are  in  a  psy¬ 
chological  state  where  they  are  so  involved  with  the 
goal-driven  activity  that  nothing  else  seems  to  matter. 
Past  research  has  shown  that  the  flow  state  has  a  pos¬ 
itive  impact  on  learning  (Salanova  et  al.,  2006;  Webster 
et  al.,  1993)  and  should  be  taken  into  account  when 
designing  learning  materials.  The  flow  theory  is  used 
by  Kiili  (2005)  as  a  framework  for  facilitating  positive 
user  experience  and  engagement  to  maximize  the 
impact  of  digital  learning  environments.  The  purpose 
of  the  study  was  to  support  an  active  learning  process, 
creative  participation,  and  learner  engagement  in 
learning  with  a  multimedia  environment.  In  addition, 
Kiili  integrated  assumptions  of  cognitive  load  theory 
into  a  participatory  multimedia  learning  model  in 
which  learners  had  to  produce  learning  materials  by 
themselves.  The  proposed  model  was  studied  through 
an  educational  game  with  university-level  students. 
Questionnaires  and  interviews  revealed  that  content 
creation  was  reported  as  being  the  main  activity  caus¬ 
ing  flow.  Additionally,  a  positive  connection  between 
flow  and  learning  was  found. 

Another  promising  field  of  research  that  is  related 
to  flow  and  engagement  includes  such  topics  as  emo¬ 
tions  and  self-efficacy.  There  is  considerable  evidence 
regarding  the  positive  effects  of  self-efficacy  on  per¬ 
formance  and  well-being  in  different  domains,  such  as 


the  workplace,  school,  and  sports  (Bandura,  1997). 
Self-efficacy  has  shown  its  power  as  a  buffer  in  stress 
processes,  and  it  has  also  been  associated  with  better 
health,  better  self-development,  and  greater  social  inte¬ 
gration.  In  addition,  there  is  also  evidence  regarding 
the  effects  of  self-efficacy  on  multimedia  learning;  for 
example,  Reid  (2006)  developed  an  integrated  model 
of  playfulness  and  flow  in  a  virtual  reality.  Key  ele¬ 
ments  in  the  model  include  self-efficacy  and  volition, 
which  the  author  sees  as  necessary  characteristics  for 
influencing  flow  and  playfulness  for  activity  in  a  vir¬ 
tual  reality.  More  specifically,  Reid  studied  how  flow 
is  related  to  playfulness.  The  findings  supported  pre¬ 
vious  research  on  self-efficacy  and  volition  as  children 
and  adults  reported  that  being  in  control  was  important 
and  stressed  the  value  of  being  able  to  do  new  things. 
Playfulness  was  expressed  as  the  ability  to  feel  pres¬ 
ence  with  the  activity. 

More  generally,  it  is  evident  that  self-efficacy  can 
be  enhanced  through  experiences  with  technology. 
Cheung  and  colleagues  (2003),  for  example  ,  examined 
the  effects  of  a  multimedia  learning  system  on  the  self- 
efficacy  of  learners  and  found  that  software  usefulness 
and  software  ease  of  use  affected  the  self-efficacy  of 
students.  Furthermore,  longer  usage  of  the  multimedia 
learning  system  led  to  improvements  in  self-efficacy 
in  learning  but  did  not  make  the  improvement  faster. 
Self-efficacy  also  depends  on  several  personal  factors, 
such  as  anxiety  and  attitudes.  Especially  for  mature 
students,  the  links  between  these  personal  factors  can 
increase  the  burden  on  the  student  (negative  past  expe¬ 
riences  with  technology).  Cavanaugh  et  al.  (2000) 
hypothesized  that  technology  self-efficacy  will  affect 
participants’  attitudes  toward  technology  and  their  per¬ 
ceptions  of  the  effectiveness  of  training.  The  results 
indicate  that  attitudes  toward  technology  mediate  the 
relationships  among  technology,  self-efficacy,  and  dif¬ 
ferent  measures  of  effectiveness.  Meyer  and  Stem- 
berger  (2005)  examined  self-efficacy,  self-reliance, 
and  motivation  in  a  quasi-experimental  study  with  uni¬ 
versity  students.  Post-tests  and  course  grades  demon¬ 
strated  no  significance,  whereas  results  of  an  item-to- 
item  analysis  of  a  questionnaire  showed  overall  satis¬ 
faction  with  the  teaching  methodology  and  varied 
results  for  self-efficacy,  self-reliance,  and  motivation. 

The  Uses-and-Gratifications  Approach 

This  approach  has  an  influential  tradition  in  media 
research  and  focuses  on  why  people  use  particular 
media  rather  than  on  their  content.  The  uses-and-grat- 
ification  approach  goes  back  to  Blunder  and  Katz 
(1974),  who  suggested  that  media  users  play  an  active 
role  in  choosing  and  using  the  media  that  best  fulfills 
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the  needs  of  the  user.  From  the  perspective  of  motiva¬ 
tion  psychology,  the  uses-and-gratifications  approach 
can  be  considered  a  specific  variant  of  expectation- 
value  models  of  motivation.  These  models  traditionally 
assume  that  both  expectancy  and  value  are  required 
for  goal  commitment. 

For  a  long  time,  the  focus  of  the  uses-and-gratifi¬ 
cations  approach  was  clearly  on  mass  media,  such  as 
television.  Sufficient  evidence  suggests  that  television 
viewing  use  is  often  habitual,  ritualistic,  and  unselec- 
tive  (Barwise  and  Ehrenberg,  1988)  but  television 
viewing  can  sometimes  also  be  an  aesthetic  experience 
involving  intrinsic  motivation.  Furthermore,  Zillmann 
(cited  by  McQuail,  1987,  p.  236)  has  shown  the  influ¬ 
ence  of  mood  on  media  choice:  Boredom  encourages 
the  choice  of  exciting  content,  whereas  stress  encour¬ 
ages  a  choice  of  relaxing  content. 

Due  to  the  fact  that  the  World  Wide  Web  has 
become  a  major  contemporary  mass  medium,  the 
uses-and-gratifications  approach  has  been  applied  for 
studying  the  Web  as  a  whole  and  for  examining  spe¬ 
cific  types  of  sites  on  the  Web.  Kaye  (2002),  for 
example,  examined  the  uses  and  gratifications  of 
accessing  online  sources  for  political  information.  A 
factor  analysis  revealed  four  primary  motivations  for 
connecting  to  online  political  information,  among 
which  the  need  for  entertainment  is  of  motivational 
concern.  Interestingly,  Grace-Farfaglia  and  col¬ 
leagues  (2005)  investigated  the  issue  of  gender  and 
Internet  uses  and  gratifications  with  a  representative 
sample  recruited  from  Internet  users  in  different 
countries.  In  general,  men  still  maintain  higher  Inter¬ 
net  skills  and  self-efficacy  in  computer  applications, 
but  the  gender  gap  is  narrowing  in  each  country. 
Indeed,  a  regression  analysis  of  Internet  gratification 
factors  found  gender  differences  in  Internet  motiva¬ 
tions  and  online  community  participation.  The  data 
suggest  that  real-world  social  motivations  impact  par¬ 
ticipation  in  the  virtual  space. 

An  analysis  of  the  recent  literature  shows  that 
motivations  relating  to  the  use  of  the  Internet  have  been 
well  researched  (Parker  and  Plank,  2000;  Song  et  al., 
2006);  however,  these  studies  remain  to  a  great  extent 
within  the  realm  of  the  traditional  uses-and-gratifica- 
tions  approach,  providing  a  typology  of  use  motiva¬ 
tions,  and  fail  to  develop  new  Internet-specific  gratifi¬ 
cations  (Stafford  et  al.,  2004). 


SUMMARY 

When  we  take  the  major  fields  of  research  on  memory 
and  motivation  into  consideration  with  regard  to  their 
implications  for  the  design  of  instruction,  we  find  a 


tension  between  strong  (those  that  lead  to  very  precise 
conclusions)  and  weak  (those  that  lead  to  less  precise 
conclusions)  assumptions.  Strong  assumptions  are 
helpful  when  the  assumptions  apply,  but  they  often  do 
not  apply,  which  then  invalidates  the  conclusions  pre¬ 
scribed  by  the  model.  Weak  assumptions  are  less  help¬ 
ful  in  creating  specific  instructional  systems  and  learn¬ 
ing  activities,  but  they  are  more  generally  applicable 
and  less  likely  to  be  invalidated.  Finding  the  right 
balance  is  the  challenge  for  professional  practitioners. 
Extending  the  research  so  weak  models  become  stron¬ 
ger  and  more  useful  is  the  challenge  for  instructional 
design  research. 
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ABSTRACT 

This  chapter  provides  an  overview  of  Stephen  Pepper’s 
philosophical  worldviews  (1942)  as  a  way  of  clarifying 
the  philosophical  assumptions  of  different  perspectives. 
A  detailed  analysis  of  contextualism  is  provided,  and 
the  manner  in  which  this  worldview  relates  to  both 
constructivist  and  behavioral  theories  in  instructional 
design  is  explicated. 


KEYWORDS 

Contextualism:  A  philosophical  worldview  in  which  any 
event  is  interpreted  as  an  ongoing  act  inseparable 
from  its  current  and  historical  context  and  in  which 
a  radically  functional  approach  to  truth  and  meaning 
is  adopted.  The  root  metaphor  of  contextualism  is 
the  act-in-context,  and  the  truth  criterion  of  contex¬ 
tualism  is  successful  working  or  effective  action. 
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Descriptive  contextualism:  A  variant  of  contextualism 
that  has  as  its  primary  goal  an  understanding  of  the 
complexity  and  richness  of  a  whole  event  through 
an  appreciation  of  its  participants  and  features.  It 
seeks  the  construction  of  knowledge  that  is  spe¬ 
cific,  personal,  ephemeral,  and  spatiotemporally 
restricted,  like  a  historical  narrative. 

Functional  contextualism:  A  philosophy  of  science 
and  variant  of  contextualism  that  has  as  its  primary 
goal  the  prediction  and  influence  of  events  with 
precision,  scope,  and  depth  using  empirically  based 
concepts  and  rules.  It  seeks  the  construction  of 
knowledge  that  is  general,  abstract,  and  spatiotem¬ 
porally  unrestricted,  like  a  scientific  principle. 


INTRODUCTION 

It  can  be  difficult  to  fully  appreciate  the  role  philo¬ 
sophical  assumptions  play  in  the  development  of  the¬ 
ory,  science,  and  technology.  This  is  particularly  true 
in  applied  disciplines  such  as  instructional  design, 
where  scholars  and  practitioners  often  leave  core 
assumptions  unstated,  cobble  together  patchwork  the¬ 
ories  or  techniques  in  the  hope  of  embracing  diverse 
perspectives,  or  trust  that  brute-force  empiricism  will 
reveal  all  of  the  relevant  facts.  Unfortunately,  such 
practices  make  it  difficult  to  clearly  discern  philosoph¬ 
ical  differences  and  are  unlikely  to  contribute  to  the 
development  of  a  coherent  and  progressive  discipline. 
Those  who  do  not  specify  their  core  assumptions  or 
claim  that  they  have  none  are  typically  just  adopting 
those  of  the  mainstream  culture;  those  who  carelessly 
integrate  diverse  theories  run  the  risk  of  trying  to  com¬ 
bine  mutually  exclusive  tenets  and  producing  little 
more  than  incoherence;  and  those  who  trust  in  merely 
amassing  raw  empirical  data  are  advocating  a  strategy 
that  can  be  grossly  inefficient,  uncharacteristic  of  sci¬ 
entific  research,  and  ill  suited  to  the  development  of 
true  technological  innovation.  Theories,  and  the  philo¬ 
sophical  assumptions  that  guide  their  construction  and 
evaluation,  are  the  foundation  of  scientific  and  tech¬ 
nological  progress. 

“Theory”  has  a  wide  variety  of  meanings  in  intel¬ 
lectual  discourse,  but  we  will  restrict  our  discussion  to 
a  relatively  simple  interpretation:  statements  that  are 
relatively  precise  and  have  relatively  broad  scope.  Pre¬ 
cision  refers  to  the  number  of  ways  a  particular  phe¬ 
nomenon  can  be  explained  with  a  given  set  of  analytic 
concepts  (the  fewer,  the  better),  and  scope  refers  to  the 
range  of  phenomena  that  can  be  explained  with  those 
concepts  (the  wider,  the  better).  Theories  are  valuable 
because  they  allow  for  conceptual  economy  and  par¬ 
simony,  provide  guidance  for  confronting  new  prob¬ 


lems  or  situations,  and  prevent  a  discipline  from 
becoming  increasingly  disorganized  and  incoherent 
(Hayes  et  al.,  1999). 

There  are  many  different  theories  or  statements  we 
can  generate  to  describe,  interpret,  and  understand  the 
events  of  our  world.  What  determines  the  types  of 
theories  we  construct  and  value?  Philosophy.  Whereas 
many  theories  are  obviously  tied  to  empirical  data, 
philosophical  assumptions  nonetheless  guide  the  cre¬ 
ation,  assessment,  and  evaluation  of  knowledge  claims 
and  theories.  Regardless  of  the  empirical  support  a 
particular  theory  might  enjoy,  for  example,  it  is  typi¬ 
cally  not  difficult  to  find  scholars  or  practitioners  who 
eschew  it.  Sometimes  this  is  because  the  critics  have 
different  criteria  (or  respect)  for  empirical  support, 
sometimes  it  is  because  they  do  not  view  the  theory 
as  compatible  with  their  belief  system,  and  sometimes 
it  is  because  they  have  a  competing  theory  that  can 
account  for  the  same  data.  In  all  of  these  cases,  philo¬ 
sophical  assumptions  are  impacting  their  evaluation  of 
the  theory.  As  suggested  earlier,  however,  individuals 
often  do  not  clearly  reveal  or  explicate  these  assump¬ 
tions  when  engaging  in  theoretical  debates.  It  seems 
that  clarity  about  such  issues  might  allow  more  pro¬ 
ductive  discussions  and  comparisons. 

One  approach  to  illuminating  underlying  philo¬ 
sophical  assumptions  is  the  root  metaphor  or  world¬ 
view  approach  offered  by  Stephen  Pepper  in  his  book 
World  Hypotheses:  A  Study  in  Evidence  (1942).  Pepper 
was  what  some  would  call  a  meta-philosopher:  he  phi¬ 
losophized  about  philosophy.  In  particular.  Pepper  was 
interested  in  identifying  key  similarities  and  differ¬ 
ences  among  the  many  different  schools  of  philosophy 
that  have  emerged  over  the  centuries.  As  Morris  (1997, 
p.  531)  notes.  Pepper  was  “simply  trying  to  make  sense 
of  the  welter  of  philosophical  and  scientific  ideas  and 
concepts  found  in  his  day.” 

Pepper  claimed  that  any  philosophy  can  be  cate¬ 
gorized  by  the  overarching  metaphorical  language  it 
uses  (its  root  metaphor)  and  the  rules  of  evidence  it 
embraces  (its  truth  criterion).  A  world  hypothesis  or 
worldview  consists  of  a  distinctive  root  metaphor  and 
truth  criterion,  and  Pepper  observed  that  well-devel¬ 
oped  philosophies  tend  to  cluster  around  one  of  several 
such  worldviews.  The  categories  and  concepts  derived 
from  a  worldview’s  root  metaphor  serve  as  the  basis 
for  constructing  theories  or  statements  about  the  world, 
and  its  truth  criterion  provides  the  rules  used  to  eval¬ 
uate  theories  and  knowledge  claims.  Pepper’s  frame¬ 
work  can  prove  very  useful  for  revealing  the  essential 
components,  assumptions,  and  concerns  of  different 
discourse  communities,  and  for  this  reason  his  work 
has  recently  received  renewed  attention  in  many  areas 
(Ambrose,  1998;  Berry,  1984;  Berzins,  1979;  Biglan, 
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1995;  Bredo,  1994;  Fox,  2006;  Hayes  et  al.,  1988; 
Lyddon,  1989;  Minton,  1992;  Morris,  1988;  Overton, 
1984;  Prawat  and  Floden,  1994;  Quina,  1982;  Reese, 
1991;  Seifert,  2000). 

This  chapter  briefly  reviews  Pepper’s  primary 
worldviews  and  then  focuses  on  one  that  seems  to 
inform  a  great  variety  of  instructional  design  theory: 
contextualism.  Varieties  of  contextualism  are  described 
and  related  to  the  field  of  instructional  design,  and  the 
role  that  contextualism  could  play  in  developing  a 
more  coherent,  pragmatic,  and  progressive  science  of 
learning  and  instruction  is  outlined. 


PHILOSOPHICAL  WORLDVIEWS 

Pepper  (1942)  wrote  that  a  philosophical  worldview  is 
characterized  by  a  root  metaphor  and  truth  criterion. 
A  root  metaphor  is  a  well-known,  familiar  object  or 
idea  that  serves  as  the  basic  analogy  by  which  an 
analyst  attempts  to  understand  the  complexities  of  the 
world.  Those  who  adhere  to  the  worldview  of  mecha¬ 
nism,  for  example  ,  use  the  root  metaphor  of  a  machine 
to  interpret  events:  the  entire  universe  is  considered  to 
be  like  a  machine  ,  and  the  mechanist  seeks  to  discover 
the  discrete  parts  of  this  machine  and  understand  how 
they  relate  to  one  another.  A  worldview’s  truth  crite¬ 
rion,  inextricably  linked  to  its  root  metaphor,  provides 
the  basis  for  evaluating  the  validity  of  analyses. 

Root  metaphors  roughly  correspond  to  ontological 
assumptions,  or  views  about  the  nature  of  being  or 
existence  (e.g.,  whether  or  not  the  universe  is  deter¬ 
ministic).  Truth  criteria  roughly  correspond  to  episte¬ 
mological  assumptions  or  views  about  the  nature  of 
knowledge  and  truth  (e.g.,  whether  it  is  discovered  or 
constructed).  Morris  (1997)  noted  that  a  worldview 
(e.g.,  mechanism)  is  a  broad  category  that  consists  of 
an  informally  organized  epistemology  and  ontology, 
whereas  a  specific  philosophy  (e.g.,  logical  positivism) 
represents  a  more  formally  systemized  version  of  its 
worldview’s  epistemology  and  ontology.  As  will  be 
revealed  later,  particular  philosophies  operating  under 
the  same  worldview  can  appear  rather  dissimilar  due 
to  differences  in  how  they  formalize  their  ontological 
and  epistemological  assumptions. 

Like  theories,  worldviews  can  be  evaluated  by  their 
degree  of  precision  and  scope.  Pepper,  in  fact,  uses 
these  criteria  to  identify  “relatively  adequate”  world¬ 
views:  those  that  strive  for  complete  scope  with  abso¬ 
lute  precision.  Although  theories  typically  have  good 
scope  with  regard  to  events  in  a  particular  domain, 
worldviews  strive  for  unlimited  scope  with  regard  to 
all  events.  There  are  many  different  worldviews,  and 
none  has  perfect  scope  and  precision,  but  Pepper 


claims  that  these  come  closest  while  maintaining  good 
internal  consistency:  form  ism,  mechanism ,  contextual¬ 
ism,  and  organicism . 

Before  examining  the  worldviews  in  more  detail, 
it  is  worth  considering  some  of  the  cautions  Pepper 
(1942)  offered  with  regard  to  evaluating.  One  is  that 
the  adequacy  of  one  worldview  cannot  legitimately  be 
determined  by  another  worldview,  nor  do  the  short¬ 
comings  of  one  worldview  necessarily  strengthen  the 
position  of  another.  If  one  considers  the  assumptions 
of  a  worldview  to  be  analogous  to  the  rules  for  playing 
a  game  (they  are,  in  a  sense,  rules  for  playing  the  game 
of  philosophy),  it  is  easy  to  see  why  this  would  be 
inappropriate.  It  would  be  similar  to  a  baseball  player 
criticizing  tennis  for  its  lack  of  home  runs— the  criti¬ 
cism  is  illegitimate,  obviously, because  tennis  is  played 
under  an  entirely  different  set  of  rules.  Likewise,  it  is 
illegitimate  to  criticize  one  worldview’s  analysis  using 
a  different  worldview’s  rules  for  conducting  analyses. 
A  degree  of  philosophical  humility  is  prudent  for 
everyone,  as  no  degree  of  empirical  evidence  will  ever 
“prove”  that  one  worldview  is  superior  to  another. 
Philosophical  assumptions  enable  analysis,  they  are 
not  the  result  of  analysis  (Hayes,  1993). 

A  second  noteworthy  caution  is  that  eclecticism, 
at  the  level  of  philosophical  assumptions,  is  confusing 
and  unproductive.  This  is  because  the  root  metaphors 
of  the  worldviews  are  autonomous  and  mutually  exclu¬ 
sive.  Keeping  with  our  games  analogy,  this  would  be 
similar  to  trying  to  play  a  game  that  combines  all  of 
the  rules  of  both  baseball  and  tennis .  Most  of  the  rules 
of  these  two  sports  could  not  be  implemented  simul¬ 
taneously.  When  “game  eclecticism”  is  pursued,  it 
becomes  virtually  impossible  to  determine  scoring, 
fouls,  and  winners  (the  game  of  Calvinball  from  the 
comic  strip  Calvin  and  Hobbes  by  Bill  Watterson  illus¬ 
trates  this  point  beautifully).  When  philosophical 
eclecticism  is  pursued,  it  likewise  becomes  virtually 
impossible  to  evaluate  theories  or  knowledge  claims 
in  any  kind  of  clear,  coherent,  or  systematic  way.  Both 
forms  of  eclecticism  can  be  highly  amusing,  but  nei¬ 
ther  is  likely  to  produce  very  meaningful  results. 

Formism 

Similarity  is  the  root  metaphor  of  formism,  and  corre¬ 
spondence  is  its  truth  criterion.  The  two  major  variants 
of  formism  are  termed  immanent  and  transcendent .  In 
immanent  formism,  the  root  metaphor  is  based  on  the 
perception  that  two  or  more  objects  are  similar  or  alike 
(e.g.,  “these  two  apples  are  similar”).  In  transcendent 
formism,  the  root  metaphor  is  based  on  the  perception 
that  objects  conform  to  a  preconceived  form  or  pattern 
(e.g.,  "these  apples  are  examples  of  the  transcendent 
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apple  form”).  A  key  process  of  formist  analysis  is 
either  describing  the  similarities  and  differences 
between  two  objects,  or  naming  and/or  describing  the 
forms  of  which  the  objects  are  examples.  The  measure 
of  truth  inherent  in  this  root  metaphor  is  “the  degree 
of  similarity  which  a  description  has  to  its  object  or 
reference”  (Pepper,  1942,  p.  181),  or  the  correspon¬ 
dence  between  a  description  and  its  referent.  Formism 
is  said  to  be  dispersive,  with  facts  assumed  to  be  unre¬ 
lated  unless  proven  otherwise.  It  is  perhaps  the  oldest 
relatively  adequate  world  hypothesis,  and  students  of 
philosophy  will  recognize  it  as  the  worldview  of  Plato 
(and  possibly  Aristotle). 

Mechanism 

In  mechanism,  the  events,  objects,  and  laws  of  nature 
are  not  assumed  to  be  instances  of  separate  and  uncon¬ 
nected  forms  as  in  formism,  but  integrated  parts  of  a 
single  system.  That  system  is  characterized  by  mech¬ 
anism’s  root  metaphor  of  the  machine.  This  worldview 
assumes  the  universe  to  be  organized  a  priori  into  a 
vast  structure  of  interrelated  parts  — a  machine  — with 
truth  determined  by  "the  way  ever  more  ambitious 
verbal  formulae  reveal  the  assumed  organization  of  the 
world”  (Hayes,  1993,  p.  12).  Constructivists  may  rec¬ 
ognize  mechanism  as  what  they  call  objectivism  or 
positivism,  and  it  is  the  worldview  underlying  most  of 
cognitive  psychology.  The  mechanistic  truth  criterion 
is  a  more  elaborate  version  of  the  correspondence  truth 
criterion  of  formism  but  might  be  better  termed  “pre¬ 
dictive  verification,”  as  truth  statements  about  the 
assumed  organization  of  the  world  (i.e.,  predictions) 
are  generally  tested  on  new  phenomena  (Hayes,  1993). 
This  form  of  analysis  dominates  much  of  the  research 
in  psychology  and  education,  where  there  is  a  “long¬ 
standing  tradition  of  developing  and  validating  models 
of  hypothetical  constructs  that  predict  behavior” 
(Biglan  and  Hayes,  1996,  p.  50). 

Organicism 

The  root  metaphor  of  organicism  can  be  described  as 
organic  development,  and  its  truth  criterion  is  that  of 
coherence.  Organicists  view  events  as  integrated 
organic  systems  that  are  living,  growing,  and  changing, 
and  truth  is  realized  when  “systems  of  belief  ... 
become  more  encompassing  and  integrative,  leading 
towards  an  absolute,  complete  understanding”  (Prawat 
and  Floden,  1994,  p.  42).  Fike  mechanists,  organicists 
consider  all  events  and  facts  of  the  world  to  be  inter¬ 
related.  Unlike  mechanists,  however,  organicists  do  not 
consider  the  whole  (i.e.,  the  organic  system)  to  be 
merely  a  compilation  of  its  parts;  rather,  the  whole  is 


primary,  and  the  parts  are  meaningful  only  as  they 
relate  to  the  overall  system.  Flux  is  an  inherent  char¬ 
acteristics  of  an  organic  system,  and  organicists  thus 
accept  change  and  novelty  as  a  given,  whereas  stability 
and  constancy  are  events  to  be  explained.  This  quality 
is  seen  in  developmental  theories  of  learning  and  psy¬ 
chology  that  rely  on  stage  models,  such  as  the  work 
of  Piaget  (Hayes  et  al.,  1988). 


CONTEXTUALISM 

Contextualism  is  a  worldview  based  on  philosophical 
pragmatism,  a  tradition  heavily  influenced  by  the  work 
of  figures  such  as  Charles  Sanders  Pierce,  William 
James,  Oliver  Wendell  Holmes,  Jr.,  George  Herbert 
Mead,  and  John  Dewey.  As  Menand  (2001 ,  p.  xi)  noted 
in  the  preface  to  his  historical  treatment  of  the  emer¬ 
gence  of  pragmatism  in  America: 

If  we  strain  out  the  differences,  personal  and  philosoph¬ 
ical,  they  had  with  one  another,  we  can  say  that  what 
these  ...  thinkers  had  in  common  was  not  a  group  of 
ideas,  but  a  single  idea — an  idea  about  ideas.  They  all 
believed  that  ideas  are  not  “out  there”  waiting  to  be 
discovered,  but  are  tools — like  forks  and  knives  and 
microchips — that  people  devise  to  cope  with  the  world 
in  which  they  find  themselves. 

In  pragmatism  and  contextualism,  the  truth  and  mean¬ 
ing  of  words  lie  in  their  function  or  utility,  not  in  how 
well  they  are  said  to  mirror  reality.  The  truth  criterion 
of  contextualism  is  thus  dubbed  successful  working, 
in  which  an  analysis  is  said  to  be  true  or  valid  insofar 
as  it  leads  to  effective  action,  or  achievement  of  some 
goal.  This  notion  of  truth  does  not  require— and  is  not 
concerned  with— the  existence  of  absolute,  founda¬ 
tional  truths  or  assumptions  about  the  universe.  As 
James  (1907/1948,  p.  161)  wrote,  “The  truth  of  an  idea 
is  not  a  stagnant  property  inherent  in  it.  Truth  happens 
to  an  idea.  It  becomes  true,  is  made  true  by  events.” 

For  the  contextualist,  an  idea’s  meaning  is  defined 
by  its  practical  consequences  and  its  truth  by  the 
degree  to  which  those  consequences  reflect  successful 
action.  This  extremely  functional  approach  to  meaning 
reflects  Charles  Darwin’s  influence  on  pragmatism,  as 
his  views  on  natural  selection  were  just  gaining  wide¬ 
spread  appeal  among  scholar’s  during  the  era  in  which 
pragmatism  appeared  (Menand,  2001).  Pragmatism 
can  be  viewed  as  an  application  of  Darwin’s  selection- 
ism  to  epistemology:  “In  pragmatism,  ideas  are 
‘selected’  (to  be  retained  as  true  or  valid)  if  they  lead 
to  successful  action,  just  as  in  natural  selection  traits 
are  selected  (to  be  retained  by  the  species)  if  they  lead 
to  reproductive  success”  (Fox,  2006,  p.  10). 
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The  root  metaphor  for  contextualism  is  the  com- 
monsense  notion  of  the  ongoing  act  in  context,  such 
as  reading  a  book,  eating  a  sandwich,  or  teaching  a 
class.  Such  events  are  practical,  concrete  actions  that 
are  “being  performed  by  someone  for  some  purpose 
in  some  context”  (Reese,  1993,  p.  72).  In  contextual¬ 
ism,  context  refers  to  both  the  current  and  historical 
context  of  an  act;  in  fact,  it  is  more  context  as  history 
than  context  as  place  (Morris,  1997).  Although  the 
current  context  of  an  act  is  important,  to  understand 
the  meaning,  function,  and  purpose  of  the  act  one  must 
also  have  an  appreciation  of  the  historical  context.  It 
seems  Pepper  was  basing  his  use  of  the  term  “context” 
on  Dewey’s  notion  of  context  as  “the  historical  situat¬ 
edness  of  the  meaning  and  function  of  behavior”  (Mor¬ 
ris,  1997,  p.  533).  For  these  reasons,  Pepper  also 
referred  to  contextualism’s  root  metaphor  as  "the  his¬ 
toric  event”  (1942,  p.  232). 

Contextualists  analyze  all  phenomena  as  acts  in 
context.  Events  and  their  contexts  are  considered  inte¬ 
grated  wholes  and  are  separated  into  distinct  parts  only 
to  achieve  some  practical  purpose,  not  to  reveal  the 
true  structure  or  organization  of  the  world  (as  a  mech¬ 
anist  might  claim  to  be  doing).  In  contextualism,  such 
divisions  are  utilitarian,  not  foundational.  Further,  the 
entire  universe  and  all  of  time  are  considered  part  of 
the  full  context  of  any  event.  How,  then,  does  a  con- 
textualist  know  how  much  and  which  features  of  the 
potentially  infinite  context  must  be  included  to  ade¬ 
quately  characterize  an  act?  The  answer  to  this  ques¬ 
tion  is  ultimately  a  pragmatic  one  (of  course):  It 
depends  on  one’s  purpose  for  attempting  to  character¬ 
ize  the  act  in  the  first  place.  Essentially,  whichever 
features  of  the  context  aid  in  achieving  the  goal  of  the 
analysis  may  be  included.  This  answer  highlights  the 
important  role  analytic  goals  play  in  the  contextualism. 

Analytic  Goals 

Pepper  (1942,  pp.  250-251)  noted  that,  in  contextual¬ 
ism,  “serious  analysis  ...  is  always  either  directly  or 
indirectly  practical  . . .  there  is  no  importance  in  anal¬ 
ysis  just  for  analysis.”  Analysis  without  some  ultimate 
purpose  is  problematic  in  contextualism  because  nei¬ 
ther  the  root  metaphor  nor  the  truth  criterion  makes 
much  sense  without  a  clear  analytic  goal.  The  success¬ 
ful  working  truth  criterion  is  rendered  meaningless  in 
an  analysis  without  an  explicit  goal  because  success 
can  only  be  measured  in  relation  to  the  achievement 
of  some  objective  (Dewey,  1916/1953).  Similarly,  the 
root  metaphor  of  the  act  in  context  is  difficult  to  mount 
without  an  explicit  goal  because  there  would  be  no 
basis  on  which  to  restrict  the  analysis  to  a  subset  of 
the  infinite  expanse  of  the  act’s  historical  and  environ¬ 


mental  context  (Gifford  and  Hayes,  1999).  It  is  very 
difficult  for  a  contextualist  without  an  explicit  goal  to 
construct  or  share  knowledge  (Hayes,  1993). 

Varieties  of  Contextualism 

As  noted  earlier,  worldviews  are  broad  categories  that 
can  contain  many  specific  philosophies.  Analytic  goals 
play  such  an  important  role  in  contextualism  that  the 
specific  instances  of  this  worldview  can  be  distin¬ 
guished  by  them  (Hayes,  1993).  The  types  of  analyses 
conducted  by  contextualists  and  the  kinds  of  knowl¬ 
edge  they  value  are  greatly  impacted  by  their  analytic 
goals.  Even  within  the  same  domain  (e.g.,  human 
learning) ,  their  goals  and  approaches  can  be  quite  dif¬ 
ferent.  Some  may  be  interested  in  establishing  a  natural 
science,  for  example,  while  others  may  be  interested 
in  establishing  a  natural  history  (Morris,  1993).  Such 
differences  in  purpose  can  result  in  systems  of  inquiry 
that,  despite  being  based  on  the  same  worldview, 
appear  quite  dissimilar.  Contextualistic  theories  can  be 
divided  into  two  general  categories,  based  on  their 
overarching  analytic  goals:  descriptive  contextualism 
and  functional  contextualism  (Hayes,  1993). 

Descriptive  contextualists  seek  to  understand  the 
complexity  and  richness  of  a  whole  event  through  a 
personal  and  aesthetic  appreciation  of  its  participants 
and  features  (see  Rosnow  and  Georgoudi,  1986).  This 
approach  reveals  a  strong  adherence  to  the  root  met¬ 
aphor  of  contextualism  and  can  be  likened  to  the 
enterprise  of  history,  in  which  stories  of  the  past  are 
constructed  in  an  attempt  to  understand  whole  events. 
There  is  no  one  complete  or  true  account  of  any  event, 
however;  each  account  is  localized  to  an  individual 
with  personal  and  aesthetic  meaning  and  value,  rather 
than  global  and  final  (Gifford  and  Hayes,  1999). 
Thus ,  knowledge  constructed  by  the  descriptive  con¬ 
textualist  is  like  a  historical  narrative  — personal, 
ephemeral,  specific,  and  spatiotemporally  restricted 
(Morris,  1993).  Social  constructivism  (Gergen,  1985; 
Rosnow  and  Georgoudi,  1986),  dramaturgy  (Scheibe, 
1993),  hermeneutics  (Dougher,  1993),  and  narrative 
approaches  (Sarbin,  1986)  are  all  instances  of 
descriptive  contextualism. 

Functional  contextualists,  on  the  other  hand,  seek 
to  predict  and  influence  events  using  empirically  based 
concepts  and  rules  (Biglan  and  Hayes,  1996;  Fox, 
2006;  Gifford  and  Hayes,  1999;  Hayes,  1993).  This 
approach  reveals  a  strong  adherence  to  contextualism’s 
extremely  practical  truth  criterion  and  can  be  likened 
to  the  enterprise  of  science  or  engineering,  in  which 
general  rules  and  principles  are  used  to  predict  and 
influence  events.  Rules  or  theories  that  cannot  poten¬ 
tially  contribute  to  the  prediction  and  influence  of  the 
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events  of  interest  are  rejected  or  considered  incom¬ 
plete.  Knowledge  constructed  by  the  functional  con- 
textualist  is  general,  abstract,  and  spatiotemporally 
unrestricted  (Morris,  1993).  Like  a  scientific  principle, 
it  is  knowledge  that  is  likely  to  be  applicable  to  all  (or 
many)  similar  such  events,  regardless  of  time  or  place. 
The  distinction  between  descriptive  contextualism  and 
functional  contextualism  is  analogous  to  the  distinc¬ 
tion  between  natural  history  and  natural  science  (Mor¬ 
ris,  1993). 


DESCRIPTIVE  CONTEXTUALISM 
AND  INSTRUCTIONAL  DESIGN 

Descriptive  contextualism  has  had  a  significant  impact 
on  the  field  of  instructional  design  in  recent  years, 
primarily  in  the  form  of  approaches  commonly 
grouped  under  the  label  of  constructivism.  Construc¬ 
tivists  view  knowledge  not  as  “something  we  acquire 
but  something  that  we  produce ”  (Mautner,  1996,  p. 
83)  and  have  challenged  many  of  the  traditional  prac¬ 
tices  of  the  field  on  epistemological  grounds.  They 
consider  both  behavioral  and  cognitive  psychology, 
which  have  informed  many  instructional  design  prin¬ 
ciples,  to  be  representative  of  objectivism.  Objectivist 
epistemology  holds  that  the  real  world  exists  externally 
to  the  knower  and  that  it  has  a  complete  and  correct 
structure  — or  meaning— determined  by  its  entities, 
properties,  and  relations  (Lakoff,  1987).  Because 
objectivism  is  antithetical  to  their  view  of  knowledge, 
constructivists  have  been  critical  of  many  of  the 
instructional  practices  informed  by  behavioral  and 
cognitive  psychology,  such  as  the  emphasis  on  instruc¬ 
tional  objectives,  the  use  of  task  and  content  analyses, 
and  the  reliance  on  criterion-referenced  assessment 
techniques  (Jonassen,  1991).  Instead,  they  recommend 
a  range  of  different  techniques  for  enhancing  instruc¬ 
tion,  such  as  providing  complex,  realistic  learning 
environments;  using  cooperative  learning,  problem- 
based  learning,  and  discovery  learning  strategies; 
employing  advance  organizers  and  concept  maps;  and 
nurturing  self-awareness  of  the  knowledge  construc¬ 
tion  process  (Driscoll,  2000;  Morrison  et  al.,  2007). 

Clarifying  Constructivism 

Constructivists  in  education  rarely  identify  themselves 
as  pragmatists  or  contextualists,  but  it  is  clear  that  most 
forms  of  constructivism  are  based  on  the  contextual- 
istic  worldview.  Constructivism’s  defining  premise  — 
that  knowledge  is  constructed  rather  than  discov¬ 
ered— is  merely  a  restatement  of  the  pragmatic/con- 
textualistic  view  on  truth  and  meaning.  The  functional 


truth  criterion  of  contextualism  is  also  evident  when 
constructivists  reject  the  absolute  truths  and  structur¬ 
alism  of  objectivism,  and  the  root  metaphor  of  contex¬ 
tualism  is  apparent  when  constructivists  emphasize  the 
cultural  and  historical  context  in  which  education 
occurs  or  champion  the  design  of  authentic  learning 
environments.  Further,  many  prominent  constructivist 
figures,  including  Dewey,  Gergen,  Bruner,  and 
Vygotsky,  are  typically  considered  contextualists 
(Capaldi  and  Proctor,  1999;  Reese,  1993).  The  relation 
between  constructivism  and  contextualism  is  so  appar¬ 
ent  that  Mancuso  ( 1 993 ,  p .  1 20)  claimed  that  "the  basic 
tenets  of  constructivism  as  an  epistemology  demand 
an  acceptance  of  a  contextualist  worldview.” 

Constructivism  is  difficult  to  adequately  character¬ 
ize,  however,  because  the  term  “refers  to  many  ideas, 
joined  by  the  merest  thread  of  family  resemblance  and 
often  expressing  quite  contradictory  views”  (Burbules, 
2000,  p.  308).  There  are  perhaps  dozens  of  different 
strains  of  constructivism,  and  Matthews  (2000)  sug¬ 
gested  that  there  are  more  than  20  variants  of  educa¬ 
tional  constructivism  alone.  Recent  books  on  construc¬ 
tivism  in  education  (see,  for  example,  Larochelle  et 
al.,  1998;  Phillips,  2000)  reveal  the  vastness  and  com¬ 
plexity  of  the  constructivist  landscape  and  provide  a 
sense  of  the  great  theoretical  variation  that  exists 
within  the  different  constructivist  systems. 

The  many  variants  of  constructivism  can  be  dis¬ 
tinguished  by  a  careful  analysis  of  what  they  mean 
by  “knowledge”  and  how  they  are  claiming  it  is  “con¬ 
structed.”  The  primary  dimension  on  which  the  forms 
can  be  differentiated  is  whether  they  focus  on  the 
ways  an  individual  constructs  knowledge  and  mean¬ 
ing  or  on  the  ways  communities  or  social  groups 
negotiate  knowledge  or  truth  (Phillips,  2000).  Con¬ 
structivist  perspectives  that  emphasize  individual 
knowledge  construction  are  often  identified  as  forms 
of  radical  constructivism  (or  sometimes  psychologi¬ 
cal  constructivism)  (Phillips,  2000),  whereas  those 
that  emphasize  the  construction  of  public  domains  of 
knowledge  are  generally  considered  forms  of  social 
constructivism .  Theorists  can  vary  considerably  with 
regard  to  how  much  they  gravitate  toward  either  of 
these  two  poles  and  often  disagree  on  the  process  by 
which  knowledge  is  constructed  or  negotiated  (Phil¬ 
lips,  1995;  Prawat  and  Floden,  1994).  Such  differ¬ 
ences  can  lead  to  surprisingly  different  suggestions 
on  how  best  to  improve  educational  practices  and 
make  it  easy  to  understand  the  proliferation  of  con- 
structivisms  in  the  educational  literature. 

In  instructional  design,  authors  rarely  specify  the 
type  of  constructivism  to  which  they  subscribe  and 
frequently  seem  unable  to  maintain  philosophical 
fidelity  to  a  single  strain.  Jonassen  (1994),  for  example. 
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operating  under  the  general  banner  of  constructivism, 
initially  took  a  radical  constructivist  position  that 
“learners  construct  their  own  reality  or  at  least  interpret 
it  based  upon  their  perceptions  of  experiences”  (p.  34), 
yet  in  the  very  next  paragraph  adopted  a  social  con¬ 
structivist  stance  that  “much  of  reality  is  shared 
through  a  process  of  social  negotiation”  (p.  35). 
Although  these  two  statements  are  not  necessarily  log¬ 
ically  incompatible,  they  do  represent  the  thinking  of 
quite  different  perspectives— -perspectives  that  defi¬ 
nitely  have  different  goals  and  may  even  be  based  on 
entirely  different  worldviews.  It  has  been  argued  that 
radical  constructivism  is  based  on  organicism  (Hayes 
et  al.,  1988;  Prawat  and  Floden,  1994),  whereas  social 
constructivism  is  clearly  contextualistic. 

This  lack  of  clarity  in  constructivist  writing  has  no 
doubt  contributed  to  the  many  questions  and  debates 
the  movement  has  inspired:  Does  an  extreme  construc¬ 
tivist  position  make  the  practice  of  designing  instruc¬ 
tion  pointless  or  impossible  (Winn,  1993)?  Why  do 
some  constructivists  seem  to  embrace  information¬ 
processing  theory,  but  others  reject  it  (Prawat  and 
Floden,  1994)?  Can  constructivism  serve  as  a  prescrip¬ 
tive  theory  of  instruction  at  all  (Jonassen,  1994)?  Why 
is  it  that,  when  constructivist  models  are  procedural- 
ized,  they  seem  so  similar  to  traditional  instructional 
design  models  (Dick,  1996)?  Can  constructivism  be 
both  an  educational  philosophy  and  a  method  (Lebow, 
1993)?  And,  perhaps  most  importantly,  “is  construc¬ 
tivism  destined  to  join  discovery  learning  in  the  long 
list  of  educational  enthusiasms  that  come  and  go,  never 
articulated  clearly  enough  to  be  tested,  or  in  vogue 
long  enough  to  prove  their  theoretical  interest  or  prac¬ 
tical  worth?”  (Cobb,  1999,  p.  16). 

Recognizing  the  contextualistic  worldview  under¬ 
lying  most  constructivist  theories  may  help  resolve 
some  of  the  confusion  surrounding  constructivism.  By 
taking  different  approaches  to  defining  “knowledge” 
and  “construction,”  constructivists  are  actually  adopt¬ 
ing  different  analytic  goals  and  content  areas.  Because 
the  purpose  of  analysis  in  contextualism  guides  how 
the  root  metaphor  is  used  and  how  truth  is  determined, 
the  different  constructivist  theories  are  evolving  (or 
have  evolved)  into  contextualistic  systems  that  value 
and  develop  different  types  of  knowledge  and  analyses. 
This  makes  it  difficult  for  different  types  of  construc¬ 
tivists  to  engage  in  meaningful  or  useful  discourse  and 
makes  it  particularly  unwise  to  treat  the  variety  of 
constructivist  perspectives  as  though  they  represent  a 
singular  theoretical  perspective. 

Although  there  are  important  differences  among 
the  constructivist  theories  in  education,  most  can  be 
accurately  characterized  as  forms  of  descriptive  con¬ 
textualism.  The  clearest  evidence  of  this  is  in  their 


strong  preference  for  qualitative  research  methodolo¬ 
gies.  Savenye  and  Robinson  (2004,  p.  1046)  noted  that 
qualitative  research  is  based  on  the  notion  that 
"humans  construct  their  own  reality”  and  typically 
involves  "highly  detailed  rich  descriptions  of  human 
behaviors  and  opinions.”  Such  research  typically 
includes  ethnographies,  case  studies,  surveys,  inter¬ 
views,  and  historical  and  document  analyses  (Denzin 
and  Lincoln,  1998).  All  of  these  methods  closely 
resemble  historical  narrative,  which  exemplifies  the 
type  of  knowledge  pursued  and  constructed  by  descrip¬ 
tive  contextualists. 

Limitations 

As  forms  of  descriptive  contextualism,  constructivist 
theories  get  their  strength  from  their  close  adherence 
to  contextualism’s  root  metaphor  of  the  act  in  context. 
Descriptive  contextualism’s  goal  of  achieving  a  per¬ 
sonal  understanding  and  description  of  the  whole  event 
aligns  perfectly  with  the  root  metaphor,  and  for  this 
reason  the  position  is  remarkably  consistent  (Gifford 
and  Hayes,  1999). 

Such  theories,  however,  also  share  the  inherent 
weaknesses  of  descriptive  contextualism  (Gifford  and 
Hayes,  1999;  Hayes,  1993).  The  first  is  that  it  is  diffi¬ 
cult  to  determine  when  the  somewhat  ill-defined  goal 
of  descriptive  contextualism  has  been  accomplished 
(Hayes,  1993).  When  is  the  narrative  complete,  or 
when  has  the  story  been  told  well  enough?  This  prob¬ 
lem  is  clearly  recognized  by  many  descriptive  contex¬ 
tualists:  “Qualitative  narratives  all  have  as  their  objec¬ 
tive  an  authentic  and  holistic  portrayal  of  an  intact 
social  or  cultural  scene  ...  [but]  the  issue  of  what 
constitutes  an  authentic  and  holistic  portrayal  has 
become  hotly  contested  territory”  (LeCompte  et  al., 
1992,  p.  xv).  Debates  over  the  accuracy  and  complete¬ 
ness  of  different  historical  accounts  of  events  also 
reflect  these  difficulties. 

Another  problem  faced  by  descriptive  contextual¬ 
ists  is  that  the  accomplishment  of  their  goal  does  not 
necessarily  result  in  any  practical  knowledge  or  bene¬ 
fits.  A  personal,  holistic  appreciation  of  a  specific  event 
may  or  may  not  yield  information  that  is  helpful  for 
achieving  any  other  goals  held  by  the  analyst  or  anyone 
else  (Hayes,  1993).  This  is  probably  the  most  vexing 
problem  facing  constructivists  in  education,  as  they 
struggle  to  develop  empirically  verified  practical  appli¬ 
cations  of  their  theory  (Cobb,  1999;  Driscoll,  2000). 
By  virtue  of  its  own  overarching  purpose,  constructiv¬ 
ism  is  a  descriptive  theory  of  learning  or  knowledge, 
not  a  prescriptive  theory  of  instruction  (Jonassen, 
1994).  Descriptive  contextualists  and  constructivists 
are  typically  not  concerned  about  this  lack  of  practical 
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knowledge  (it  is  not  their  purpose,  after  all).  In  advo¬ 
cating  activity  theory,  for  example,  Jonassen  (2006,  p. 
44)  noted  that  it  “is  not  a  theory  of  learning  from  which 
instructional  prescriptions  can  be  reasonably  drawn.” 
Some  constructivists  even  criticize  functional  contex- 
tualism  for  its  explicit  focus  on  being  “pragmatic”  and 
“useful”  (Hannafin,  2006,  p.  40). 

Though  strong  in  its  adherence  to  contextualism’s 
root  metaphor,  descriptive  contextualism  is  not  well 
suited  to  the  development  of  practical  knowledge.  This 
view  certainly  has  advantages  and  has  made  important 
contributions  to  the  field,  but  it  is  difficult  to  use  as  a 
philosophy  of  science  or  as  the  foundation  for  an 
applied  academic  discipline  like  instructional  design 
(Fox,  2006). 


FUNCTIONAL  CONTEXTUALISM 
AND  INSTRUCTIONAL  DESIGN 

Functional  contextualism  has  been  offered  as  a  philos¬ 
ophy  of  science  that  underlies  modem  behavioral  psy¬ 
chology  and  that  could  also  serve  as  a  strong  founda¬ 
tion  for  instructional  design  (Biglan,  1995;  Fox,  2006; 
Gifford  and  Hayes,  1999;  Hayes,  1993).  As  a  philoso¬ 
phy  that  embraces  the  scientific  principles  of  learning 
and  conditioning  developed  in  the  field  of  behavior 
analysis,  functional  contextualism  supports  many  of 
the  guidelines  for  instructional  development  behavioral 
psychologists  have  developed  over  the  years  (see 
Chapter  15  in  this  volume;  see  also  Burton  et  al.,  2004; 
Fredrick  and  Hummel,  2004;  West  and  Hamerlynck, 
1992).  In  addition,  recent  functional-contextual 
research  on  language  and  cognition  (Hayes  et  al.,  2001 ) 
promises  to  provide  a  more  pragmatically  useful  way 
of  speaking  about  complex  human  behavior  that  could 
have  a  significant  impact  on  instructional  practices.  It 
has  been  argued  that  functional  contextualism, 
although  sharing  the  same  worldview  as  most  construc¬ 
tivist  approaches,  may  offer  a  more  cohesive  view  for 
conducting  a  science  of  learning  and  instruction  “with 
increased  clarity,  precision,  and  concern  for  the  con¬ 
struction  of  practical  knowledge”  (Fox,  2006,  p.  21). 

Behavioral  psychology  has  traditionally  been  char¬ 
acterized  as  mechanistic,  objectivistic ,  realistic,  posi¬ 
tivistic,  and  reductionistic.  Whereas  this  is  accurate  for 
some  versions  of  behaviorism  (and  there  are  more  than 
a  dozen  varieties;  see  O’ Donohue  and  Kitchener, 
1999),  it  is  not  entirely  true  for  the  system  developed 
by  Skinner.  Behavior  analysis  has  contained  elements 
of  both  mechanism  and  contextualism  (Hayes  et  al., 
1988),  primarily  due  to  philosophical  inconsistencies 
in  Skinner’s  writings  and  the  gradual  evolution  of  his 
ideas  (Gifford  and  Hayes,  1999;  Hayes  et  al.,  1988; 


Moxley,  1999, 2001).  Skinner’s  early  work  was  decid¬ 
edly  more  mechanistic  and  positivistic,  but  by  1945 
his  work  reflected  a  much  closer  connection  to  prag¬ 
matism,  selectionism,  and  contextualism  (Moxley, 
1999 , 200 1 ) .  When  Skinner  described  scientific  knowl¬ 
edge  as  “a  corpus  of  rules  for  effective  action,  and 
these  is  a  special  sense  in  which  it  could  be  ‘true’  if 
it  yields  the  most  effective  action  possible”  (1974,  p. 
235),  he  was  clearly  embracing  the  pragmatic  truth 
criterion  and  rejecting  the  objectivist  and  realist  view 
of  knowledge.  In  fact,  many  behavioral  psychologists 
now  explicitly  embrace  contextualism  as  the  philoso¬ 
phy  underlying  their  work  (e.g.,  Barnes-Holmes,  2000; 
Biglan,  1995;  Gifford  and  Hayes,  1999;  Hayes  et  al., 
1993;  Fee,  1988;  Morris,  1988;  Odom  and  Haring, 
1994;  Roche,  1999).  Functional  contextualism  can  be 
seen  as  a  clarification  and  refinement  of  the  pragmatic 
philosophy  underlying  Skinner’s  later  work  and  mod¬ 
ern  behavioral  psychology. 

Implications  of  the  Analytic  Goal 

Due  to  differences  in  analytic  goals,  work  based  on 
functional  contextualism  (e.g.,  behavior  analysis) 
appears  quite  dissimilar  to  work  based  on  descriptive 
contextualism  (e.g.,  social  constructivism).  Whereas 
descriptive  contextualism  is  focused  on  creating  a 
descriptive  natural  history  of  events,  functional  con¬ 
textualism  is  focused  on  creating  a  pragmatic  natural 
science  that  seeks  “the  development  of  an  organized 
system  of  empirically  based  verbal  concepts  and  rules 
that  allow  behavioral  phenomena  to  be  predicted  and 
influenced  with  precision,  scope,  and  depth”  (Biglan 
and  Hayes ,  1 996 ,  pp .  50-5 1 ) .  Functional  contextualists 
study  the  current  and  historical  context  in  which 
behavior  evolves  in  an  effort  to  construct  general  laws, 
principles,  and  rules  that  are  useful  for  predicting  and 
changing  psychological  events  in  a  variety  of  settings. 

Adopting  the  analytic  goal  of  the  prediction  and 
influence  of  psychological  events  has  important  rami¬ 
fications  for  a  science  of  behavior  (Biglan  and  Hayes, 
1996;  Fox,  2006).  First,  analyses  focus  on  functional 
relations  between  psychological  events  and  manipula¬ 
te  (at  least  in  principle)  events  in  the  environment. 
Much  of  educational  and  psychological  theorizing 
focuses  on  relations  between  one  type  of  psychological 
event  (e.g.,  cognitive  schema)  and  another  (e.g.,  overt 
performance  on  a  task).  This  is  not  productive  for  the 
functional  contextualist,  because  we  can  only  directly 
manipulate  events  in  a  person’s  environment  (or  the 
context  of  their  behavior)  (Hayes  and  Brownstein, 
1986).  By  emphasizing  relations  between  two  types  of 
psychological  events,  cognitive  theories  provide  little 
guidance  on  how  to  influence  either  event;  it  is  like 
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focusing  on  a  correlation  between  two  dependent  vari¬ 
ables  rather  than  a  functional  relation  between  an  inde¬ 
pendent  variable  and  a  dependent  variable. 

Although  functional  contextualists  agree  with  con¬ 
structivists  that  "the  learner  is  an  active,  changing 
entity”  (Hannafin  and  Hill,  2002,  p.  77),  they  do  not 
downplay  the  vital  roles  the  environment  and  experi¬ 
ence  play  in  how  the  learner  acts  or  changes.  The 
learner  does  not  act  or  change  in  a  vacuum,  after  all. 
Thus,  behavior  analysts  attempt  to  identify  aspects  of 
the  manipulable  environment  that  influence  the  occur¬ 
rence,  incidence,  prevalence,  or  probability  of  both 
private  and  overt  psychological  events.  Cognition  and 
other  internal  mental  events  are  interpreted  by  appeal¬ 
ing  to  a  person’s  learning  history,  rather  than  assuming 
they  are  underlying  processes  causing  and  controlling 
other  psychological  events  or  behavior. 

In  addition,  the  ultimate  purpose  of  prediction  and 
influence  demands  an  emphasis  on  experimental 
research.  To  examine  the  impact  a  particular  contextual 
variable  has  on  an  event,  it  is  necessary  to  systemati¬ 
cally  vary  that  variable  and  measure  the  resulting 
impact  on  the  event  of  interest;  in  other  words,  it  is 
necessary  to  conduct  an  experiment  (Biglan,  1995; 
Hayes,  1993).  Purely  descriptive  or  correlational 
research  does  not  isolate  which  features  of  the  context 
are  influencing  changes  in  the  psychological  event. 
Functional  contextualists  favor  experimental  tech¬ 
niques  but  value  any  methodology  that  may  contribute 
to  their  pragmatic  goals  (Biglan  and  Hayes,  1996). 
Both  traditional  group  designs  and  time-series  analyses 
of  individual  behavior  can  be  employed  effectively  for 
the  purposes  of  functional  contextualism,  and  even  cor¬ 
relational  or  predictive  research  can  be  useful  for  sug¬ 
gesting  which  contextual  variables  might  be  relevant 
to  the  event  of  interest.  Qualitative  methodologies  also 
have  their  uses  in  functional  contextualism  but  are  not 
as  effective  as  experimental  procedures  for  testing  the 
influence  of  environmental  variables  on  behavior  or  for 
verifying  the  general  utility  of  principles. 

Behavioral  Principles  and  Applications 

The  most  well-established  principles  for  predicting 
and  influencing  psychological  events  are  those  related 
to  classical  and  operant  conditioning.  This  relatively 
small  set  of  principles  and  concepts  provides  a  precise 
way  of  speaking  about  learning  that  also  has  remark¬ 
able  scope.  For  several  decades,  the  concepts  and  prin¬ 
ciples  constructed  by  behavior  analysts  have  proven 
remarkably  effective  in  allowing  teachers,  parents, 
therapists,  managers,  administrators,  trainers,  and 
many  others  to  change  and  improve  the  behavior  of 
both  humans  and  animals  in  many  different  contexts 


(Austin  and  Carr,  2000;  Biglan,  1995;  Dougher,  1999; 
Martin  and  Pear,  2003;  Mattaini  and  Thyer,  1996). 

Procedures  derived  from  behavior  analysis  are  typ¬ 
ically  considered  best  practice  in  the  treatment  and  edu¬ 
cation  of  individuals  with  developmental  disabilities;  for 
example,  Scotti  et  al.  (1996)  and  Matson  et  al.  (1996) 
reported  on  over  550  studies  showing  the  efficacy  of 
such  methods  with  persons  with  autism.  In  clinical  psy¬ 
chology,  a  1995  report  by  the  American  Psychological 
Association’s  theoretically  diverse  Task  Force  on  Pro¬ 
motion  and  Dissemination  of  Psychological  Procedures 
indicated  that  over  20  of  the  27  clinical  interventions 
listed  as  empirically  validated  are  behavior  therapy  tech¬ 
niques  (as  cited  in  O’ Donohue  and  Kitchener,  1999). 

In  instructional  design,  most  are  familiar  with  the 
considerable  influence  Skinner’s  programmed  instruc¬ 
tion  movement  (1954,  1968)  has  had  on  both  instruc¬ 
tional  systems  design  and  human  performance  tech¬ 
nology  (HPT)  (Binder,  1995;  Reiser,  2001)  but  may 
be  unaware  of  other  behavioral  contributions  to  edu¬ 
cation.  Instructional  methods,  such  as  the  Personalized 
System  of  Instruction  (PSI)  (Fox,  2004;  Keller,  1968), 
Direct  Instruction  (Becker  and  Carnine,  1980;  Engel- 
mann  and  Carnine,  1991;  Kinder  and  Carnine,  1991), 
Precision  Teaching  (Merbitz  et  al.,  2004a ,b).  Head- 
sprout  Reading  Basics  (Layng  et  al.,  2004),  the  Com¬ 
prehensive  Application  of  Behavior  Analysis  to 
Schooling  (CABAS)  (Greer,  2002),  and  the  Morning- 
side  Model  of  Generative  Instruction  (Johnson  and 
Layng,  1992;  Johnson  and  Street,  2004),  are  all  explic¬ 
itly  based  on  behavioral  principles  and  enjoy  an  exten¬ 
sive  amount  of  empirical  support  (see  Moran  and  Mal- 
ott,  2004,  for  a  review  of  many  of  these  methods). 
Because  most  constructivists  mistakenly  consider 
behavioral  psychology  to  be  objectivistic  or  mechanis¬ 
tic,  they  often  reject  these  empirically  supported  meth¬ 
ods  simply  because  they  are  “inconsistent  with  those 
espoused  by  constructivists”  (Hannafin,  2006,  p.  39). 
When  the  pragmatic  core  of  modem  behavioral  psy¬ 
chology  is  elucidated,  as  it  is  with  functional  contex¬ 
tualism,  it  is  difficult  to  understand  what  quarrel  con¬ 
structivists  could  have  with  these  methods. 

Functional  contextualists  do  not  reject  instruc¬ 
tional  methods  simply  because  they  originate  from  a 
different  worldview  or  perspective.  If  a  technique  has 
been  demonstrated  to  reliably  enhance  learning  and 
performance,  functional  contextualists  will  embrace  it 
(although  they  may  speak  of  the  learning  process  in  a 
manner  more  consistent  with  their  perspective).  Con¬ 
textualism  as  a  worldview  is  particularly  well  suited 
to  technological  eclecticism  because  of  its  pragmatic 
truth  criterion— contextualists  can  readily  adopt  tech¬ 
niques  and  methods  of  other  worldviews  if  it  helps 
them  achieve  their  analytic  goal  (Hayes  et  al.,  1988). 
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Recent  advances  in  behavioral  research  on  lan¬ 
guage  and  cognition  also  hold  great  promise  for  edu¬ 
cation.  In  particular.  Relational  Frame  Theory  (RFT) 
(Hayes  et  al.,  2001)  is  an  approach  to  understanding 
complex  human  behavior  that  is  based  on  functional 
contextualism  and  basic  behavioral  principles.  RFT 
also  introduces  a  new  behavioral  principle  that 
explains  how  basic  stimulus  functions— such  as  rein¬ 
forcing,  punishing,  motivational,  and  discriminative 
functions— can  be  altered  by  verbal  processes  and  rela¬ 
tions.  This  new  principle,  while  entirely  consistent 
with  operant  theory,  has  important  implications  for 
how  complex  learning  can  be  influenced  (via  instruc¬ 
tional  means  or  otherwise)  and  provides  a  functional 
account  of  the  structure  of  verbal  knowledge  and  cog¬ 
nition.  A  full  account  of  RFT,  its  empirical  support, 
and  its  implications  is  not  possible  here,  but  interested 
readers  are  referred  to  a  comprehensive  text  on  the 
theory  (Hayes  et  al.,  2001),  an  online  tutorial  introduc¬ 
ing  its  basic  concepts  (Fox,  2005),  a  text  on  some  of 
its  applied  extensions  (Hayes  et  al.,  1999),  a  special 
issue  of  the  Journal  of  Organizational  Behavior  Man¬ 
agement  (Austin,  2006),  and  the  RFT  section  of  the 
website  for  the  Association  for  Contextual  Behavioral 
Science  (ACBS,  2005). 

CONCLUSION 

Pepper’s  worldview  analysis  ( 1942)  can  be  useful  for 
clarifying  underlying  philosophical  assumptions.  By 
highlighting  the  contextualistic  worldview  shared  by 
both  constructivist  and  behavioral  theories,  shared 
assumptions  and  key  differences  are  illuminated.  Both 
constructivism  and  behaviorism  have  contributed  sig¬ 
nificantly  to  instructional  design  and  educational  tech¬ 
nology  but  often  reach  contrary  conclusions  regarding 
instructional  practices.  A  better  understanding  of  their 
common  philosophical  heritage  may  lead  to  improved 
communication  and  collaboration,  with  the  under¬ 
standing  that  the  relative  value  of  each  perspective  will 
depend  upon  purpose  and  context  (Morris,  1993). 
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ABSTRACT 

Philosophical  perspectives  are  worldviews  that  define 
the  nature  of  the  world,  the  individual’s  place  in  it,  and 
the  possible  relationships  to  that  world  and  its  parts. 
Learning  and  instructional  theories  are  developed  with 
respect  to  a  particular  set  of  assumptions  regarding 
what  it  means  to  know  and  learn.  It  is  our  contention 
that  when  situational  variables  require  some  decision 
on  the  part  of  the  educator  (and  we  believe  this  is 
always  the  case),  an  underlying  set  of  assumptions 
(whether  they  be  tacit  or  explicit)  will,  and  should, 
drive  the  decision.  In  this  chapter  we  provide  overview 
descriptions  of  five  psychological  perspectives,  con¬ 
trasted  in  terms  of  epistemology,  ontology,  unit  of  anal¬ 
ysis,  and  whether  they  suggest  dualist  relationships. 
These  theories  (behaviorism,  cognitivism,  cognitive 
constructivism,  sociocultural/historicism,  and  situativ¬ 
ity  theory)  provide  frameworks  for  describing  learning 
and  designing  instruction.  It  is  the  goal  of  this  chapter 
to  clarify  these  distinctions  and  the  underlying  assump¬ 
tions  so  instructional  designers,  teachers,  and  research¬ 
ers  may  make  pedagogical  decisions  more  explicitly. 


KEYWORDS 

Behaviorism:  An  objectivist  and  monist  perspective 
with  regard  to  individual  actions  and  decisions . 

Cognitive  constructivism:  A  form  of  realism  that 
stresses  the  reorganization  of  mental  structures  of 
an  individual  making  sense  of  the  world. 

Cognitivism:  An  objectivist  and  rationalist  perspective 
with  regard  to  individual  cognitive  structures. 


Dualism:  When  two  apparently  related  items  are 
treated  as  separate  and  distinct  (e.g.,  mind/body  or 
individual/environment) . 

Empiricism:  An  epistemology  that  states  that  knowledge 
comes  from  experience  and  through  the  senses. 

Epistemology:  How  we  come  to  know  about  what 
exists. 

Idealism:  A  view  of  reality  as  mental,  implying  that 
the  world  is  not  separate  from  the  mind. 

Objectivism:  An  ontological  and  epistemological  view 
that  contends  that  reality  exists  outside  of  the  indi¬ 
vidual  and  consists  of  specific  entities. 

Ontology:  What  exists  in  the  world. 

Pragmatism:  The  view  that  knowledge  is  derived  from 
interaction  among  groups  of  individuals  and  the 
artifacts  in  their  environment,  which  together  cre¬ 
ate  a  reality. 

Rationalism:  An  epistemological  view  where  reason  is 
the  principle  source  of  knowledge. 

Realism:  A  form  of  objectivism  that  assumes  that  there 
is  some  sort  of  reality  that  is  separate  from  the  mind 
and  that  knowing  involves  a  correspondence 
between  the  world  and  the  mind. 

Relativism:  A  general  principle  that  places  the  meaning 
of  experiential  and  physical  events  in  the  relation¬ 
ships  that  exist  among  them. 

Situativity  theory:  A  form  of  realism  that  stresses  an 
individual’s  direct  perception  of  events  and  phe¬ 
nomena. 

Sociocultural/historicism:  A  relativist  perspective  that 
emphasizes  relations  and  processes  between  the 
individual  and  society. 

Unit  of  analysis:  Boundaries  of  the  phenomena  of 
interest. 
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INTRODUCTION 

Theoretical  perspectives  such  as  behaviorism,  cogni¬ 
tivism,  cognitive  constructivism,  sociocultural/histori- 
cism,  and  situativity  theory  provide  frameworks  for 
describing  learning  and  designing  instruction.  Finding 
roots  in  philosophy,  these  perspectives  differ  with 
respect  to  their  ontological  and  epistemological 
assumptions.  Learning  theories  and  instructional  the¬ 
ories  are  developed  and  linked  to  a  particular  set  of 
assumptions,  supposedly  consistent  with  one  of  the 
theoretical  perspectives.  Duffy  and  Jonassen  (1992) 
argued  that  instructional  strategies  and  methods  are 
clearly  influenced  by  the  philosophical  assumptions 
and  that  theories  of  knowing  and  learning  are  implicit 
in  the  instructional  design.  If  not  implemented  entirely 
by  a  cookbook  approach,  then  when  situational  vari¬ 
ables  require  some  decision  on  the  part  of  the  educator 
an  underlying  set  of  assumptions  (whether  they  be  tacit 
or  explicit)  will  drive  the  decision  (Barab  and  Duffy, 
2000).  It  is  inconceivable  that  a  teacher  or  instructional 
designer  would  advocate  a  particular  lesson  or  activity 
without  at  least  a  tacit  theory  of  how  students  think 
and  learn. 

In  the  literature,  we  see  various  classifications  for 
these  different  perspectives;  for  example,  Greeno  et  al. 
(1996)  described  behaviorist/empiricist,  cognitivist/ 
rationalist,  and  situative/pragmatist-sociohistorical 
perspectives.  Prawat  and  Floden  (1994)  used  world¬ 
views  to  define  their  classifications:  mechanistic 
(including  information  processing  approaches),  organ- 
ismic  (including  radical  constructivism),  and  contex- 
tualist  (including  social  constructivism) .  Wood  (1995) 
grounded  his  categories  in  the  application  of  learning 
theory  to  technology:  Skinner  and  neo-behaviorism, 
Piaget  and  constructivist  theory,  Vygotsky  and  social 
constructivism  and  situated  cognition. 

Unfortunately,  when  considering  theoretical  per¬ 
spectives  and  the  learning  theories  that  have  developed 
within  them,  it  is  not  always  clear  what  the  underlying 
philosophical  roots  are.  In  fact,  it  becomes  confusing 
when  considering  the  descriptions  from  authors  distin¬ 
guishing  differently  among  theoretical  perspectives. 
Driscoll  (1994),  for  example,  stated  that  Piaget’s  devel¬ 
opmental  theory  and  constructivism  were  interpretivist 
based.  Cobb  (1994),  when  distinguishing  among  the 
cognitive  and  socioconstructivists,  aligned  the  cogni¬ 
tive  constructivist  with  the  views  of  von  Glasersfeld 
(1989),  who  used  Piaget  as  his  example;  yet,  von  Gla¬ 
sersfeld  described  the  basis  as  pragmatist.  Greeno  and 
colleagues  (1996),  Cobb  (1994),  and  Driscoll  (1994) 
placed  the  socioconstructivist  or  sociohistorical  per¬ 
spective  under  the  roots  of  pragmatism  as  well,  calling 
on  the  views  of  Vygotsky.  Phillips  (1995)  distin¬ 


guished  among  the  various  sects  of  constructivism, 
placing  the  perspectives  by  Piaget  and  Vygotsky 
together  based  on  the  unit  of  analysis.  Greeno  and 
colleagues  (1996)  classified  constructivism  along  with 
cognitivism,  finding  roots  in  rationalism.  Ertmer  and 
Newby  (1993)  located  cognitivism  and  behaviorism 
within  the  objectivist  perspective,  with  cognitivism  as 
well  as  constructivism  being  rationalism. 

To  further  confound  things,  Garrison  (1995) 
equated  contemporary  social  constructivism  with  what 
he  defined  as  pragmatic  social  behaviorism  based  on 
the  work  of  Dewey,  indicating  a  relationship  between 
constructivism  and  adaptations  of  behaviorism.  Garri¬ 
son  also  stated  that  situated  cognition  (classified  within 
the  situative/pragmatist-sociohistoric  category  by 
Greeno  et  al.,  1996)  has  made  an  important  contribution 
to  social  constructivism.  Greeno  ( 1998)  stated  that  the 
situative  perspective  could  subsume  both  the  behavior- 
ist  and  the  cognitivist  perspectives.  Whereas  Prawat  and 
Floden  (1994)  combined  social  constructivism  and  sit¬ 
uativity  perspectives,  Derry  (1992)  distinguished 
between  constructivist  and  culturally  situated  learning 
views.  Among  situated  perspectives.  Lave  (1997)  fur¬ 
ther  divided  what  she  termed  cognition  plus,  interpre¬ 
tivist  views,  and  her  situated  social  practice  view. 

What  factors  are  being  used  to  distinguish  the 
above  theoretical  perspectives?  Epistemological  or 
ontological  assumptions?  For  seasoned  theorists  in  the 
field,  these  distinctions  may  be  trivial  to  sort  out,  or 
maybe  some  believe  that  these  distinctions  have  little 
practical  significance.  For  those  beginning  their  schol¬ 
arship  in  the  field,  confusion  seems  to  reign.  Further, 
for  those  interested  in  designing  practical  applications 
of  instruction  and  seeking  the  grounding  that  a  theo¬ 
retical  foundation  can  provide,  it  may  be  difficult  to 
understand  the  foundation  on  which  they  are  building. 

It  is  in  response  to  these  questions,  and  with  the 
goal  of  providing  sharper  boundaries  among  these  cat¬ 
egories  to  inform  learning  architects,  that  we  have 
written  this  chapter.  We  describe  theoretical  perspec¬ 
tives  as  subsumed  under  five  categories  and  clarify 
these  categories  by  defining  the  mind/body  relations, 
epistemology,  ontology,  and  the  unit  of  analysis.  Then, 
we  turn  to  the  instructional  implications  of  these  five 
categories.  It  is  important  to  recognize  that  these  dis¬ 
tinctions  are  situated  within  the  context  of  providing 
sharper  boundaries  and  stimulating  discussion.  As 
such,  we  have  drawn  lines  among  perspectives  that 
may  seem  overly  defined  and  may  not  even  exist  within 
other  contexts  and  for  other  purposes;  for  example, 
some  theorists  would  not  separate  situativity  and  con¬ 
structivist  perspectives  or  would  not  build  connections 
between  objectivism  and  cognitivism.  In  writing  this 
paper,  therefore,  we  have  made  epistemological  and 
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ontological  commitments  — a  process  that  we  suggest 
is  useful  for  instructional  designers  and  educational 
psychologists  to  undertake. 

This  is  not  to  imply  that  we  have  simply  constructed 
these  distinctions  based  on  our  fancy;  in  fact,  we  have 
aligned  each  conjecture  with  citations  from  colleagues 
in  the  field,  stacking  our  allies  if  you  will  (Latour, 
1987).  Our  hesitancy  in  forwarding  these  categories  is 
that  readers  will  take  these  brief  and  overly  simplified 
categories  as  fixed  and  rigid  rules  or,  even  worse,  as 
substitutes  for  involved  study  of  the  particular  philo¬ 
sophical  works  discussed.  It  is  our  intention  that  these 
categories  should  not  be  used  for  compartmentalizing 
but  should  serve  as  a  backdrop  for  continued  discussion 
and  for  broader  discourse  among  our  colleagues. 


BACKGROUND 

Epistemology 

Epistemology  and  ontology  are  within  the  founda¬ 
tional  realm  of  philosophy  and  mutually  support  one 
another  (Lombardo,  1987;  Reber,  1995).  Epistemology 
addresses  the  “origins,  nature,  methods,  and  limits  of 
human  knowledge”  (Reber,  1995,  p.  256),  focusing  on 
questions  about  knowledge  and  the  nature  of  knowl¬ 
edge  (Everitt  and  Fisher,  1995).  Those  interested  in 
learning  and  instruction  thus  have  an  epistemological 
purpose  (i.e.,  supporting  learners  in  coming  to  know) 
regardless  of  the  perspective  with  which  they  choose 
to  be  aligned.  Understanding  how  a  learner  comes  to 
know  and  how  that  process  can  be  facilitated  forms  a 
basis  for  research  in  learning  and  instruction. 

Ontology 

Ontology  is  a  branch  of  philosophy  (within  metaphys¬ 
ics)  that  addresses  the  nature  of  being  and  reality  (Lom¬ 
bardo,  1987;  Reber,  1995);  in  other  words,  an  ontology 
defines  what  is  real  in  the  world,  whether  physical  or 
abstract  structures.  Those  interested  in  learning  and 
instruction  indicate  their  ontological  preference  by  spec¬ 
ifying  what  are  considered  truths  about  knowledge, 
information,  and  the  world.  To  be  redundant,  yet  suc¬ 
cinct,  ontology  refers  to  “what  exists”  while  epistemol¬ 
ogy  is  concerned  with  "how  we  come  to  know  about” 
what  exists  (Barab  et  al.,  1999;  Jonassen,  1991). 

Unit  of  Analysis 

The  unit  of  analysis,  from  an  assessment  perspective, 
refers  to  the  phenomenon  of  interest  or,  more  specifi¬ 
cally,  the  boundaries  of  the  phenomenon  that  one  is 


attempting  to  measure  (Young  et  al.,  1997).  Salomon 
(1991),  for  example,  distinguished  between  the  ana¬ 
lytic  approach,  in  which  units  are  studied  in  isolation 
because  they  are  considered  to  be  discrete,  and  the 
systemic  approach,  in  which  the  units  are  considered 
to  be  interdependent  and  inseparable.  Units  of  analysis 
are  not  objective  features  that  are  selected  indepen¬ 
dently  of  a  theoretical  perspective;  rather,  the  bound¬ 
aries  of  the  phenomenon  that  one  is  attempting  to 
measure  are  influenced  by  a  theoretical  perspective 
(Barab  and  Duffy,  2000).  If  one  views  knowledge  as 
structures  existing  in  the  brain,  then  a  viable  unit  of 
analysis  would  be  to  examine  the  individual  (or  the 
cognitive  structures  of  that  individual)  in  isolation, 
whereas  one  who  views  knowledge  as  situationally 
constructed  would  pay  more  homage  and  would  nec¬ 
essarily  expand  the  unit  of  analysis  to  include  the 
surrounding  context  in  which  thinking  is  occurring. 
This  is  not  to  imply  there  is  a  one-to-one  correspon¬ 
dence  between  unit  of  analysis  and  philosophical 
assumptions;  instead,  the  unit  of  analysis  is  influenced 
and  constrained  by  underlying  ontological  and  episte¬ 
mological  assumptions.  In  this  chapter,  we  describe 
what  is  an  appropriate  (viable)  unit  of  analysis  given 
a  particular  set  of  assumptions. 

Dualisms 

The  Cartesian  dualism,  in  which  the  mind  is  consid¬ 
ered  distinct  from  the  body,  has  been  talked  about  in 
philosophy  and  psychology  since  the  inception  of 
these  two  disciplines.  Turvey  and  Shaw  (1995)  iden¬ 
tified  four  dualisms  that  have  been  central  to  psychol¬ 
ogy:  mind-body,  symbol-matter,  subjective-objec¬ 
tive,  and  perception-action.  They  claim  that  the 
organism-environment  dualism  subsumes  these  other 
four.  In  this  dualism,  the  organism  or  knower  is  con¬ 
sidered  to  be  independent  of  the  environment  or  what 
is  known.  Nowhere  have  these  dualisms  been  more 
apparent  than  in  theories  regarding  perception  (Barab 
et  al.,  1999;  Reed,  1996).  Various  theories  have  been 
forwarded  in  an  attempt  to  explain  how  the  mind 
perceives  objects  based  on  the  meaningless  points  of 
light  reaching  the  eye;  for  example,  in  addition  to  the 
dualist  theory  of  the  structuralists  and  the  monist  the¬ 
ory  of  the  materialists,  functionalism  has  been  for¬ 
warded  in  which  it  is  postulated  that  mental  states 
exist  as  a  function  of  a  system.  In  other  words,  the 
way  the  system  is  put  together  is  what  is  critical, 
rather  than  the  material  that  the  system  is  made  of, 
thus  allowing  that  a  system,  and  not  necessarily  the 
human  brain,  can  give  rise  to  mental  states  (Fodor, 
1994).  From  a  functionalist  lens,  the  mind  and  brain 
are  viewed  as  one,  with  “the  mind  being  viewed  as 
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TABLE  7.1 

Characteristics  for  Differentiating  among 
Philosophical  Perspectives 


Factors 

Ontology 
Epistemology 
Unit  of  analysis 
Dualisms 


Definition 

What  exists  in  the  world 
How  we  come  to  know  about  what  exists 
Boundaries  of  the  phenomenon  of  interest 
When  two  items  are  treated  as  separate  (e.g., 
mind/body  or  individual/environment) 


‘the  brain  looked  at  from  the  inside’  and  the  brain  as 
‘the  mind  looked  at  from  the  outside’”  (Turvey  and 
Shaw,  1995,  p.  146). 

Categories  Summarized 

It  is  these  categories  that  will  provide  a  basis  for  dis¬ 
tinguishing  among  philosophical  perspectives.  Table 
7.1  provides  a  summary  of  these  categories. 


outside  of  the  individual  (Bednar  et  al.,  1995;  Driscoll, 
1994;  Jonassen,  1991;  Jonassen  et  al.,  1993).  Episte¬ 
mologically,  the  mind  functions  as  a  mirror  of  nature, 
creating  representations  of  the  real  world  that  require 
a  correspondence  to  the  external  world.  To  know  is  to 
have  these  correct  representations  (Lakoff,  1987). 

Realism 

Realism  is  an  ontological  view  of  which  objectivism  is 
one  form  (Lakoff,  1987).  Both  realism  and  objectivism 
support  the  existence  of  a  real,  physical  world  that  is 
external  to  individuals  and  includes  human  experience. 
Although  objectivism  provides  a  specific  description  of 
what  the  real  world  must  be,  in  terms  of  entities  and 
properties,  realism  “merely  assumes  that  there  is  a  real¬ 
ity  of  some  sort”  (Lombardo,  1987,  p.  159).  From  this 
point  of  view,  the  physical  world  is  a  separate  reality 
from  perception  and  the  mind  (Mackay,  1997;  Reber, 
1995)  and  truth  or  knowledge  is  ascertained  as  having 
a  correspondence  between  the  structures  of  the  mind 
and  what  is  present  in  the  world  (Prawat,  1995). 


PHILOSOPHICAL  PERSPECTIVES 

Philosophical  perspectives  reflect  certain  assumptions 
with  respect  to  the  nature  of  the  world  and  how  we  come 
to  know  about  it;  however,  these  are  sets  of  beliefs  and 
are  not  open  to  proof  in  the  positivist  sense  of  the  word: 
“There  is  no  way  to  elevate  one  over  another  on  the 
basis  of  ultimate  ,  foundational  criteria”  (Guba  and  Lin¬ 
coln,  1983,  p.  108).  This  does  not  mean  that  we  submit 
to  a  radical  relativist  posture  (Bereiter,  1994a;  Guba, 
1992).  These  perspectives  represent  certain  sets  of 
assumptions  and  commitments  with  respect  to  world¬ 
views,  and  advocates  of  a  particular  perspective  must 
rely  on  persuasiveness,  assemblage  of  allies  (other  col¬ 
leagues)  (Latour,  1987),  and  utility  rather  than  proof  in 
arguing  their  position  (Bereiter,  1994b).  In  this  section, 
we  present  our  interpretations  (again,  citing  our  col¬ 
leagues  to  ground  and  add  credibility  to  our  conjectures) 
on  the  defining  characteristics  of  these  categories. 

Objectivism 

Objectivism  is  described  as  both  an  ontology  and  epis¬ 
temology  (Lakoff,  1987).  Ontologically,  “all  reality 
consists  of  entities,  which  have  fixed  properties  and 
relations  holding  among  them  at  any  instant”  (Lakoff, 
1987,  p.  160).  The  world  consists  of  these  entities, 
their  properties,  and  the  relations  that  exist  among 
them.  Reality  exists  through  the  structures  of  these 
entities  and  is  independent  of  any  human  understand¬ 
ing  (Lakoff,  1987);  thus,  the  world  is  real  and  exists 


Empiricism 

As  “typified  by  Locke  and  Thorndike,  [empiricism] 
emphasizes  consistency  of  knowledge  with  experi¬ 
ence”  (Greeno  et  al.,  1996,  p.  16).  It  is  an  epistemo¬ 
logical  perspective  that  holds  that  knowledge  builds 
from  experience,  more  specifically,  from  the  senses 
(Driscoll,  1994;  Ertmer  and  Newby,  1993;  Gardner, 
1985;  Lombardo,  1987;  Reber,  1995;  Traiger,  1994). 
Empiricism  rejects  the  notion  that  the  human  mind 
enters  the  world  with  a  priori  ideas  and  concepts  that 
exist  independently  of  personal  experience  (Reber, 
1995);  thus,  what  is  learned  comes  from  interactions 
with  the  environment  (Ertmer  and  Newby,  1993).  An 
empiricist  would  choose  actual  data  over  theoretical 
conjectures  and  would  formulate  an  argument  based 
on  the  evidence  of  experience. 

Rationalism  and  Idealism 

Rationalism  is  generally  discussed  from  an  epistemo¬ 
logical  view.  From  a  rationalist  perspective,  reason  is 
the  principle  source  of  knowledge  (Lombardo,  1987; 
Reber,  1995;  Traiger,  1994).  This  reasoning  power 
imposes  upon  the  sensory  experience  that  arises  in  the 
world,  thus  creating  the  world  itself  (Gardner,  1985). 
Early  versions  of  rationalism  posited  that  everything 
existed  in  one’s  mind  a  priori ,  and  a  learner’s  task  was 
to  discover  what  was  already  there,  as  in  Plato’s  Meno 
(Plato,  1977).  The  ontological  base  of  idealism,  more 
pronounced  than  what  might  be  implied  by  rationalism. 
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holds  that  reality  is  psychological  and  all  knowledge 
and  experience  are  formed  by  these  mental  representa¬ 
tions  (Reber,  1995);  thus,  a  separate  world  of  physical 
entities  or  matter  is  not  supported  (Lombardo,  1987). 

Relativism 

Relativism,  a  general  principle  rather  than  specifically 
a  philosophical  perspective,  puts  the  meaning  of  expe¬ 
riential  and  physical  events  in  the  relationships  that 
exist  among  them.  In  this,  there  is  no  intrinsic  meaning 
that  is  independent  of  other  events  (Reber,  1995).  Real¬ 
ity  from  this  perspective  is  socially  and  experientially 
based,  being  local  and  specific  to  observer  and  context 
(Guba  and  Lincoln,  1983).  There  is  no  absolute  truth 
to  the  world;  instead,  there  are  individual  constructions 
that  are  highly  dependent  on  the  individual  building 
the  constructions. 

Pragmatism 

Based  on  an  Aristotelian  heritage,  a  line  of  thought 
emerged  that  challenged  the  analytic,  static,  and  seg¬ 
mented  thought  of  absolute  dualism.  This  was  partic¬ 
ularly  evident  in  the  natural  sciences,  where  "the  struc¬ 
tures  and  capacities  of  animals  were  described  relative 
to  their  ways  of  life  within  an  environment;  in  turn,  the 
environment  was  described  relative  to  the  ways  of  life 
of  animals”  (Lombardo,  1987,  p.  5).  In  psychology,  the 
pragmatists  (also  called  functionalists)  were  less  con¬ 
cerned  with  the  inherent  structure  of  the  mind  than  with 
what  the  mind  could  do.  The  central  focus  of  pragma¬ 
tists  (C.S.  Peirce,  William  James,  and  John  Dewey) 
was  on  what  adaptive  purposes  justify  the  existence  of 
mind  (Turvey  and  Shaw,  1996).  Rorty  (1991)  stated 
that  pragmatism  requires  neither  “a  metaphysics 
[ontology]  or  an  epistemology.  They  [pragmatists] 
view  truth  as,  in  William  James’  phrase,  what  is  good 
for  us  to  believe”  (p.  22,  emphasis  in  original).  The 
truth,  or  knowledge,  is  equivalent  to  the  consequences 
that  derive  from  these  interactions  (Reber,  1995).  What 
makes  a  particular  stance  count  as  a  truth  is  not  some 
correspondence  with  the  real  world  but  its  appropriate¬ 
ness  in  terms  of  whether  it  is  progressive  (functional) 
(Barab  and  Squire,  2004;  Bereiter,  1994b). 

Philosophical  Views:  Summarized 

The  above  by  no  means  accounts  for  all  philosoph¬ 
ical  perspectives .  We  have  chosen  to  define  those  that 
will  be  most  salient  to  our  discussion  of  psycholog¬ 
ical  perspectives  as  they  relate  to  learning  and 
instruction.  Table  7.2  contains  a  summary  of  these 
philosophical  perspectives. 


TABLE  7.2 

Philosophical  Perspectives  Summarized 


Philosophical 

Perspective 

Objectivism 


Realism 


Empiricism 

Rationalism 

Idealism 

Relativism 


Pragmatism 


Definition 

Ontological  and  epistemological  view  that 
contends  that  reality  (the  world)  exists  outside 
of  the  individual  and  consists  of  specific  entities; 
to  know  is  to  have  a  mirroring  of  this  world 
Form  of  objectivism  that  assumes  that  some  sort 
of  reality  is  separate  from  the  mind  (ontology); 
to  know  is  to  have  a  correspondence  between 
the  word  and  the  mind  (epistemology) 
Epistemology  that  states  that  knowledge  comes 
from  experience  and  through  the  senses 
Essentially  an  epistemological  view  where 
reason  is  the  principle  source  of  knowledge 
Defines  reality  as  mental,  meaning  that  the  world 
is  not  separate  from  the  mind  (ontology) 

A  general  principle  that  places  the  meaning  of 
experiential  and  physical  events  in  the 
relationships  that  exist  among  them 
Neither  an  epistemology  or  an  ontology; 
knowledge  is  derived  from  interaction  among 
groups  of  individuals  and  the  artifacts  in  their 
environment,  both  of  which  create  a  reality 


PSYCHOLOGICAL  PERSPECTIVES 
DISTINGUISHED 

Considering  ontological  and  epistemological  assump¬ 
tions,  units  of  analysis,  and  the  mind-body  relation,  we 
propose  a  classification  model  that  includes  five  main 
categories  of  current  psychological  perspectives  which 
have  provided  a  foundation  for  learning  and  instruc¬ 
tional  theories.  It  is  not  our  intent  to  add  to  the  myriad 
of  terms  used  to  describe  theoretical  categories;  con¬ 
sider,  for  example,  the  many  names  of  various  types 
of  constructivism:  information-processing  constructiv¬ 
ism  (Prawat  and  Floden,  1994),  cognitive  information 
processing,  (Derry,  1992),  radical  constructivism  or 
cognitive  constructivism  (Cobb,  1994;  Derry,  1992; 
Duffy  and  Cunningham,  1996;  Prawat  and  Floden, 
1994;  von  Glasersfeld,  1995),  and  sociocultural  con¬ 
structivism  (sociohistorical)  (Cobb,  1994;  Duffy  and 
Cunningham,  1996;  Prawat  and  Floden,  1994). 

With  that  in  mind,  we  have  chosen  the  following 
categories:  behaviorism,  cognitivism,  cognitive  con¬ 
structivism  (to  keep  in  the  forefront  the  focus  on  indi¬ 
vidual  mind),  sociocultural/historicism  (to  keep  in  the 
forefront  the  focus  on  interactions  among  individuals 
and  among  individuals  and  society) ,  and  situativity  the¬ 
ory  (to  keep  in  the  forefront  the  focus  on  interactions 
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among  individuals  and  the  situations  in  which  they  are 
acting).  As  Greeno  and  colleagues  (1996)  stated  about 
the  boundaries  for  the  three  perspectives  they  outlined, 
boundaries  are,  of  course,  relatively  arbitrary.  The  cat¬ 
egories  we  discuss  are  not  intended  to  define  single 
boundaries  among  perspectives  but  rather  to  illuminate 
distinctions  that  become  clear  only  by  looking  at  the 
jigsaw  that  emerges  as  one  variable  (ontology,  episte¬ 
mology,  unit  of  analysis,  dualism)  provides  a  distinc¬ 
tion  among  two  perspectives  while  another  variable 
suggests  their  similarity. 

Behaviorism 

Behaviorism  was  the  predominant  psychological 
school  of  thought  during  the  first  half  the  20th  century. 
Proponents  of  the  field  were  Pavlov,  Thorndike,  Wat¬ 
son,  Tolman,  Hull,  and  Skinner. 

Ontology  I  Epistemology 

There  seems  to  be  much  agreement  that  behaviorism’s 
ontological  roots  are  objeetivist  (Driscoll,  1994;  Duffy 
and  Jonassen,  1992;  Ertmer  and  Newby,  1993;  Greeno 
et  al.,  1996;  Jonassen,  1991).  The  world  is  real  and 
exists  outside  of  the  individual.  To  come  to  know 
something  within  a  behaviorist  framework  is  to  come 
to  engage  in  specific  behaviors  in  the  context  of  par¬ 
ticular  stimuli.  Burton  et  al.  (1996)  summarize  three 
types  of  learning  through  which  this  can  occur:  respon¬ 
dent  learning  (e.g.,  use  of  classical  conditioning  where 
involuntary  actions  are  elicited),  operant  conditioning 
(development  of  a  relationship  between  a  stimulus  and 
response),  and  observational  learning  (change  of 
behavior  brought  about  by  experience  of  observing 
others,  of  which  Bandura’s  work  is  an  example) .  Thus , 
the  epistemological  framework  for  behaviorism  also 
finds  roots  in  empiricism  (Ertmer  and  Newby,  1993; 
Greeno  et  al.,  1996). 

Unit  of  Analysis 

To  illuminate  the  focal  point  of  knowledge,  from  a 
behaviorist  perspective  the  unit  of  analysis  is  the 
behavior  of  an  individual  and  the  stimuli  that  elicit  it. 
The  mind  is  considered  beyond  inspection  and  not 
relevant  to  explaining  behavior— no  need  to  open  up 
the  black  box  (Gardner,  1985). 

Dualism  Perspective 

Because  of  its  objeetivist  ontological  roots,  behavior¬ 
ism  supports  a  type  of  dualism  that  distinguishes 
between  knower  and  world;  however,  behaviorism  is 


considered  an  anti-Cartesian  school  of  psychology,  as 
“all  mentalist  terms  can  be  redefined  in  terms  of 
observable,  physically  describable  behavior”  (Garfield, 
1995,  p.  336),  thus  nothing  can  have  a  mental  property 
without  a  physical  property.  Rather  than  dualism, 
behaviorism  is  a  form  of  monism. 

Cognitivism 

Cognitivism,  as  we  describe  it  here,  is  that  which 
initially  emerged  with  the  cognitive  revolution  during 
the  1950s,  stressing  a  renewed  focus  on  mind.  As 
described  by  Bruner  (1990,  p.  1),  the  “revolution  was 
intended  to  bring  the  ‘mind’  back  into  the  human 
sciences  after  a  long  cold  winter  of  objectivism.” 
They  “were  not  out  to  ‘reform’  behaviorism,  but  to 
replace  it”  (1990,  p.  3).  Cognitivism  grew  from  struc¬ 
turalism,  where  mental  states  are  viewed  as  the  com¬ 
putational  states  of  a  Turing  machine  or  as  the  time- 
evolving  states  of  a  connectionist  (neural  network) 
machine”  (Turvey  and  Shaw,  1995,  pp.  146-147). 
This  focus  on  mind,  in  which  the  mind  was  viewed 
as  an  information-processing  system  as  exemplified 
by  the  mind-as-computer  metaphor  that  emerged, 
sought  an  understanding  of  the  organization,  encod¬ 
ing,  and  retrieval  of  knowledge.  We  consider  sym¬ 
bolic  information  processing  as  described  by  Greeno 
et  al.  (1996)  and  cognitive  symbolic  processing 
(Derry,  1992)  to  be  synonymous  and  exemplify  this 
view. 

Ontology /Epistemology 

Although,  as  Bruner  (1990)  stated,  the  cognitive  rev¬ 
olution  was  a  reaction  to  the  cold  winter  of  objectiv¬ 
ism,  cognitivism  still  finds  roots  in  objectivism  as  an 
ontological  base.  As  stated  by  Ertmer  and  Newby 
(1993),  behaviorism  and  cognitivism  are  both  prima¬ 
rily  objective,  and  the  world  is  real  and  external  to  the 
learner.  Duffy  and  Jonassen  (1992,  p.  3)  stated  that 
much  of  the  information-processing  view  in  cognitive 
psychology  is  based  on  an  objeetivist  epistemology, 
supporting  this  statement  by  citing  the  “independent 
existence  of  information  and  the  acquisition  of  that 
information.”  A  direct  mapping  of  the  world,  a  knowl¬ 
edge  base  of  expert  information,  can  be  accomplished 
through  learning.  Although  this  acquisition  of  a  knowl¬ 
edge  structure  does  underlie  the  cognitive  perspective, 
we  feel  that  a  rationalist  epistemological  base  provides 
the  distinction  required  to  define  a  meaningful  bound¬ 
ary  between  behaviorism  and  cognitivism  (Ertmer  and 
Newby,  1993).  Greeno  and  colleagues’  (1996)  theoret¬ 
ical  conceptualization  is  based  on  issues  about  the 
nature  of  knowing,  essentially  epistemology,  as  well 
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as  learning,  transfer,  motivation,  and  engagement,  and 
describes  behaviorism  as  objectivism  and  cognitivism 
as  rationalism  as  well. 

Unit  of  Analysis 

Like  the  behaviorists,  the  unit  of  analysis  remains  with 
the  individual;  however,  rather  than  behavior,  it  is  an 
analysis  of  an  individual’s  mind  structure  and  the  rep¬ 
resentations  developed. 

Dualism  Perspective 

With  cognitivism’s  roots  in  rationalism,  mind  and  envi¬ 
ronment  would  not  be  separated,  the  world  existing 
only  through  mind;  however,  this  is  an  extreme  inter¬ 
pretation  of  rationalism  as  applied  to  cognitivism. 
Bredo  (1994)  itemized  a  number  of  dualisms  that  arise 
within  this  view:  language  and  reality,  mind  and  body, 
and  individual  and  group.  For  our  discussion,  we  align 
more  with  Turvey  and  Shaw  (1995),  who  described 
cognitivism  as  growing  out  of  19th-century  structural¬ 
ism,  where  the  mind  and  body  are  seen  as  separate  and 
interactive. 

Cognitive  Constructivism 

Cognitive  constructivism  is  generally  aligned  with  the 
work  of  Piaget.  This  link  to  Piaget  synthesized  the 
various  names  that  had  been  applied  to  this  perspec¬ 
tive:  radical  constructivism  (Derry,  1996;  von  Glaser- 
sfeld,  1995),  cognitive  constructivism  (Cobb,  1994; 
Duffy  and  Cunningham,  1996),  psychological  con¬ 
structivism  (Prawat,  1995),  and  constructivism  (Bed- 
nar  et  al.,  1995).  This  view,  we  believe,  has  emerged 
as  an  entity  separate  from  cognitivism  and  social  con¬ 
structivism  based  on  ontological  differences  on  the  one 
hand  and  the  unit  of  analysis  on  the  other. 

Ontology /Epistemology 

From  the  cognitive  constructivist  perspective,  there 
is  a  real  world  that  we  experience  (Duffy  and  Jonas- 
sen,  1992),  thus  appearing  to  find  an  ontological  base 
in  objectivism;  however,  this  world  cannot  be 
directly  known  (Derry,  1992;  von  Glaserfeld,  1995), 
which  broadens  the  nature  of  the  ontology  to  realism. 
That  reality  exists  is  not  denied;  however,  what  we 
know  of  the  world  is  only  an  interpretation  based  on 
our  experiences  (von  Glasersfeld,  1995).  As  such, 
cognitive  constructivism  is  subjective  and  relativist, 
providing  for  no  absolute  in  what  is  right  or  wrong 
and  also  has  a  base  in  empiricism  (Bednar  et  al., 
1995).  Cognitive  constructivism  also  finds  its  epis¬ 


temological  basis  largely  in  rationalism  (Greeno  et 
al.  1996).  The  emphasis  in  knowing  is  in  the  cogni¬ 
tive  activity  of  an  individual  as  they  make  sense  of 
the  world  (Cobb,  1994). 

Unit  of  Analysis 

The  unit  of  analysis  within  the  cognitive  constructiv¬ 
ist  perspective  remains  with  the  individual  as  in  cog¬ 
nitivism  but  focuses  on  the  conceptual  reorganization 
of  one’s  knowledge  rather  than  on  the  extant  structure 
of  an  individual’s  knowledge.  Looking  at  the  cogni¬ 
tive  self-organization  of  the  individual,  Cobb  (1994, 
p.  15)  stated  that  “constructivists  are  typically  con¬ 
cerned  with  the  quality  of  individual  interpretive 
activity.” 

Dualism  Perspective 

As  described  by  Cobb  (1994),  the  cognitive  construc¬ 
tivist  perspective  places  the  mind  in  the  head  of  the 
individual.  Culture  and  context  play  a  role  in  the  mean¬ 
ing  making  of  each  individual.  The  individual  mind, 
although  influenced  by  social  context,  is  not  one  with 
the  social  context  (Bereiter,  1994b);  thus,  mind  is  sep¬ 
arate  from  environment  (Prawat  and  Floden,  1994). 

Sociocultural/Historicism 

Socioculturalism  is  often  distinguished  from  cognitive 
constructivism  (Cobb,  1994;  Duffy  and  Cunningham, 
1996;  Phillips,  1995;  Wood,  1995).  Generally,  this  dis¬ 
tinction  is  noted  either  through  the  name  of  construc¬ 
tivism  as  socioconstrucdvism,  sociocultural  construc¬ 
tivism  (Cobb,  1994),  social  constructivism,  or 
sociohistoricism  (Strauss,  1993)  or  by  distinguishing 
among  the  proponents  of  the  perspective.  Sociocon¬ 
structivism,  as  we  will  identify  it,  has  traditionally 
been  aligned  with  the  views  of  Vygotsky. 

Ontology /Epistemology 

Distinguishing  among  the  forms  of  constructivism  is 
largely  an  epistemological  distinction  (Cobb,  1994; 
Phillips,  1995).  In  this  perspective,  knowing  is  dis¬ 
tributed  in  the  world,  among  objects  and  individuals. 
Knowledge  creation  is  a  shared  rather  than  an  indi¬ 
vidual  experience  (Prawat  and  Floden,  1994)  and 
evolves  through  social  negotiation  (Savery  and 
Duffy,  1995).  Phillips  (1995)  described  this  empha¬ 
sis  on  sociopolitical  processes  or  consensus  as  a 
tendency  toward  relativism,  where  meaning  was  only 
incurred  with  respect  to  relationships,  thus  providing 
an  epistemological  distinction  among  constructivist 
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theoretical  perspectives.  Prawat  and  Floden  (1994) 
discussed  socioconstructivism  from  a  contextualist 
worldview,  supporting  the  notion  that  from  this  per¬ 
spective  knowledge  by  verification  is  linked  to 
actions  and  events  that  occur.  Thus ,  ontologically,  as 
with  the  cognitive  constructivists,  reality  exists 
through  interpretations;  however,  from  the  sociocul¬ 
tural  perspective,  society  and  an  individual’s  rela¬ 
tionship  to  society  have  a  primary  role  in  the  shaping 
of  that  reality  (Prawat  and  Floden,  1994). 

Unit  of  Analysis 

Socioculturalists  are  distinctive  in  their  insistence  that 
knowledge  creation  is  a  shared  rather  than  an  individ¬ 
ual  experience  (Prawat  and  Floden,  1994).  It  is  this 
process,  rather  than  the  mental  structures  of  the  indi¬ 
vidual  or  the  environment,  that  is  the  unit  of  analysis 
(Strauss,  1993);  thus,  the  mind  is  placed  in  society, 
and  the  individual’s  cognitive  structure  as  a  unit  of 
analysis  is  essentially  meaningless. 

Dualism  Perspective 

To  understand  the  view  on  dualism  from  this  perspec¬ 
tive,  we  find  Bakhurst’s  description  on  the  nature  of 
thought  as  cited  in  Cobb  (1994,  p.  14)  to  be  helpful: 

[TJhought  should  be  viewed  as  something  essentially  “on 
the  surface,”  as  something  located  ...  on  the  borderline 
between  the  organism  and  the  outside  world.  For  thought 
...  has  a  life  only  in  an  environment  of  socially  consti¬ 
tuted  meanings. 

Thus,  a  type  of  dualism  is  supported  between  individ¬ 
ual  and  environment.  Although  a  thought  may  only 
exist  in  the  socially  constructed  world,  the  individual 
and  environment  are  not  one  and  the  same. 

Situativity  Theory 

Young  (1993)  and  Greeno  (1998)  have  discussed  the 
ecological  tenets  of  situativity  theory— that  is,  the 
notion  of  direct  perception  without  the  need  for 
mediating  variables  that  exist  in  one’s  head.  Other 
roots  to  situativity  theory  can  be  found  in  the  work 
of  various  anthropologists  such  as  Lave  and  Wenger 
(1991),  describing  learning  and  cognition  in  the 
everyday  world  (Kirshner  and  Whitson,  1997).  For 
our  discussion,  we  will  not  distinguish  among  these 
perspectives  of  theory,  choosing  features  of  each 
which,  to  us,  clarify  the  essence  of  situativity  theory 
and  distinguish  it  from  the  other  psychological  per¬ 
spectives  discussed  above. 


Ontology /Epistemology 

Lombardo  (1987)  described  the  ontological  roots  of 
ecological  psychology  as  ecological  reciprocity,  the 
dynamical  relationships  that  occur  among  aspects  of 
the  environment.  The  environment,  in  this  perspective, 
is  not  distinct  from  the  individual;  the  individual  is 
merely  a  part  of  a  highly  interconnected  system  of 
relationships  (Barab  and  Roth,  2006).  Within  this  envi¬ 
ronment,  we  perceive  and  act  based  upon  the  affor- 
dances  and  constraints  of  the  environment  and  the 
situations  that  arise  within  it.  The  epistemological 
basis  of  situativity,  although  seemingly  not  distin¬ 
guishable  from  its  ontology,  was  described  by  Lom¬ 
bardo  (1996)  as  direct  realism,  which  was  foundational 
to  the  work  of  Gibson.  Rather  than  viewing  experi¬ 
ences  as  constituting  the  mind,  the  issue  becomes  what 
adaptive  purposes  exist  to  justify  the  existence  of  the 
mind  (Turvey  and  Shaw,  1995). 

A  fundamental  notion  for  the  situativity  theorist 
from  the  anthropological  view  is  that  “cognition  must 
be  viewed  as  an  integral  part  of  the  physical,  social, 
and  cultural  contexts  to  which  it  belongs”  (Barab  and 
Plucker,  2002;  Derry,  1996,  p.  416).  Learning,  or  com¬ 
ing  to  know,  has  a  defining  characteristic  of  an  indi¬ 
vidual  involved  in  legitimate  peripheral  participation 
within  the  practices  of  a  community  (Lave  and  Wenger, 
1991).  From  this  perspective,  knowing,  identity,  and 
context  stand  in  dialectic,  not  dualistic,  relations  and 
are  all  constituted  in  the  learning  process  (Barab  and 
Duffy,  2000;  Lave,  1997).  Not  only  does  this  learning 
take  place  within  the  practices  of  the  community,  but 
also  the  social  practices  of  the  world  are  developed 
through  this  process.  Thus,  a  reciprocity  emerges  as 
also  defined  from  the  ecological  view. 

Unit  of  Analysis 

The  unit  of  analysis  in  this  perspective  is  the  “socio¬ 
cultural  setting  in  which  the  activities  are  embedded” 
(Kirshner  and  Whitson,  1997,  p.  5).  More  so,  it  is  the 
ecosystem  that  exists  of  which  the  learner  is  one  part, 
and  it  is  the  individual-ecosystem  interactions  that 
must  be  captured  from  this  perspective  (Barab  and 
Kirshner,  2001) 

Dualism  Perspective 

The  situative  perspective  requires  unification  of  cogni¬ 
tion  and  nature  (Turvey  and  Shaw,  1995).  In  this,  there 
is  not  a  dualism  of  mind  and  environment;  instead, 
there  exists,  as  described  by  Turvey  and  Shaw,  organ¬ 
ism-  environment  mutuality  and  reciprocity.  Thus,  we 
find  a  distinction  between  the  situative  perspective  and 
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TABLE  7.3 

Psychological  Perspectives:  Epistemology,  Ontology,  Unit  of  Analysis,  and  Dualism  Perspective 


Behaviorism 

Cognitivism 

Cognitive  constructivism 

Sociocultural/historicism 
Situativity  theory 


Epistemology/ 

Ontology 

Objectivism 

Objectivism 

Rationalism 

Rationalism 

Realism 

Relativism 

Ecological  realism 


Unit  of  Analysis 

Behavior  of  the  individual 
Cognitive  structures  of  the  individual 

Reorganization  of  mental  structures  of  an  individual  making 
sense  of  the  world 

Relation  (and  processes)  between  the  individual  and  society 
Ecosystem  of  which  the  individual  is  a  part 


Dualism  Perspective 

Monism 

Mind/environment 

Mind/ environment 

Individual/environment 

Mutualism 


the  social  constructivist  perspective.  Situativity  theory 
defines  no  borderline  between  an  organism  and  the 
environment  as  did  Bakhurst;  instead,  from  a  situativ¬ 
ity  perspective  they  are  one. 

Psychological  Views:  Summarized 

We  have  outlined  five  categories  of  psychological  per¬ 
spectives  that  provide  a  basis  in  describing  learning 
and  instruction.  Epistemology,  ontology,  unit  of  anal¬ 
ysis,  and  the  position  on  mind-environment  relation¬ 
ships  define  these  categories  as  we  see  them.  Table  7.3 
provides  a  summary. 


IMPLICATIONS  FOR 
LEARNING  THEORY  AND 
INSTRUCTIONAL  METHODS 

Theoretical  perspectives  have  been  used  as  a  founda¬ 
tion  for  learning  theories  and,  in  turn,  instructional 
theories  and  their  associated  methods.  Learning  theo¬ 
ries  are  descriptive  (Prawat,  1992;  Reigeluth,  1999b), 
describing  how  learning  occurs  within  a  particular 
instance.  An  example  of  a  learning  theory  within  the 
cognitive  perspective  is  schema  theory,  which 
describes  how  learning  occurs  through  accretion,  tun¬ 
ing,  and  restructuring  (Rumelhart,  1981).  In  contrast, 
instructional  theories  are  prescriptive  and  provide 
guidance  about  how  to  design  instruction  to  facilitate 
learning.  An  instructional  theory  identifies  methods  to 
be  used  and,  more  importantly,  identities  situations  in 
which  these  methods  should  or  should  not  be  used. 

The  elaboration  theory  (Reigeluth,  1999a)  is  an 
example  of  an  instructional  theory  identifying  how 
instruction  should  be  sequenced  so  the  learning  pro¬ 
cess  can  be  meaningful  and  motivational  for  learning, 
allowing  for  simplification  of  content  without  decon¬ 
structing  the  learning  task  into  meaningless,  decontex- 
tualized  pieces.  An  instructional  theory  is  probabilis¬ 


tic,  rather  than  deterministic,  increasing  the  chances 
of  achieving  learning  goals  rather  than  guaranteeing 
them.  The  methods  identified  within  an  instructional 
theory  provide  specific  ways  in  which  to  support  and 
facilitate  learning.  The  simplifying  conditions  method 
provides  a  means  to  analyze,  select,  and  sequence  the 
content  of  what  is  to  be  learned  for  an  instructional 
designer  who  is  subscribing  to  the  elaboration  theory 
in  designing  a  learning  activity  (Reigeluth,  1999a). 

Instructional  theories  and  their  associated  methods 
are  based  on  learning  theories,  providing  an  important 
linkage  between  how  learning  occurs  and  how  to  facil¬ 
itate  that  process.  Learning  theories  reflect  the  theoret¬ 
ical  perspective,  defining  learning  in  terms  of  the  nature 
of  knowledge  of  the  world  and  how  one  comes  to  know 
about  these;  however,  the  linkages  among  psychologi¬ 
cal  perspective,  learning  theory,  and  instructional  the¬ 
ory  and  associated  methods  are  not  necessarily  simple 
and  direct.  Historically,  learning  theories  may  have 
been  confined  to  a  single  psychological  perspective  (C. 
M.  Reigeluth,  pers.  comm.,  August  24,  1998);  for 
example,  there  is  little  question  that  operant  condition¬ 
ing  is  aligned  with  behaviorism.  Yet,  this  clear  distinc¬ 
tion  may  not  be  the  norm  as  psychological  perspectives 
are  further  delineated  and  new  and  existing  theories  are 
considered  in  light  of  those  perspectives;  for  example, 
Reigeluth  (pers.  comm.,  August  24,  1998)  stated  the 
following  about  his  theories: 

While  the  cognitive  perspective  provides  rationale  for 
many  of  the  prescriptions  in  the  elaboration  theory,  con¬ 
structivism  provides  rationale  for  others,  and  behavior¬ 
ism  could  even  provide  rationale  for  some.  Also,  the 
elaboration  theory  can  be  used  in  different  ways  depend¬ 
ing  on  the  philosophical  orientation  of  the  user.  For 
example,  it  could  be  a  central  part  of  self-directed,  prob¬ 
lem-based  learning,  or  it  could  be  used  to  sequence 
behaviorist  instruction.  I  think  we  often  do  a  disservice 
by  trying  to  pigeon-hole  particular  methods  into  different 
philosophical  and  descriptive-theoretical  orientation.  I 
find  that  the  real  world  is  a  bit  more  complex  than  such 
simplistic  categorizations. 
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We  agree  with  Reigeluth’s  concern  about  categorizing 
particular  theories  and  methods  with  particular  philo¬ 
sophical  perspectives;  however,  we  also  believe  that 
those  who  are  designing  instruction  (instructional 
designers  as  well  as  classroom  instructors)  will  benefit 
from  a  discussion  of  the  differences  in  learning  theo¬ 
ries  and  instructional  theories.  What  these  theories 
imply  about  the  nature  of  the  world,  knowledge,  and 
the  learning  unit  that  is  of  interest  provides  a  theoret¬ 
ical  basis  for  designing  instruction. 

In  the  following  paragraphs,  we  discuss  the  learn¬ 
ing  and  instructional  implications  of  each  of  the  five 
identified  theoretical  perspectives.  For  each  theoretical 
perspective,  we  identify  a  learning  theory  and  instruc¬ 
tional  theory  or  method  that  we  feel  is  an  exemplar 
given  our  description  of  the  psychological  perspective. 
The  description  provided  of  each  exemplar  is  brief, 
allowing  the  reader  to  rely  on  his  or  her  own  resources 
for  a  more  involved  description  of  the  theories.  It  is 
important,  again,  to  keep  in  mind  the  fragileness  and 
contextual  nature  of  the  boundaries  we  have  defined. 

Behaviorism 

Because  of  its  objectivist  roots,  those  basing  instruc¬ 
tion  on  the  behaviorist  theoretical  perspective  find  it 
appropriate  to  define  the  learning  that  will  occur, 
describing  specific  observable  outcomes  that  are  indic¬ 
ative  of  the  learning.  Because  of  the  role  of  knowledge 
as  a  mirror  of  the  world,  instructional  content  can  be 
preplanned,  organized,  and  programmed  with  specific 
outcomes  defined.  An  exemplar  learning  theory  that 
aligns  with  this  perspective  as  we  have  described  it  is 
Skinner’s  operant  conditioning.  In  this  theory,  learning 
is  viewed  as  conditioning  where  behavior  that  is  fol¬ 
lowed  by  a  reinforcer  will  increase  in  frequency  or 
probability.  Ways  of  arranging  the  contingencies 
between  the  desired  behavior  and  the  reinforcer  as  well 
as  schedules  for  determining  the  effects  of  reinforcers 
are  specified  (Barker,  1994;  Goetz  et  al.,  1992). 

From  this  learning  theory,  programmed  instruction 
provides  an  example  of  an  instructional  method  that 
facilitates  learning  by  utilizing  reinforcement  and 
feedback.  The  content  is  analyzed  and  preplanned 
based  on  an  objective  ontology,  providing  the  learner 
with  a  direct  map  of  what  specifically  is  to  be  learned. 
Teaching  machines  and  computer-aided  instruction, 
descendants  of  programmed  instruction,  provide  tech¬ 
nological  vehicles  to  facilitate  this  reinforcement  pro¬ 
cess.  Although  technology  has  facilitated  the  process 
of  this  instructional  method,  the  important  aspect  of 
these  methods  is  the  “arrangements  of  the  materials  so 
that  the  student  could  make  correct  responses  and 
receive  reinforcement  when  the  correct  responses  were 


made”  (Saettler,  1995,  p.  294).  Given  the  focus  of 
mapping  correct  responses  to  the  learner,  this  theory 
and  method  align  well  with  behaviorism’s  objective 
tradition.  The  behavior  of  the  learner,  that  of  providing 
correct  responses,  provides  the  unit  of  analysis  we  have 
identified. 

Cognitivism 

Cognitivism  retains  the  objectivist  ontological  look  at 
the  world,  thus  having  some  of  the  same  design  goals 
as  behaviorism.  Material  is  analyzed  and  sequenced, 
again  often  in  a  simple  to  complex  or  hierarchical  type 
of  organization  (Gagne  et  al.,  1992).  The  rationalist 
epistemology  guides  the  focus  on  the  study  of  learners’ 
knowledge  structures,  thus  providing  for  their  cogni¬ 
tive  structures  as  a  meaningful  unit  of  analysis.  With 
this  focus  on  rationalist  knowledge  building,  it  is  pos¬ 
sible  to  detach  the  learner  from  the  environment,  thus 
supporting  a  dualist  view. 

Learning  theories  within  this  perspective  focus  on 
the  organization  of  the  information  to  facilitate  its 
acquisition  by  the  learner.  In  his  discussion  of  mean¬ 
ingful  reception  learning,  Ausubel  (1977)  described  a 
cognitive  structure  in  learners  where  ideas  were  orga¬ 
nized  hierarchically  and  which  contained  inclusive 
concepts  under  which  were  subsumed  subconcepts  and 
other  information.  The  cognitive  structures  were  exist¬ 
ing,  were  organized  for  stability  and  clarity,  and  influ¬ 
enced  learning  and  retention  (Ausubel,  1963),  and 
learning  was  the  process  of  subsuming  new  meaning¬ 
ful  material  into  this  structure.  Instructionally,  content 
was  presented  in  its  final  form  (Ausubel,  1961),  and 
"the  learner  [was]  required  to  internalize  the  informa¬ 
tion  in  a  form  that  will  be  available  for  later  use” 
(Driscoll,  1994,  p.  115),  thus  aligning  with  the  objec¬ 
tivist  ontology  while  at  the  same  time  acknowledging 
the  role  of  cognitive  structure  as  needed  for  the  ratio¬ 
nalist  epistemology. 

Gagne’s  (1985)  theory  of  instruction  provides  an 
exemplar  within  the  cognitive  perspective.  In  his  Con¬ 
ditions  of  Learning,  he  described  five  types  of  learning 
capabilities:  intellectual  skills,  verbal  information, 
cognitive  strategies,  motor  skills,  and  attitudes.  Within 
these  comprehensive  categories  he  defines  the  condi¬ 
tions  by  which  learning  can  be  facilitated;  for  example, 
within  cognitive  strategies,  internal  conditions  for  the 
learner  require  an  encoding  strategy.  As  with  the 
behaviorists,  the  resulting  learning  objective  or  out¬ 
come  can  be  made  explicit  and  objective.  The  instruc¬ 
tional  process  is  designed  to  align  with  the  components 
and  stages  central  to  an  information-processing  type 
of  learning  theory,  an  early  learning  theory  in  the  cog¬ 
nitive  perspective. 
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Within  this  theoretical  perspective  encoding  and 
retrieval  strategies  align  with  the  rationalist  epistemol¬ 
ogy,  the  objective  ontological  view  found  in  the  explic¬ 
itly  defined  skills  and  knowledge  that  the  learner  will 
produce.  Although  the  learning  outcome  may  be  a 
behavior,  the  unit  of  analysis  is  the  learner’s  knowl¬ 
edge  structure  formed  through  a  mapping  of  well- 
structured  information  to  the  learner. 

Cognitive  Constructivism 

With  its  realist  roots,  cognitive  constructivism  supports 
that  the  world  exists,  with  learning  proceeding  from 
an  individual’s  uniquely  and  individually  constructed 
interpretation  of  that  world.  As  with  the  cognitivist, 
the  rationalist  epistemology  provides  for  the  impor¬ 
tance  of  the  development  of  cognitive  structures;  how¬ 
ever,  in  contrast  to  the  cognitive  view,  information  is 
not  prestructured  and  presumed  to  be  mapped  into  an 
individual’s  mind. 

Von  Glasersfeld  (1995)  identified  Piaget’s  work 
as  providing  a  learning  theory  that  is  consistent  with 
the  cognitive  constructivist  perspective  as  we  have 
described  it.  Von  Glasersfeld  (1989,  p.  125),  describ¬ 
ing  Piaget  as  the  “most  prolific  constructivist  in  our 
century,”  stated  that  Piaget’s  schemes  were  adaptable 
conceptual  structures  and  could  never  be  represen¬ 
tations  of  the  real  world,  always  being  based  on  the 
individual’s  experiential  world.  Piaget’s  scheme  pro¬ 
vided  a  means  for  individuals  to  construct  their 
world  using  assimilation,  a  process  by  which  an 
individual  fits  an  experience  into  an  existing  concep¬ 
tual  structure.  Should  the  experience  and  scheme  not 
fit  together  well,  perturbation  develops,  and  the  new 
experience  will  not  be  assimilated  into  the  existing 
structure.  Accommodation  then  occurs,  and  a  new 
scheme  is  developed.  In  Piaget’s  scheme,  both 
assimilation  and  accommodation  are  based  on  sub¬ 
jective  experiences  where  the  individual  is  construct¬ 
ing  a  personal  interpretation  of  the  world  (von  Gla¬ 
sersfeld,  1995). 

Instructionally,  sense-making  opportunities  are 
provided  for  learners  to  experience  and  thus  construct 
new  understandings.  Teachers  take  a  more  interactive, 
less  directive  role  in  the  instructional  process  of  learn¬ 
ers  (Greeno,  1998).  Discovery  methods  of  instruction 
provide  for  this  type  of  learning.  Ausubel  (1961) 
described  the  difference  between  reception  learning 
and  discovery  learning;  in  the  former,  all  the  content 
is  given  to  the  learner,  whereas  in  the  latter  the  learner 
is  to  rearrange  the  information  to  integrate  it  into  his 
or  her  own  existing  cognitive  structure  (i.e.,  discover 
it).  Bruner  (1961)  described  discovery  learning  in  his 
early  work.  He  included  in  discovery  not  just  “finding 


out  something  that  was  unknown  to  mankind,  but  ... 
[to]  include  all  forms  of  obtaining  knowledge  for  one¬ 
self  by  the  use  of  one’s  own  mind”  (Bruner,  1961,  p. 
22).  His  view  of  learning  was  that  it  was  an  “active 
process  of  imposing  organization  or  order  on  experi¬ 
ence”  (Goetz  et  al.,  1992,  p.  313).  From  this,  discovery 
learning  was  intended  to  support  the  experience  that 
allowed  learners  to  explore  new  concepts  and  develop 
new  skills.  The  realist  ontology  provided  that  what  was 
discovered  or  made  sense  of  did  exist,  and  a  corre¬ 
spondence  was  developed  between  the  world  and  the 
mind.  Of  interest  was  the  development  of  the  structures 
of  the  learner,  the  order  that  a  learner  imposed  upon 
the  experience,  thus  supporting  the  restructuring  of 
cognitive  structures  of  the  learner  as  a  unit  of  analysis. 

Sociocultural/Historicism 

The  unit  of  analysis  provides  the  most  informative 
distinction  between  cognitive  constructivism  and 
socioculturalism.  Seeking  to  understand  learning  from 
this  perspective  required  a  look  at  the  process  by  which 
it  occurred  among  individuals.  Although  other  per¬ 
spectives  also  support  interaction  and  peer  collabora¬ 
tion  as  ways  of  constructing  or  acquiring  knowledge, 
socioculturalism  requires  that  this  process  occur;  thus, 
the  knowledge  developed  is  not  with  the  individual  but 
in  the  interactions  among  individuals. 

For  a  learning  theory  within  this  perspective  we 
draw  on  the  work  of  Vygotsky.  According  to  Vygotsky 
(1978),  learning,  in  particular  good  learning,  takes 
place  in  advance  of  development  in  what  he  termed 
the  zone  of  proximal  development .  The  zone  of  prox¬ 
imal  development  is  the  distance  between  the  actual 
developmental  level  of  the  learner  and  what  he  or  she 
is  capable  of  performing  with  the  assistance  of  an  adult 
or  more  capable  peer.  Learning  was  also  not  an  indi¬ 
vidual  endeavor,  relying  on  the  interactions  in  which 
the  child  participates  (Vygotsky,  1978,  p.  90): 

[L]eaming  awakens  a  variety  of  internal  developmental 
processes  that  are  able  to  operate  only  when  the  child  is 
interacting  with  people  in  his  environment  and  in  coop¬ 
eration  with  his  peers.  These  processes  are  internalized, 
they  become  part  of  the  child’s  independent  developmen¬ 
tal  achievement. 

Knowledge  exists  in  the  interactions  between  the 
learner  and  adult,  although  the  interaction  is  internal¬ 
ized  by  the  learner,  becoming  a  new  function  for  the 
learner.  This  internalization  of  knowledge  from  inter¬ 
action  is  termed  appropriation  (Wertsch,  1998). 

An  instructional  strategy  within  this  perspective 
that  focused  on  the  zone  of  proximal  development  is 
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TABLE  7.4 

Psychological  Perspectives:  Exemplars  of  Associated  Learning  Theories  and  Instructional  Theories 
or  Methods 


Psychological  Perspective 

Behaviorism 

Cognitivism 

Cognitive  constructivism 
Sociocultural/historicism 
Situativity  theory 


Learning  Theory 

Skinner’s  operant  conditioning 

Ausubel’s  meaningful  reception  learning 

Piaget’s  scheme  theory 

Vygotsky’s  zone  of  proximal  development 

Lave  and  Wenger’s  legitimate  peripheral  participant 


Instruction  Theory  or  Method 

Programmed  instruction  and  computer-aided  instruction 

Gagne’s  conditions  of  learning 

Discovery  learning 

Reciprocal  teaching  or  scaffolding 

Anchored  instruction 


the  use  of  instructional  scaffolding  as  described  by 
Collins  et  al.  (1989).  The  idea  is  that  instructional 
supports  are  provided  that  allow  the  learner  to  develop 
increasing  competence,  at  which  time  the  extra  sup¬ 
ports  can  be  removed.  It  is  in  this  way  that  the  learner 
can  engage  in  activities  at  the  upper  limit  of  his  or  her 
zone  of  proximal  development.  This  support  allows  the 
tasks  to  remain  complex  and  motivating  yet  still  within 
the  learner’s  level  of  functioning. 

Situativity  Theory 

The  learning  theory  that  we  identify  for  the  situativ¬ 
ity  perspective  draws  on  the  work  of  Lave  and 
Wenger  (1991).  In  contrast  to  the  internalization  pro¬ 
cess,  which  was  evident  in  the  other  perspectives, 
learning,  from  this  perspective,  involves  the  whole 
person  and  the  role  of  that  person  as  he  or  she 
becomes  a  fully  participating  participant  of  a  com¬ 
munity  (Barab  et  al.,  1999).  Based  on  a  theory  of 
social  practice,  the  emphasis  is  on  “the  relational 
interdependencies  of  agent  and  world,  activity, 
meaning,  cognition,  learning,  and  knowing”  (Barab 
and  Duffy,  2000,  p.  50).  Learning  as  a  legitimate 
peripheral  participant  is  an  evolving  form  of  mem¬ 
bership  where  the  learner  also  reproduces  and  trans¬ 
forms  the  community  of  practice  of  which  they  seek 
membership.  The  unit  of  analysis  is  thus  the  entire 
system  of  which  the  learner  is  a  part,  with  knowledge 
residing  in  the  dynamic  relationship  of  the  system. 
Becoming  knowledgeably  skillful  involves  appropri¬ 
ating  the  practices  of  the  community,  emphasizing 
community-defined  practices  that  wed  individuals  to 
a  community,  instead  of  cognitive  processing. 

Instructionally,  the  situated  perspective  is  often 
grouped  with  the  sociocultural  view.  In  fact.  Garrison 
(1995)  described  situated  cognition  as  making  an 
important  contribution  to  social  constructivism.  For 
our  purposes,  we  draw  on  the  work  of  the  Cognition 
and  Technology  Group  at  Vanderbilt  ( 1993)  to  provide 
an  instructional  theory  (method)  for  this  perspective. 


focusing  on  characteristics  that  distinguish  it  from 
socioculturalism.  Anchored  instruction  situates  learn¬ 
ing  activities  in  information-rich  video  environments. 
Within  the  environment,  learners  are  presented  com¬ 
plex,  realistic  problems  that  they  solve  using  informa¬ 
tion  that  is  embedded  within  the  anchoring  story  pre¬ 
sented.  To  solve  the  problem,  learners  will  engage  in 
a  number  of  problem-solving  activities  such  as  math¬ 
ematical  problem  solving  as  presented  in  the  Jasper 
Woodbury  Problem  Solving  Series. 

As  an  example  of  ecological  realism,  the  anchor 
on  which  the  instruction  is  built  is  viewed  in  terms 
of  affordances  as  described  by  Gibson;  the  anchors 
are  designed  to  support  certain  types  of  teaching  and 
learning  activities .  That  is ,  the  learning  opportunities 
emerge  as  a  result  of  the  learner’s  problem-solving 
role  within  the  instruction;  the  anchor  is  designed  to 
set  up  a  rich  macrocontext  in  which  conceptual  tools 
can  be  used  to  address  the  anchor  problem  (CTGV, 
1993).  Although  the  Jasper  series  lacks  the  authen¬ 
ticity  of  a  problem-solving  encounter  that  exists  in 
the  real  world,  it  provides  the  opportunity  for  the 
learner  to  participate  in  a  simulated  type  of  world. 
Thus,  even  in  this  technology -based  world,  the  unit 
of  analysis  rests  not  with  the  learner  but  with  the 
learner  as  a  part  of  the  system  in  which  the  problem 
solving  takes  place. 

Implications  for  Learning  Theory  and 
Instructional  Methods:  Summary 

We  have  identified  an  exemplar  learning  theory  and 
instructional  theory  or  method  that  captures  the  char¬ 
acteristics  of  each  of  the  five  theoretical  perspectives 
outlined  above.  Within  each  perspective  there  is  ample 
room  for  debate  and  discussion  about  the  merits  of 
including  these  theories  within  a  perspective  as  well 
as  what  other  theories  and  methods  might  better  exem¬ 
plify  the  characteristics.  Table  7.4  provides  a  summary 
of  the  learning  theories  and  instructional  theories  or 
methods  used  in  our  discussion. 
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CONCLUDING  REMARKS 

Philosophical  perspectives  are  worldviews  that  define 
the  nature  of  the  world,  the  individual's  place  in  it,  and 
the  possible  relationships  to  that  world  and  its  parts. 
These  perspectives  form  the  foundation  for  any  line  of 
inquiry,  including  those  associated  with  a  particular  dis¬ 
cipline.  As  such,  it  is  essential  before  beginning  a  line 
of  inquiry  or  the  profession  of  teaching  that  individuals 
examine  their  ontological  and  epistemological  commit¬ 
ments.  In  this  article,  we  presented  five  main  categories 
of  psychological  perspectives  and  contrasted  them  in 
terms  of  epistemology,  ontology,  unit  of  analysis,  and 
whether  they  suggested  dualist  relationships. 

In  terms  of  the  domain  of  psychology,  we  have 
seen  the  emergence  of  psychological  perspectives  aris¬ 
ing  from  varying  philosophical  commitments.  It  is 
important  to  acknowledge  that  philosophies  with 
respect  to  epistemological  and  ontological  assump¬ 
tions  are  not  instructional  methods  and  that  methods 
are  not  philosophies.  It  has  been  argued,  however,  that 
instructional  strategies  and  methods  are  clearly  influ¬ 
enced  by  the  philosophical  assumptions  and  that  the¬ 
ories  of  knowing  and  learning  are  implicit  to  the 
instructional  design  (Barab  and  Duffy,  2000;  Duffy 
and  Jonassen,  1991;  Reigeluth,  1999b).  We  find  it 
inconceivable  that  a  teacher  or  instructional  designer 
would  advocate  a  particular  lesson  or  activity  without 
at  least  a  tacit  theory  of  how  students  think  and  learn. 
In  this  article,  we  have  differentiated  among  philo¬ 
sophical  perspectives  and  suggested  alignments 
between  assumptions  associated  with  various  perspec¬ 
tives  and  learning  and  instructional  theories.  Obvi¬ 
ously,  these  alignments  only  scratch  the  surface  and 
may  not  be  consistent  with  those  advanced  by  others; 
however,  we  have  found  these  alignments  useful  in  that 
they  begin  to  capture  some  of  the  relationships  and 
serve  as  beginnings  for  discussion. 

We  hope  this  chapter  pushes  educators  to  examine 
and  question  their  ontological  and  epistemological 
assumptions  and  whether  or  not  their  instructional 
practices  are  consistent  with  those  assumptions.  If 
there  is  incongruency  between  philosophical  assump¬ 
tions  and  instructional  practices,  then,  we  argue,  it  is 
important  to  ask  oneself  “Why  the  inconsistency?”  and 
"How  do  I  best  resolve  it?”  We  have  found  this  reflec¬ 
tive  practice  to  be  most  useful  in  terms  of  our  own 
teaching  and  research.  At  times,  this  has  meant  chang¬ 
ing  teaching  practices  and  at  other  times  challenging 
philosophical  assumptions  (see  Glaser  and  Strauss, 
1967,  for  a  discussion  of  grounded  theory  develop¬ 
ment).  Some  educators  have  advocated  for  eclecticism 
in  which  one  draws  from  various  philosophies  and 


learning  and  instructional  theories  depending  on  the 
task  at  hand  (Ertmer  and  Newby,  1993;  Reigeluth, 
1999b;  Sfard,  1998).  In  spite  of  the  merit  of  not  getting 
trapped  and  closed-minded,  we  have  found  it  useful  in 
moving  forward  with  a  research  and  teaching  program 
to  make  ontological  and  epistemological  commit¬ 
ments.  From  here,  we  as  researchers  and  reflective 
practitioners  can  then  empirically  examine  the  merits 
of  such  a  commitment;  that  is,  we  can  consider  how 
our  theoretical  assumptions  align  with  empirical  obser¬ 
vation  and  adjust  accordingly. 

We  have  noted  that  the  categories  and  descriptions 
forwarded  in  this  article  are  not  hard  and  fast  rules  or 
socially  negotiated  truths  agreed  upon  by  the  commu¬ 
nity;  rather,  they  are  based  on  our  understandings  and 
on  a  review  and  interpretation  of  the  literature  and  were 
written  in  the  context  of  this  paper.  There  is  no  way 
to  prove,  in  the  conventional  positivistic  sense,  that 
one  theory  is  better  than  another.  This  is  partly  because 
the  methods  one  would  use  to  make  such  arguments 
are  based  on  a  set  of  ontological  and  epistemological 
commitments  as  well  (Greeno,  1997).  As  such,  we 
expect  theoretical  debates  to  remain  an  important  prac¬ 
tice  in  the  field.  We  have  aligned  our  categories  with 
colleagues  in  the  field  and,  at  the  very  least,  argue  that 
much  of  our  conjectures  are  consistent  with  some  of 
the  socially  negotiated  meanings  and  inconsistent  with 
others.  But,  again,  our  goal  is  for  this  chapter  to  pro¬ 
vide  a  springboard  for  discussion  and  to  prompt  edu¬ 
cators  to  question  the  relationships  among  philosoph¬ 
ical  assumptions,  psychological  perspectives,  learning 
and  instructional  theories,  and  instructional  practice, 
not  to  suggest  indisputable  facts. 
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Part  II 

Strategies 


This  Strategies  part  of  the  Handbook  was  led  by  M. 
David  Merrill.  The  various  chapters  in  Strategies 
cover  both  instructional  and  learning  strategies, 
although  the  emphasis  is  on  implications  for  design 
and  development.  These  various  strategies  can  be 
linked  with  subsequent  chapters  in  Parts  III,  IV,  and 
V  of  the  Handbook ,  in  accordance  with  our  organi¬ 
zational  framework.  This  part  of  the  Handbook  con¬ 
sists  of  seven  chapters  covering:  (1)  representation 


strategies,  (2)  experiential  strategies,  (3)  generative 
strategies,  (4)  practice  and  feedback  strategies,  (5) 
support  strategies,  (6)  collaboration  strategies,  and 
(7)  strategies  for  different  outcomes.  This  part  of  the 
Handbook  provides  readers  with  a  comprehensive 
review  of  recent  research  pertaining  to  a  variety  of 
instructional  strategies  that  can  be  linked  with  Tech¬ 
nologies  in  Part  III,  with  Models  in  Part  IV,  and  with 
Design  and  Development  in  Part  V. 
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ABSTRACT 

Throughout  the  history  of  education  as  a  social  science, 
empirical  research  has  been  conducted  on  various  rep¬ 
resentation  modes  of  knowledge  and  their  impact  on 
learners.  Although  a  number  of  nonobjectivist  philos¬ 
ophies  have  been  put  forth  in  education,  research  in 


knowledge  representations  continues  to  be  dominated 
by  a  paradigm  of  knowledge  transfer.  This  chapter  uses 
a  three-tier  toolbox  metaphor  for  media-based  instruc¬ 
tional  design.  The  top  tier  addresses  learning  theories, 
the  middle  tier  addresses  multimedia  models,  and  the 
bottom  tier  addresses  specific  text,  visuals,  audio,  and 
animation/video  design  guidelines. 
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KEYWORDS 

Baddeley’s  memory  model:  An  information-processing 
model  that  emphasizes  the  different  short-term 
memory  stores  for  visual  (the  sketchpad)  and  audi¬ 
tory  (the  phonological  loop)  information. 

Cognitive  load  theory:  Cognitive  model  of  information 
processing  that  emphasizes  a  conceptual  mental 
workload  in  understanding  human  thought. 

Cognitive  models:  Descriptions  of  human  thought  pro¬ 
cesses  via  metaphorical  constructs.  These  constructs 
may  or  may  not  represent  actual  biological  struc¬ 
tures.  The  value  of  any  model  is  judged  by  its  utility 
in  representing  or  predicting  actual  thought,  not  by 
the  degree  of  accuracy  in  depicting  brain  structure. 

Construction-integration  model:  Cognitive  model  for 
understanding  the  processing  of  text;  this  model 
suggests  a  continual  multi-leveled  process  of  build¬ 
ing  and  confirming  a  cohesive  mental  model  from 
a  text  document  and  a  reader’s  prior  knowledge. 

Dual-coding  theory:  Cognitive  model  of  information 
processing  that  emphasizes  the  unique  contribu¬ 
tions  of  verbal  and  visual  subsystems  in  under¬ 
standing  human  cognition. 

Extraneous  load:  In  cognitive  load  theory,  the  work¬ 
load  component  associated  with  information  that 
is  not  directly  relevant  to  a  particular  content  area. 

Germane  load:  In  cognitive  load  theory,  the  workload 
component  associated  with  strategies  that  require 
processing  but  in  doing  so  make  the  relevant  con¬ 
tent  more  accessible. 

Interactive  multimedia:  The  use  of  more  than  one  form 
of  media  (such  as  text,  visuals,  video,  animation,  and 
audio)  in  a  way  in  which  a  user  has  a  great  deal  of 
control  over  the  choice  or  progress  of  the  program. 

Intrinsic  load:  In  cognitive  load  theory,  the  workload 
component  associated  with  a  particular  content 
area  and  its  level  of  complexity. 

Long-term  memory:  Component  of  the  information-pro¬ 
cessing  model  of  cognition  that  represents  informa¬ 
tion  stored,  presumably  for  the  life  of  an  individual. 

Phonological  loop:  In  Baddeley’s  memory  model  a 
short-term  memory  component  devoted  to  retain¬ 
ing  auditory  information. 

Redundancy:  The  presentation  of  information  multiple 
times  either  in  the  same  or  in  different  forms;  the 
value  of  redundancy  in  communication  is  context 
dependent  and  debated. 

Sensory  memory.  Component  of  the  information-pro¬ 
cessing  model  of  cognition  that  describes  the  initial 
input  of  information  (such  as  vision  or  hearing). 

Short-term  memory:  Component  of  the  information¬ 
processing  model  of  cognition  that  describes  a  per¬ 
son’s  attention. 


Visual  sketchpad:  In  Baddeley’s  memory  model  a 
short-term  memory  component  devoted  to  visual 
and  spatial  information. 

Visuals:  A  form  of  media  in  which  information  is  pre¬ 
sented  visually;  text  may  or  may  not  be  considered 
a  visual  form. 

Zone  of  learnability :  In  the  construction-integration 
model  of  text  processing,  a  hypothesized  optimal 
overlap  between  a  text  document  and  a  reader’s 
prior  knowledge. 


INTRODUCTION 

This  chapter  presents  a  number  of  research-based  guide¬ 
lines  for  using  media  to  support  learning.  Advances  in 
technology  and  easy  access  to  images,  sound,  video, 
and  animation,  make  the  development  of  information- 
rich  learning  environments  within  the  reach  of  most 
instructional  designers  today.  Although  these  media  are 
increasingly  sophisticated  and  varied,  access  does  not 
necessarily  turn  the  average  designer  into  an  expert  one. 
Efforts  to  enhance  instruction  are  equally  capable  of  the 
opposite  effect  when  media  are  poorly  implemented. 

Increasingly  experimental,  modality-specific,  and 
cognitively  focused  research  provides  new  ways  to 
measure  and  map  brain  activity  in  the  context  of 
media-facilitated  instruction.  Today’s  instructional 
designer  is  equipped  with  a  growing  body  of  skills  and 
knowledge,  a  metaphorical  toolbox  overflowing  with 
information.  Knowing  how  to  use  this  information  is 
increasingly  challenging.  For  one,  the  toolbox  is 
messy,  as  is  the  process  of  design  (Meikle,  2005)  and 
models  used  by  instructional  designers  (Bichelmeyer, 
2005).  Unclear  guidelines  and  the  nature  of  academic 
rhetoric  contribute  to  the  confusion. 

The  purpose  of  this  chapter  is  to  break  from  con¬ 
vention  to  untangle  some  of  the  confusion.  Recent 
developments  in  cognitive  load  theory  (Merrienboer 
and  Sweller,  2005)  restate  the  importance  of  decreas¬ 
ing  content  complexity  to  increase  meaningful  learn¬ 
ing.  In  keeping  with  this  goal,  we  share  here  a  simpli¬ 
fied  view  of  media  research,  returning  to  a  number  of 
selected  theories  and  guidelines  considered  worthy  of 
attention.  Given  their  observed  application  in  a  number 
of  settings  they  possess  an  arguable  degree  of  merit. 


AN  ORGANIZATION  SCHEME 
FOR  DESIGN  PRINCIPLES 

Figure  8.1  uses  a  toolbox  analogy  to  illustrate  a  sug¬ 
gested  framework  for  organizing  media-based  design 
principles.  A  three-tier  approach  consists  of  general 
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Learning  Theories 
Multimedia  Models 
Mode-Specific  Guidelines 


Figure  8.1  Instructional  design  toolbox. 

learning  theories  at  the  top  tier,  multimedia-learning 
theories  at  the  middle  tier,  and  specific  mode-based 
guidelines  for  text,  visuals,  audio,  and  animations/ 
video  at  the  lowest  and  most  easily  accessed  tier. 

Four  Related  Learning  Theories 

Theories  of  learning  contribute  to  our  understanding 
of  how  media  might  be  presented  for  effective  learning 
and  performance,  particularly  those  theories  that  are 
based  in  the  cognitive  sciences,  the  study  of  how 
knowledge  is  acquired.  Four  related  learning  theories 
important  to  designers  and  researcher  are  summarized: 
(1)  information-processing  theory,  (2)  dual-coding  the¬ 
ory,  (3)  cognitive  load  theory,  and  (4)  Baddeley’s 
model  of  memory  (see  Figure  8.2).  Information-pro¬ 
cessing  theory  provides  an  overview  and  perspective 
of  memory  structure  important  to  the  understanding 
the  significance  of  dual-coding  theory,  cognitive  load 
theory,  and  Baddeley’s  model  of  memory. 


Information-Processing  Theory 

A  number  of  theories  describe  the  transfer  of  infor¬ 
mation  through  memory  (Atkinson  and  Shiffrin, 
1968;  Broadbent,  1984;  Lockhart  and  Craik,  1994; 
Norman  and  Bobrow,  1975;  Waugh  and  Norman, 
1965).  Atkinson  and  Shiffrin  (1968)  proposed  a 
model  based  on  two  types  of  memory;  short-term 
memory  (including  sensory  and  working  memory) 
and  long-term  memory.  This  dual-store  model  of 
memory  is  commonly  referred  to  as  information¬ 
processing  theory.  In  this  model,  short-term  mem¬ 
ory  is  very  limited  in  duration  (only  seconds)  and 
capacity.  A  component  of  short-term  memory  is 
working  memory,  a  system  that  performs  an  exec¬ 
utive  capacity  by  managing  and  manipulating  infor¬ 
mation  that  constitutes  a  learner’s  current  attention. 
Long-term  memory  has  a  seemingly  infinite  dura¬ 
tion  and  capacity. 

The  interaction  of  short-term  and  long-term  com¬ 
ponents  is  the  focus  of  learning.  Although  virtually 
anyone  can  attend  to  a  particular  stimulus,  learning 
is  dependant  on  the  transfer  of  relevant  information 
to  long-term  memory  and  its  retrieval  when  perfor¬ 
mance  is  required.  Instructional  designers  are  funda¬ 
mentally  concerned  with  the  function  of  working 
memory.  Regardless  of  the  medium,  relevant  infor¬ 
mation  must  gain  the  attention  of  the  learner,  be  held 
in  working  memory,  and  ideally  be  in  a  form  that  is 
readily  incorporated  into  long-term  memory. 


Learning  Theories 
Multimedia  Models 
Mode-Specific  Guidelines 


Cognitive 


Figure  8.2  Four  related  learning  theories  important  to  instructional  design. 
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Dual-Coding  Theory 

Pavio’s  research  (1971,  1986)  extended  the  general 
information-processing  theory  by  suggesting  separate 
verbal  and  visual  subsystems  of  memory.  Information 
presented  in  either  form,  verbal  or  visual,  is  coded  in 
either  a  visual  memory  store  or  a  verbal  memory  store. 
These  separate  memory  systems  are  each  capable  of 
activating  the  other,  as  well  as  converting  information 
from  one  form  to  another.  Words  can  be  coded  in  a 
verbal  format  but  are  also  capable  of  being  converted 
to  an  image  format,  if  prior  knowledge  allows.  The 
same  can  be  said  for  images  whose  form  can  be  con¬ 
verted  to  a  verbal  description.  From  Paivio’s  point  of 
view,  the  connection  between  the  visual  and  verbal 
codes  strengthens  memory.  To  information  designers, 
presenting  the  combination  of  visual  and  verbal  infor¬ 
mation  is  likely  to  increase  the  chances  of  recognition 
and  recall,  due  to  the  strengthened  associations 
afforded  by  dual  coding. 

Cognitive  Load  Theory 

Cognitive  load  theory  (CLT)  relies  on  the  core  com¬ 
ponents  of  the  information-processing  model  (short¬ 
term,  working,  and  long-term  memory)  but  focuses 
particularly  on  the  limitations  of  working  memory. 
From  the  perspective  of  cognitive  load  theory,  instruc¬ 
tional  materials  should  be  created  for  a  theoretical 
optimal  cognitive  load.  In  other  words,  cognitive  over¬ 
load  impairs  learning;  cognitive  underload  does  not 
generate  interest.  Cognitive  load  theory  is  particularly 
applicable  to  instructional  designers,  and  this  chapter. 
According  to  Sweller  et  al.  (1998,  p.  262),  "Limited 
working  memory  is  one  of  the  defining  aspects  of 
human  cognitive  architecture  and,  accordingly,  all 
instructional  designs  should  be  analyzed  from  a  cog¬ 
nitive  load  perspective.” 

Cognitive  load  refers  to  the  amount  of  information 
presented  and  how  well  that  amount  compares  with 
the  size  of  working  memory.  Optimal  load  varies, 
based  on  the  level  of  learner  expertise.  Novice  learners 
with  limited  prior  knowledge  are  more  likely  to  pro¬ 
cess  simple,  sequential  structures  with  limited  threats 
to  overloading  memory.  Expert  learners,  on  the  other 
hand,  are  able  to  accommodate  richer,  more  complex 
information  loads  due  to  their  developed  schemas  or 
knowledge  representations  that  make  integration  of 
new  information  possible.  Optimally,  the  designer’s 
task  is  to  structure  information  that  fits  within  existing 
schema  (i.e.,  prior  knowledge).  Individual  differences 
in  both  prior  knowledge  and  effective  working  mem¬ 
ory  size  are  particularly  troublesome  when  designing 
materials  to  be  used  by  wide  ranges  of  learners. 


The  research  on  cognitive  load  describes  three  cat¬ 
egories  of  load:  intrinsic  load,  extraneous  load,  and 
germane  load  (Paas  et  al.,  2003).  Intrinsic  load  refers 
to  the  nature  of  the  content  and  its  level  of  complexity. 
Complexity  can  be  defined  in  terms  of  element  inter¬ 
activity,  or  the  extent  to  which  a  learner  must  under¬ 
stand  instructional  content  that  overlaps  and  interacts 
with  other  instructional  content.  High  content  interac¬ 
tivity  describes  complex  relationships  in  which  the 
various  components  can  only  be  understood  as  part  of 
a  larger  system.  Low  content  interactivity  describes 
information  that  is  more  easily  understood  in  isolation, 
because  it  requires  an  understanding  of  fewer  ele¬ 
ments.  Learning  concepts,  for  example,  would  be  more 
likely  to  involve  high  element  interactivity  than  learn¬ 
ing  facts,  which  would  involve  low  element  interactiv¬ 
ity.  An  instructional  designer  cannot  modify  intrinsic 
load  because  it  refers  to  the  complexity  of  the  infor¬ 
mation  itself. 

Extraneous  load  can  be  thought  of  as  the  noise,  or 
superfluous  elements  of  communication,  that  act  as 
barriers  to  learning  due  to  the  increased  load  they  place 
on  memory.  For  example,  using  a  large  number  of  fonts 
in  a  section  of  text  does  not  add  to  the  content  but 
rather  adds  to  the  extraneous  load  as  the  reader 
attempts  to  assign  meaning  to  the  various  changes. 

Germane  load  can  be  thought  of  as  those  things 
that  a  designer  can  do  to  facilitate  optimal  load,  such 
as  chunking  content,  sequencing  it,  and  providing  anal¬ 
ogies  that  can  help  people  understand  new  information 
more  quickly.  A  designer  can  work  to  reduce  a  high 
intrinsic  load  by  both  reducing  extraneous  and  increas¬ 
ing  germane  load. 

Although  cognitive  load  theory  is  particularly  well 
suited  for  the  discussion  of  issues  regarding  the  cre¬ 
ation  of  instructional  materials,  it  is  not  the  sole  model 
for  understanding  mental  workload.  Terms  such  as 
exerted  mental  effort  are  often  used  in  research  studies, 
but  other  than  a  gross  comparison  of  higher  and  lower 
load  or  difficulty,  the  concept  is  not  well  defined.  A 
mental  workload  example  from  human  factors 
describes  a  model  in  which  a  limited  amount  of  atten¬ 
tion  resources  must  be  shared  by  perception,  working 
memory  functions,  metacognitive  oversight,  and  exe¬ 
cution  of  a  response  (Wickens,  1984). 

Baddeley’s  Model  of  Memory 

Baddeley’s  research  (2000)  attempted  to  further  clarify 
the  capacities  of  working  memory.  Working  memory 
is  composed  of  a  central  executive  function.  This  cen¬ 
tral  executive  function  is  involved  in  focusing  atten¬ 
tion,  switching  attention,  and  dividing  attention.  In 
Baddeley’s  model,  the  executive  function  monitors  a 
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Figure  8.3  Multimedia  guidelines. 


visual  sketchpad  (visual  and  spatial  memory),  a  pho¬ 
nological  loop  (auditory  memory),  and  an  episodic 
buffer.  This  episodic  buffer  is  the  area  that  interfaces 
with  the  visual  sketchpad  and  the  phonological  loop 
and  binds  or  integrates  this  information.  In  a  sense, 
Baddeley’s  model  attempts  to  reconcile  the  under¬ 
standing  of  short-  and  long-term  memory  from  infor¬ 
mation  processing  with  the  channels  of  different  infor¬ 
mation  streams  from  dual  coding. 

Of  interest  to  instructional  designers  is  the  idea  that 
the  episodic  buffer  is  not  considered  part  of  long-term 
memory.  It  may  be  possible  to  design  and  organize 
information  for  optimal  use  of  this  buffer.  If  the  learner 
does  not  need  to  perform  low-level  integration  and 
organization,  more  mental  effort  would  be  freed  to  per¬ 
form  other  functions  conducive  to  long-term  storage 
and  recall.  Working  memory  can  retrieve  information 
from  the  episodic  buffer,  creating  relatively  new  repre¬ 
sentations  while  at  the  same  time  also  retrieving  infor¬ 
mation  from  long-term  memory.  The  unique  juxtapo¬ 
sition  of  information  in  the  episodic  buffer  may  explain 
some  aspects  of  problem  solving  and  creativity. 


MULTIMEDIA  GUIDELINES 

Two  prominent  theories  of  multimedia  (see  Figure  8 .3) 
(Mayer,  2001,  Park  and  Hannafin,  1993)  suggest  a 
number  of  guidelines,  which  are  summarized  in  Table 
8.1.  Park  and  Hannafin  (1993)  based  their  work  on  an 
extensive  review  of  the  research  and  proposed  an  over¬ 


arching  framework  for  interactive  multimedia  design 
using  20  design  principles.  Recently,  Mayer’s  (2001) 
multimedia  theory  suggested  seven  design  principles 
that  focus  specifically  on  the  interaction  of  audio,  text, 
and  visuals  in  a  number  of  controlled  experiments. 
Table  8.1  provides  the  authors’  classification  of  com¬ 
bined  multimedia  design  principles  in  the  context  of 
cognitive  load  theory.  Principles  are  placed  in  cell  loca¬ 
tions  most  directly  related  to  their  connotation  with 
the  purpose  of  showing  potential  connections  between 
theories. 

Presentation  Guidelines  for  Text, 

Visuals,  Audio,  and  Animation/Video 

Guidelines  for  text,  visuals,  audio,  and  animation/ 
video  are  reviewed  in  this  section  (see  Figure  8.4). 

Text  Guidelines 

Textual  information  has  long  been  the  backbone  of 
formal  education.  Although  the  use  visual  and  spoken 
language  information  predates  the  development  of 
written  text  in  human  history,  the  use  of  text  as  an 
augment  to  human  memory  provided  a  great  leap  for¬ 
ward  in  educational  settings.  The  guidelines  in  this 
section  of  the  chapter  focus  on  the  work  of  Walther 
Kintsch  (1998).  Whereas  many  forms  of  mediated 
instruction  have  been  used  only  over  the  last  100  years, 
written  text  has  been  used  and  developed  for  thousands 
of  years.  During  the  1970s,  in  part  as  an  attempt  to 
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TABLE  8.1 

Cognitive  Framework  for  Multimedia 


CLT  Theory 
Components 

Increasing  germane 
load 

Reducing  intrinsic 
load 


Reducing  extraneous 
load 


Park  and  Hannafin  (1993) 

Principles  that  address  the  importance  of: 

Prior  knowledge  and  learner  familiarity  (principles  1, 2,  4,  6) 
Authentic  contexts  (principle  10) 

Adaptive,  individual  attention  and  feedback  (principles  17,18,19) 
Organization  to  complement  cognitive  processes/intemal 
organization  (principles  3,  7,  9) 

Multiple  perspectives  and  cross-referencing  (principle  11) 
Organizing  activities  (invested  mental  effort,  elaboration, 
articulation,  knowledge  differentiation  (principles  5,8,  12, 14) 
Clear  system  features,  procedures,  access,  and  visual 
representations  (principles  15,  16,  20) 

Structure  and  feedback  relevancy  (principle  13) 


Mayer  (2001) 

Principles  that  address  the  importance  of: 

Visuals  for  low  knowledge  and  high-spatial  learners 
(principle  7) 

Using  words  and  pictures  rather  than  words  alone 
(principle  1) 

Placing  words  and  visuals  close  together  (principle  2) 

Simultaneously  presenting  words  and  pictures 
(principle  3) 

Excluding  extraneous  words,  pictures  and  sounds 
(principle  4) 

Using  animation  and  narration  rather  than  animation 
and  on-screen  text  or  animation, 
on-screen  text,  and  narration  (principles  5,  6) 


Text 

Visuals 


Figure  8.4  Mode-specific  guidelines. 


represent  knowledge  with  computers,  two  key  ideas  for 
further  understanding  of  text  processing  were  explored: 
proposition  representation  and  schema  theory. 

From  the  perspective  of  text  processing,  a  propo¬ 
sition  is  a  predicate  and  a  number  of  arguments  that 
together  form  an  idea  or  unit  of  information.  For  those 
doing  research  with  text,  it  was  important  to  distin¬ 
guish  between  an  idea  and  the  multiple  ways  of 
expressing  that  idea  into  English  or  any  other  lan¬ 
guage.  Decoding  natural  language  into  propositions, 
or  a  propositional  network  (complex  ideas  required 


that  propositions  could  refer  to  other  propositions  in  a 
tree-like  structure  of  meaning),  provided  a  standard¬ 
ized  way  of  referring  to  meaning  that  attempted  to 
minimize  variations  in  language,  word  choice,  or  other 
subtleties  of  expression. 

Whereas  propositions  focused  on  lower-level 
understanding  that  would  be  built  up,  schema  theory 
provided  a  top-down  view  of  understanding.  Although 
propositions  could  explain  understanding  in  terms  of 
basic  units  of  understanding,  it  did  very  little  to  explain 
the  advantages  of  prior  knowledge.  Rarely  are  we 
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forced  to  interpret  new  information  or  situations  that 
are  so  foreign  that  previous  knowledge  does  not  play 
a  large  role.  Proponents  of  schema  theory  have  pointed 
out  that  a  very  large  proportion  of  situations  we 
encounter  are  actually  routine  in  nature  .  These  routines 
can  be  expressed  as  schemas,  which  are  scripts  or 
templates  for  understanding.  Although  the  details  and 
exceptions  can  be  numerous,  the  concept  of  schema 
provided  an  essential  top-down  unifying  structure  and 
reflected  the  pattern-seeking  behavior  of  individuals. 
When  presented  with  novel  situations,  it  is  not  uncom¬ 
mon  for  learners  to  attempt  to  make  the  information 
fit  a  preexisting  schema. 

Propositional  representation  and  schema  theory 
have  provoked  numerous  studies  and  research  in  the 
areas  of  linguistics,  computer  science,  and  cognitive 
psychology.  Over  a  period  of  years,  Kintsch  devel¬ 
oped  a  model  of  text  comprehension  that  built  on  this 
pioneering  work.  In  1998,  he  summarized  his  con¬ 
struction-integration  model  in  a  text  appropriately 
called  Comprehension :  A  Paradigm  for  Cognition 
(Kintsch,  1988). 

The  construction-integration  model  proposes  two 
phases  to  understanding  text.  First,  during  the  construc¬ 
tion  phase,  readers  create  an  approximate,  but  incoher¬ 
ent,  mental  model  from  textual  input  and  their  own 
goals  and  prior  knowledge.  The  second  (integration) 
phase  involves  consolidating  local  constructions  into  a 
cohesive  meaning  and  discarding  those  local  construc¬ 
tions  that  do  not  fit.  These  phases  occur  at  the  word 
level,  at  the  sentence  level,  and  with  larger  portions  of 
text.  They  are  performed  automatically  with  familiar 
material  and  under  learner  control  by  active  readers. 
The  construction-integration  model  takes  into  account 
both  bottom- up  and  top-down  processing.  It  also  pro¬ 
vides  a  way  to  understand  very  different  types  of  tex¬ 
tual  comprehension  (e.g.,  metaphors,  humor,  abstrac¬ 
tions)  with  simple,  robust  processes.  Kintsch  made  an 
important  distinction  between  the  textbase,  which  is  a 
representation  of  the  information  contained  in  the  text, 
and  the  situational  model,  which  is  the  information  the 
reader  takes  from  the  text,  including  elaborations  and 
connections  based  on  prior  knowledge. 

Although  Kintsch’s  construction-integration  model 
has  many  implications  for  both  researchers  and  instruc¬ 
tional  designers,  those  that  are  particularly  relevant  to 
this  chapter  are  provided  in  Table  8.2.  These  are  con¬ 
sistent  with  generally  accepted  guidelines  for  writing 
texts  simply  because  the  model  was  constructed  to  sub¬ 
sume  findings  from  previous  research,  and  the  model 
suggests  that  empirical  data  provide  the  foundation  for 
creating  good  instructional  texts  for  readers. 

One  aspect  of  the  model  that  is  particularly  impor¬ 
tant  to  point  out  is  Kintsch’s  zone  of  learnability .  This 


is  an  intentional  analog  to  Vygotsky’s  zone  of  proximal 
development  (1978).  Kintsch  described  a  good  text  as 
one  that  provides  an  appropriate  amount  of  overlap 
between  the  new  information  in  the  text  and  the 
learner’s  situational  model  (i.e.,  relevant  prior  knowl¬ 
edge).  An  obvious  implication  is  that  if  there  is  too 
much  overlap  the  reader  can  gain  no  new  information 
from  the  text.  The  less  obvious  implication  is  that  the 
presence  of  familiar  information  in  terms  or  words, 
structure,  and  other  content  is  a  necessary  scaffold  to 
promote  deeper  understanding.  A  minimal  text,  in 
which  information  is  broken  down  and  context  is 
removed,  may  be  as  ineffective  as  an  overly  complex 
text.  Additional  and  redundant  text  may  be  necessary 
to  support  the  construction  of  deeper  meaning. 

As  with  earlier  theoretical  work  with  text,  the  con¬ 
struction-integration  model  has  spawned  further 
research  that  may  extend  the  model  or  suggest  alterna¬ 
tives.  It  has  also  led  to  a  particular  method  of  study— 
latent  semantic  analysis— which  provides  additional 
computational  techniques  for  understanding  text. 

Visual! Graphic  Guidelines 

The  instructional  benefit  of  visuals  has  a  long  history 
in  the  research  (Tversky  et  al.,  2002)  and  perhaps  is 
best  summarized  in  Mayer’s  (2001 ,  p.  184)  multimedia 
principle:  “Students  learn  better  from  words  and  pic¬ 
tures  than  from  words  alone.”  The  terms  visual , 
graphic,  and  image  are  used  interchangeably  in  the 
literature.  A  visual  is  typically  thought  of  as  a  form  of 
communication  that  is  not  verbal.  Braden  (1996)  iden¬ 
tified  five  categories  of  visuals  that  have  been  studied 
by  educational  researchers: 

•  Semiotics  and  film/video  conventions 

•  Signs,  symbols,  and  icons 

•  Images  and  illustrations 

•  Multi-images 

•  Graphic  representation,  including  text  as 
visuals 

Saunders  (1994)  defined  graphics  as  a  prepared  form 
of  visual  communication.  Graphics  can  be: 

•  Symbols  (pictographic  or  abstract) 

•  Maps 

•  Graphs 

•  Diagrams 

•  Illustrations  or  rendered  pictures  (realistic  to 
abstract) 

•  Models 

•  Composite  graphics  (multi-images) 

•  Photographs  (still  or  moving) 
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TABLE  8.2 

Cognitive  Framework  for  Text 

Characteristics 

Good  readers  Good  readers  are  good  word  decoders. 


Good  readers  have  language  skills  (specifically,  the 
ability  to  derive  meaning  from  lexical  structures). 


Good  readers  have  domain  knowledge  (i.e.,  reading  is 
more  easily  peformed  in  familiar  domains). 


Good  texts  Good  texts  use  words  consistently  with  little  ambiguity. 


Good  texts  use  explicit  structures  that  support  meaning. 


The  explicitness  of  the  text  is  appropriate  for  the 
intended  audience. 


Matching  readers  The  text  and  the  situational  model  of  the  reader  must 
to  texts  overlap,  but  not  too  much  (zone  of  leamability). 


Recommendation 

The  use  of  familiar  words  frees  more  processing  for  higher  level 
functioning. 

Good  decoding  leads  to  increased  vocabulary,  so  new 
vocabulary  should  be  systematically  added  in  familiar  contexts. 

Poor  decoders  can  take  more  time  or  use  higher  order  processing 
to  compensate. 

Use  of  familiar  text  structures  aids  in  the  understanding  of  text 
material. 

Good  language  skills  allow  readers  to  make  inferences  and 
construct  greater  meaning  from  the  text. 

A  deficit  in  language  skills  can  be  overcome  by  making 
inferences  explicit  or  prompting  readers  to  make  them. 

Creating  texts  around  familiar  domains  reduces  the  limitations 
of  poor  decoding  and  language  skills. 

Unfamiliar  domains  require  a  greater  emphasis  on  decoding  and 
language  skills. 

Because  of  their  extensive  domain  knowledge,  experts  are  able 
to  derive  meaning  from  even  poorly  written  texts  and  therefore 
may  be  poor  judges  of  the  value  of  those  texts  to  nonexperts. 

Explicitly  define  words  with  special  connotations  within  the 
text. 

When  choosing  words,  select  those  that  are  most  familiar  and 
less  ambiguous. 

Use  the  appropriate  domain  when  selecting  words. 

Structural  cues  should  be  explicit. 

The  organization  of  the  text  is  as  important  to  the  meaning  as 
the  words  themselves. 

Redundancy  of  content  provides  an  important  function  of 
reducing  ambiguity,  particularly  for  longer  texts. 

When  a  complete,  explicit  text  is  used,  less  prior  knowledge  is 
required  by  the  reader. 

When  readers  have  extensive  prior  knowledge,  prompting  higher 
level  processing,  less  explicit  texts  can  be  used. 

When  there  is  no  overlap  between  the  text  and  the  prior 
knowledge  of  the  reader,  the  text  will  be  too  difficult  to 
comprehend  (all  processing  will  be  at  the  lowest  literal  level). 

When  there  is  too  much  overlap  between  the  text  and  the  prior 
knowledge  of  the  reader,  the  text  has  no  educational  value  for 
that  reader. 

To  determine  the  appropriateness  of  texts  for  groups  of  like 
readers,  empirical  data  provide  the  most  effective  method. 


Source:  Adapted  from  Kintsch,  W.,  Comprehension:  A  Paradigm  for  Cognition ,  Cambridge  University  Press,  Cambridge,  U.K.,  1998. 


Guidelines  for  the  design  of  visuals  based  on  cognitive 
theory  are  presented  in  Table  8.3  (Lohr,  2007).  These 
guidelines  are  based  on  Mayer’s  learning  principles  of 
(1)  selection,  (2)  organization,  and  (3)  integration  and 
are  directly  related  to  the  three  parts  of  learner  mem¬ 
ory:  short-term  memory,  working  memory,  and  long- 
term  memory. 


Auditory  Guidelines 

Educationally,  auditory  information  is  generally 
thought  of  in  the  form  of  spoken  language.  Although 
semantically  similar,  the  information  found  in  spoken 
vs.  written  language  is  processed  differently.  Auditory 
information  also  describes  nonsemantic  information. 
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TABLE  8.3 

Guidelines  for  the  Design  of  Instructional  Visuals 


Learning 

Principle  Description 

Selection  The  selection  process  addresses  the 

mind’s  tendency  to  organize 
information  into  figure  and  ground 
categories. 


Organization  Organization  is  based  on  the  mind’s 
tendency  to  process  and  remember 
chunks  of  information  that  in  turn 
are  arranged  hierarchically. 


Integration  The  integration  principle  is  based  on 
the  Gestalt  theory  that  that  the  whole 
is  greater  than  the  sum  of  its  parts. 
A  person’s  prior  experience  allows 
him  or  her  to  see  that  whole,  even 
when  presented  with  only  a  part. 


Guidelines  for  Text,  Images,  and  Data  Displays 

Make  figure  and  ground  distinctions  as  clear  as  possible  to  reduce  the  amount  of  information 
that  memory  needs  to  process. 

Contrast  type,  shape,  color,  and  size  in  graphic  and  data  displays  to  separate  important  from 
less  important  information. 

(Principle  is  related  to  Park  and  Hannafin’s  principles  3,7,  and  9  to  structure  information 
in  ways  that  complement  the  cognitive  process  and  internal  representations. 

Shape  information  structures  to  show  subordinate,  superordinate,  and  coordinate 
relationships,  including  those  related  to  time  and  direction.  Outlines,  arrows,  and  lists  are 
commonly  used  to  establish  hierarchy  in  a  visual. 

Align  type  and  shapes  to  indicate  relationships.  Vertical  and  horizontal  alignment  establishes 
relationships  based  on  cultural  interpretation  and  reading  order.  Western  cultures  assign 
more  importance  to  items  on  the  top  and  left  because  reading  takes  place  in  a  top-down, 
left-right  sequence. 

Repeat  color,  size,  and  type  for  elements  with  equal  status. 

Place  similar  items  in  closer  proximity  than  dissimilar  items. 

Use  grid  structures  to  organize  and  integrate  information. 

Group  items  into  meaningful  units.  Greater  understanding  is  achieved  when  parts,  or 
elements  of  a  message,  are  attended  to  as  a  whole,  not  selectively. 

Mayer’s  principle  7  focuses  on  the  importance  of  presenting  information  in  ways  that  require 
less  cognitive  effort  from  the  learner— in  other  words,  reduce  the  cognitive  load  yet 
maintain  a  focus  on  the  parts-to- whole  relationships. 

Repeat  and  align  text,  shape,  color,  and  size  to  connect  items. 

Place  similar  items  in  closer  proximity  than  dissimilar  items. 


such  as  alarms  and  sound  effects,  and  highly  special¬ 
ized  information  in  the  form  of  music.  In  the  case  of 
music,  some  skills  such  as  recognizing  relative  pitch 
may  be  taught;  however,  others,  such  as  the  recognition 
of  absolute  pitch  (requiring  a  stable  long-term  memory 
of  a  reference  pitch),  may  involve  specialized  brain 
mechanisms  and  require  acquisition  relatively  early  in 
life  (Levitin  and  Rogers,  2005). 


Bishop  and  Cates  (2001)  provided  a  framework  for 
thinking  of  instructional  sound  from  a  cognitive  per¬ 
spective  (see  Table  8.4).  Although  their  primary  goal 
was  investigating  the  potential  of  sound  in  support  of 
instructional  software,  the  framework  also  is  useful  for 
considering  the  medium  of  sound  in  isolation.  The 
three  types  of  noise  come  from  the  work  of  Shannon 
and  Weaver,  which  has  been  interpreted  by  Bishop  and 


TABLE  8.4 

Cognitive  Framework  for  Instructional  Sound 


To  overcome: 

The  message 
should  contain: 

Selection 

Analysis 

Synthesis 


Acquisition  noise 
Level  A:  Technical  problems. 
Competing  internal  and  external  stimuli 
cause  message-transmission  problems. 

Content  redundancy  to  amplify  the 
content  for  message  transmission 
(encourages  noise-defeating  learner 
acquisition  states) 

1.  Use  sound  to  help  learners  direct 
attention. 

4.  Use  sound  to  help  learners  focus 
attention. 

7.  Use  sound  to  help  learners  hold 
attention. 


Processing  noise 

Level  B:  Semantic  problems.  Missing 
interpretive  frameworks  cause 
message-interpretation  problems. 

Context  redundancy  to  supply  the 
context  for  message  interpretation 
(encourages  noise-defeating  learning 
processing  strategies) 

2.  Use  sound  to  help  learners  isolate 
information. 

5.  Use  sound  to  help  learners  organize 
information. 

8.  Use  sound  to  help  learners  elaborate 
on  information. 


Source:  Adapted  from  Bishop,  M.  J.  and  Cates,  W.M.,  Educ.  Technol.  Res.  Dev.,  49(3),  5-22,  2001. 


Retrieval  noise 

Level  C:  Effectiveness  problems. 
Prompt/schema  mismatches  cause 
message-understanding  problems. 

Construct  redundancy  to  cue  appropriate 
constructs  for  message  understanding 
(encourages  noise-defeating  learner 
retrieval  schemes) 

3.  Use  sound  to  help  learners  tie  into 
previous  knowledge. 

6.  Use  sound  to  help  learners  build  on 
existing  knowledge. 

9.  Use  sound  to  help  learners  prepare 
knowledge  for  later  use. 
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Cates  (2001)  from  a  cognitive  instructional  viewpoint. 
Acquisition  noise  interferes  with  the  reception  of  an 
instructional  message.  Processing  noise  refers  to  prob¬ 
lems  with  understanding  the  message  that  has  been 
received.  Retrieval  noise  is  a  mismatch  between  an 
understood  message  and  prior  knowledge,  experience, 
or  attitudes. 

Redundancy  in  auditory  information  is  important 
to  understand  in  the  context  of  the  temporally  based 
nature  of  the  medium.  Baddeley’s  phonological  loop 
(2001)  is  the  short-term  memory  related  to  auditory 
information.  Although  his  focus  was  on  the  processing 
of  spoken  language,  one  can  consider  the  short-term 
memory  of  other  forms  of  sound  (such  as  music  or 
sound  effects)  as  being  under  the  same  limitations. 
Neisser  (1967)  referred  to  this  as  echoic  memory.  Upon 
hearing  a  sound,  a  person  is  able  to  replay  a  brief 
portion  of  it  in  this  short-term  memory  for  further 
processing.  This  partially  explains  the  “cocktail  party 
effect,”  in  which  a  person  is  engaged  in  a  conversation 
to  the  exclusion  of  numerous  others  talking  in  the  same 
room.  Short-term  audio  memory  allows  that  person,  in 
a  limited  sense,  to  monitor  and  switch  attention  to  a 
different  conversation  upon  hearing  his  or  her  name  or 
some  other  attention-grabbing  piece  of  information. 

Bishop  and  Cates  (2001)  (see  Table  8.4)  suggested 
the  use  of  redundancy  to  overcome  the  temporal  nature 
of  audio  communication.  Content  redundancy  refers 
to  the  retransmission  or  amplification  of  the  audio 
information.  Context  redundancy  involves  presenting 
semantic  information  in  multiple  ways  to  ensure  that 
it  is  perceived  correctly.  Finally,  construct  redundancy 
attempts  to  emphasize  the  link  between  the  informa¬ 
tion  and  prior  knowledge. 

Animation/Video  Guidelines 

Animation  and  video  are  described  together  because 
they  share  similar  characteristics  (e.g.,  a  time  compo¬ 
nent,  presentation  of  motion  or  change).  In  addition, 
the  advent  of  digital  technologies  for  both  animation 
and  video  production  has  blurred  the  distinction 
between  artificial  and  real  representations.  Researchers 
who  examine  the  impact  of  animation  in  education 
must  consider  the  critiques  leveled  at  any  media  com¬ 
parison  study.  As  is  often  the  case,  animation  provides 
a  simplified  visual  view  of  a  process  or  behavior  that 
otherwise  might  be  presented  in  a  classroom  with  mod¬ 
els  or  via  video  footage;  however,  animation  may  be 
able  to  uniquely  portray  an  educational  concept  that 
might  otherwise  be  difficult  to  visualize  .  It  may  also  be 
used  for  financial  or  logistical  reasons  in  lieu  of  another 
approach;  for  example,  Adamo-Villani  and  Beni  (2004) 
described  a  system  that  uses  realistic  three-dimensional 


animation  for  the  purpose  of  teaching  finger  spelling 
via  sign  language.  Research  focused  on  the  use  of  ani¬ 
mation  most  often  describes  its  use  with  other  forms 
of  presentation.  In  general,  guidelines  for  its  effective¬ 
ness  can  be  categorized  using  a  cognitive  framework 
similar  to  the  one  used  for  multimedia.  Table  8.5  cat¬ 
egorizes  findings  from  a  number  of  studies  within  the 
context  of  the  components  of  cognitive  load  theory. 

THE  NEED  FOR 
CONTINUED  RESEARCH 

Although  no  single  cognitive  theory  of  learning  or  mul¬ 
timedia  model  can  explain  in  sufficient  detail  all  the 
functions  of  memory,  the  theories  and  models  presented 
in  this  chapter  have  much  to  offer  to  an  understanding 
of  instructional  media  design.  Perhaps  a  more  complex 
theory  and  model  will  be  eventually  proposed  that  sub¬ 
sumes  a  majority  of  research  in  cognitive  psychology 
and  instructional  multimedia;  however,  it  is  likely  that 
such  a  complicated  theory  and  model  would  be  more 
difficult  to  translate  into  prescriptions  for  practice. 

Although  most  of  the  guidelines  in  this  chapter 
represent  the  work  of  many  experimental  studies,  their 
selection  and  categorization  are  dependent  on  the 
authors’  interpretation  of  the  research  and  are  therefore 
subject  to  the  biases  of  their  perspectives.  A  compre¬ 
hensive  review  of  new  research  will  be  necessary  on 
a  periodic  basis  to  ensure  that  the  most  worthy  design 
guidelines  are  shared.  Some  might  consider  a  more 
rigorous  approach  to  the  assignment  of  guideline  clas¬ 
sifications  needed. 

Perhaps  most  needed  is  a  more  integrative  set  of 
guidelines  within  the  various  presentation  modes. 
Whereas  visual  and  animation/ video  media  share  an 
organizational  theme  similar  to  the  multimedia  frame¬ 
work,  the  text  and  audio  guidelines  have  unique  orga¬ 
nization  schemes,  which  makes  the  relationships 
between  the  different  models  more  difficult  to  under¬ 
stand. 

Other  topics  of  future  research  include  the  com¬ 
parison  of  research-based  guidelines  to  the  guidelines 
used  in  settings  that  are  more  applied  and  less  research 
oriented.  The  restrictions  encountered  by  the  realities 
of  production  are  worthy  of  investigation.  Although 
design  focuses  on  optimal  presentation,  dynamics 
within  the  workplace  often  curtail  the  effectiveness  of 
a  product.  As  a  case  in  point,  the  first  author’s  graphic 
design  textbook  promoted  guidelines  for  the  effective 
display  of  graphics  that  were  clearly  violated  by  the 
book  itself  due  to  the  overriding  importance  of  reduc¬ 
ing  publication  costs.  Knowing  ahead  of  time  what 
these  realities  are  would  be  beneficial  to  the  designer. 
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TABLE  8.5 

Cognitive  Framework  for  Animation/Video 


Theory 

Components 

Increasing 
germane  load 


Reducing 
intrinsic  load 


Reducing 
extraneous  load 


Description 

Hays  (1996)  demonstrated  that  low-spatial-ability  participants  made  greater  gains 
when  animations  were  used  as  opposed  to  static  graphics  or  no  graphics. 

Schnotz  and  Rasch  (2005)  described  the  value  of  animations  for  learners  with  low 
learning  prerequisites.  Learners  with  high  learning  prerequisites  received  more 
value  from  manipulating  pictures  themselves. 

Bishop  and  Cates  (2001)  described  the  technique  of  visual  description  that  provides 
access  to  television,  movies,  etc.,  for  individuals  with  visual  impairments.  They 
suggested  that  the  technique  might  also  be  useful  for  promoting  comprehension 
for  those  with  average  vision. 

Reiber  (1996)  found  that  animated  graphic  feedback  could  produce  gains  in  tacit 
knowledge  when  compared  with  text-only  feedback. 

Garcia  (1998)  described  the  benefit  of  students  creating  animations  in  their 
understanding  of  science  content. 

Catrambone  and  Seay  (2002)  found  that  animations  aided  in  students’ 
understanding  of  algorithms  when  compared  to  the  same  text  followed  by  still 
frames.  It  was  particularly  helpful  for  students  in  a  weak-text  vs.  an  improved- 
text  condition. 

Bodemer  et  al.  (2005)  demonstrated  how  exposure  to  static  sources  of  information 
prior  to  work  with  dynamic,  interactive  visualizations  can  support  learning. 

Koroghlanian  and  Klein  (2004)  found  that  participants  exposed  to  animations  will 
spend  more  time  in  the  instruction  than  those  given  only  static  images. 

Hegarty  et  al.  (2003)  found  no  difference  between  static  and  dynamic  diagrams  in 
the  understanding  of  dynamic  processes;  however,  they  suggested  that 
participants’  ability  to  create  animated  mental  models  from  static  diagrams  might 
be  the  reason  for  the  lack  of  difference. 

Linebarger  (2001)  found  that  the  use  of  captions  with  video  programs  resulted  in 
greater  word  recognition  for  young  children  and  related  positive  effects.  The 
researcher  also  reported  that  captions  lessened  attention  to  visual  and  audio 
distractions  in  the  video  presentation. 

Caspi  et  al.  (2005)  found  that  video  recordings  of  lectures  did  influence  the  message 
being  delivered  to  students  and  that  mismatches  of  the  medium  and  the  message 
may  have  a  negative  result  on  cognitive  and  affective  outcomes. 


Guideline 

Design  effects  are  stronger  for 
low-knowledge  learners  than  for 
high-knowledge  learners  and  for  high 
spatial  learners  rather  than  low  spatial 
learners  (Mayer’s  principle  7). 


Employ  organizing  activities  (invested 
mental  effort,  elaboration,  articulation, 
knowledge  differentiation)  (Park  and 
Hannafin’s  principles  5,  8,  12,  and  14). 

Present  corresponding  words  and  pictures 
simultaneously  rather  than  successively 
(Mayer’s  principle  3). 


Finding  conflicts  with  Mayer’s  principle  to 
use  animation  and  narration  rather  than 
animation,  narration,  and  text  (Mayer’s 
principle  5). 

Exclude  rather  than  include  extraneous 
words,  pictures,  or  sounds  (Mayer’s 
principle  4). 


SUMMARY 

This  chapter  presents  a  three-tier  toolbox  metaphor 
for  guiding  the  design  of  a  number  of  presentation 
media.  The  designer  typically  reaches  for  the  ele¬ 
ments  on  the  third  tier,  the  most  concrete  and  acces¬ 
sible  elements  of  design:  text,  visuals,  audio,  and 
animation/video.  Understanding  first-tier  elements 
that  include  cognitively  based  theories  of  learning 
(information-processing  theory,  dual-coding  theory, 
cognitive  load  theory,  and  Baddeley’s  memory  the¬ 
ory)  in  turn  helps  the  designer  more  clearly  under¬ 
stand  second-tier  multimedia  models  (Mayer,  2001; 
Park  and  Hannafin,  1993)  and  their  relationship  to 
mode-based  guidelines.  The  relationship  between 
the  three  tiers  is  appealing,  but  more  research  is 
needed  to  articulate  a  more  fully  integrative  design 
toolbox. 
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ABSTRACT 

This  chapter  provides  a  conceptual  framework  related 
to  designing  for  situational  embodiment;  discusses 
three  types  of  curricular  designs,  ranging  from  designed 
and  emergent  simulation  models  to  participation  mod¬ 
els;  and  overviews  various  strategies  for  achieving  each 
design,  including  examples  from  the  literature.  Specif¬ 


ically,  we  discuss  designed  simulation  models  (e.g., 
anchored  instruction,  problem-based  learning,  and  cog¬ 
nitive  apprenticeship);  emergent  simulation  models 
(e.g.,  case-based  reasoning,  project-based  learning,  and 
classroom  learning  communities);  and  participation 
models  (e.g.,  participatory  simulations,  academic  play 
spaces,  and  communities  of  practice).  Looking  across 
the  different  examples,  we  also  discern  tensions  that 
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emerge  in  working  toward  curricular  embodiment— 
namely,  tensions  concerning  the  quality  of  the  context 
(noisy  vs.  tailored)  and  the  quality  of  the  formalisms 
(explicit  vs.  implicit). 


KEYWORDS 

Authenticity:  Learner-perceived  relations  between  the 
associated  practices  and  one’s  projected  or  envi¬ 
sioned  use  value  of  those  practices. 

Formalism:  The  formal  structure  and  abstract  princi¬ 
ples  that  underlie  the  conceptual  framework  of  the 
content  area;  for  example,  the  concept  of  erosion 
is  a  formalism  in  science,  division  is  a  formalism 
in  mathematics,  and  metaphor  is  a  type  of  formal¬ 
ism  in  language  arts 

Participation  models  of  authenticity :  Models  that 
establish  a  sense  of  authenticity  by  engaging  learn¬ 
ers  in  the  authentic  practices  as  they  work  on  real- 
world  tasks  as  part  of  authentic  communities  and 
in  contexts  that  value  the  outcomes  of  those  tasks. 

Simulation  models  of  authenticity:  Models  that  build 
upon  the  assumption  that  classroom  activity  should 
be  made  to  resemble  as  much  as  possible  the  activ¬ 
ities  in  which  real-world  practitioners  engage. 

Situational  embodiment:  When  to-be-leamed  content 
is  experienced  in  relation  to  a  particular  context  of 
use  that  provides  legitimacy  to  the  content  and 
student  actions  and  a  meaningful  goal  and  set  of 
actions  for  the  learner  and  on  which  learner  actions 
have  some  consequence. 


INTRODUCTION 

A  number  of  theorists  have  argued  that  the  cognitive 
revolution  has  consisted  of  two  phases.  In  the  first 
phase,  it  was  individual  thinkers  and  their  isolated 
minds  that  were  the  emphasis  (Gardner,  1985).  Clearly, 
during  this  period  theorists  attempted  to  de  emphasize 
context  (partly  in  an  attempt  to  remove  the  behaviorist 
shackles  fastened  by  Skinner  and  others)  and  focus  on 
the  individual  mind  as  disembodied  from  the  environ¬ 
mental  particulars  in  which  it  is  situated  (Fodor,  1975; 
Shannon,  1988).  The  second  phase  relocated  cognitive 
functioning  within  its  social,  cultural,  and  historical 
framework  (Barron  et  al.,  1995)  and  is  central  to  many 
so-called  situative  perspectives  (Brown  et  al.,  1989; 
Cobb  and  Yackel,  1995;  Greeno,  1989;  Kirshner  and 
Whitson,  1998;  Suchman,  1987;  Young,  1993). 

Consistent  with  ecological  notions  advanced  by 
Gibson  and  others  (Gibson,  1966,  1979/1986;  Reed, 
1991;  Reed  and  Jones,  1982;  Shaw  and  Bransford, 


1977;  Young,  1993),  learners  were  no  longer  conceived 
of  as  isolated  from  the  environment  in  which  learning 
occurs;  rather,  both  individuals  and  environments  were 
considered  functionally  coupled,  with  intention-con¬ 
tingent  practices  serving  as  the  bond  that  weds  the  two 
(Reed,  1991).  These  ecological  descriptions  challenge 
the  belief  that  knowledge  is  a  thing  to  be  acquired  and 
that  knowing  is  simply  a  cognitive  act  that  occurs  in 
the  confines  of  isolated  minds.  Instead,  from  an  eco¬ 
logical  perspective,  knowledge  refers  to  an  activity 
(not  a  thing),  is  always  contextualized  (not  abstract), 
is  reciprocally  constructed  as  part  of  the  individual- 
environment  interaction  (not  objectively  defined  or 
subjectively  created) ,  and  involves  whole  persons  (not 
disembodied  minds)  (Barab  and  Duffy,  2000).  Cogni¬ 
tions  are  not  acts  of  the  mind  or  decontextualized  bits 
of  knowledge  to  be  transmitted,  but  contextually 
embedded  practices  that  engage  individuals  who  are 
vitally  situated  within  and  across  rich  and  meaningful 
environments. 

At  its  core,  this  theorizing  suggests  an  emphasis 
of  supporting  meaningful  participation  within  experi- 
entially  rich  contexts.  Sfard  (1998)  characterized  the 
shift  as  a  move  away  from  the  acquisition  metaphor, 
which  has  guided  much  of  the  practice  in  K-12 
schools,  toward  a  participation  metaphor  in  which 
knowledge  is  considered  fundamentally  situated  in 
practice.  In  a  very  real  way,  what  is  being  argued  for 
is  the  primacy  of  experience,  but  the  arrangement  of 
schools  and  the  focus  on  supporting  abstracted  under¬ 
standings  make  the  adoption  of  a  situated  or  experien¬ 
tial  curriculum  difficult.  Schools  are  designed  to  struc¬ 
ture  not  simply  experience  in  a  general  sense  but  also 
pedagogically  useful  experiences  intended  to  support 
the  learning  of  particular  content.  In  this  chapter,  we 
first  present  a  brief  overview  of  an  embodied  curricu¬ 
lum  along  with  a  framework  for  conceptualizing  the 
challenge  of  truly  embodying  curricular  understand¬ 
ings.  This  is  then  followed  by  an  overview  of  nine 
instructional  approaches  that  work  to  situate  under¬ 
standings. 


AN  EMBODIED  CURRICULUM 

Many  educators  have  argued  that  the  didactic  format 
of  teaching  that  concentrates  on  memorization  of  fac¬ 
tual  information  promotes  the  development  of  super¬ 
ficial  conceptual  understanding  (Lave  and  Wenger, 
1991;  Roth,  1996;  Ruopp  et  al.,  1993;  Wandersee  et 
al.,  1994).  In  response  to  the  limitations  of  these  teacher- 
centered  or  lecture -based  environments,  an  increasing 
number  of  educators  are  abandoning  such  predomi¬ 
nantly  didactic  modes  of  instruction  and  moving 


98 


Strategies  for  Designing  Embodied  Curriculum 


toward  more  experiential  or  situated  models  of  instruc¬ 
tion  (Barab  and  Hay,  2001;  Barab  and  Plucker,  2002; 
Brown,  1992;  Brown  et  al.,  1994;  Collins  et  al.,  1989; 
Cognition  and  Technology  Group  at  Vanderbilt 
[CTGV],  1991, 1993;  Duffy  and  Jonassen,  1992;  Roth, 
1996).  These  participatory  learning  environments  sup¬ 
port  the  natural  complexity  of  content,  avoid  over¬ 
simplification,  engage  students  in  the  construction  of 
products  requiring  practices  that  embody  complex 
contexts,  encourage  collaboration,  and  present  instruc¬ 
tion  within  both  simulated  and  real-world  contexts 
(Barab  and  Duffy,  2000). 

Central  to  the  various  efforts  is  a  common  belief 
that,  when  the  underlying  content  to  be  taught  becomes 
situated  in  terms  of  authentic  use,  it  changes  the  essen¬ 
tial  meanings  of  that  content  for  the  learner.  At  its  core, 
what  is  being  argued  is  the  need  for  a  situationally 
embodied  curriculum ,  one  that  is  owned  by  the  learner 
and  that  evidences  the  functional  value  of,  and  need 
for,  the  target  formalisms.  Building  on  Nathan’s  (2005) 
description  of  formalisms,  this  refers  to  the  “formal 
structure  and  abstract  principles  that  underlie  the  con¬ 
ceptual  framework  of  the  content  area”  (p.  5).  Each 
discipline  entails  core  concepts,  methodologies, 
frameworks,  tasks,  and  principles  that  provide  a  central 
organizing  role  in  the  discipline  (e.g.,  symbolic  equa¬ 
tions,  graphs  and  diagrams,  scientific  laws,  rules  of 
grammar,  historic  patterns,  music  notations,  artistic 
movements).  Formalisms  are  assumed  to  bear  some 
generalizable  power  or  utility  for  illuminating  the  deep 
structure  of  otherwise  unrelated  contexts;  yet,  though 
we  agree  that  formalisms  may  be  powerful  for  knowl¬ 
edgeable  experts,  they  do  not  bear  the  same  utility  for 
those  only  beginning  to  know  a  domain.  In  the  latter 
case,  an  explicit  focus  on  formalisms  abstracted  from 
their  functional  contexts  is  not  the  most  productive 
pedagogical  move. 

Still,  building  an  embodied  curriculum  in  the  con¬ 
text  of  schools  represents  a  formidable  challenge,  and 
at  its  core  is  selecting  an  appropriate  context  and  deter¬ 
mining  the  amount  of  noise  required  for  an  underlying 
formalism  to  become  embodied.  When  a  learning  envi¬ 
ronment  is  overly  tailored,  it  begins  to  feel  more  like 
school  work  and  becomes  less  experiential  (Dewey, 
1938/1997);  the  potential  for  enmeshment  becomes 
unlikely,  and  the  knowledge  is  more  likely  to  remain 
inert  (Whitehead,  1929).  As  one  adds  more  situational 
aspects,  the  potential  for  mystery,  reality,  and  discov¬ 
ery  increases,  but  one  sacrifices  guidance,  efficiency, 
and  clarity.  In  fact,  life  itself  presents  the  ultimate 
noisy  context,  with  schools  being  developed  in  part  to 
present  formalisms  removed  from  this  noise  — a  place 
where  pure  reason  can  prevail,  where  explicit  focus  on 
formalisms  can  occur. 


As  long  as  educators  continue  to  assign  pure 
thought  and  disembodied  formalisms  a  reverent  quality 
in  terms  of  the  goals  of  education,  we  will  continue  to 
undermine  the  meanings  and  understandings  that  we 
wish  to  support.  The  goal  of  a  situationally  embodied 
design  is  to  evolve  the  learning  trajectory  in  relation 
to  the  curricular  narrative  framework,  thereby  ground¬ 
ing  content  with  respect  to  those  situations  in  which 
it  has  value.  What  makes  the  curriculum  situationally 
embodied  is  that  the  to-be-learned  content  is  experi¬ 
enced  in  relation  to  a  particular  narrative  context.  For 
us,  a  narrative  context  is  not  simply  any  surround,  but 
a  storyline  that  provides  legitimacy  to  the  content  and 
student  actions  and  a  meaningful  goal  and  set  of 
actions  for  the  learner,  and  on  which  learner  actions 
have  some  consequence.  To  the  extent  that  a  particular 
formalism  is  not  connected  to  a  related  context  of  use, 
it  runs  the  risk  of  becoming  a  disembodied  fact  to  be 
memorized,  with  no  contextual  anchor  by  which  a 
student  can  see  its  authentic  application.  It  is  the 
learner  intentions  and  use  affordances  emerging  from 
the  context  of  use  that  transform  a  formalism  from  a 
disembodied  fact  to  an  embedded  or  even  embodied 
one  (Barab  et  al.,  1999).  In  building  a  particular  con¬ 
text  of  use,  educators  must  examine  the  intentions 
likely  to  emerge  from  the  core  context  and  understand 
how  satisfying  these  emergent  intentions  both  requires 
and  validates  a  particular  formalism  as  a  useful  tool. 
Below,  we  overview  nine  instructional  approaches  that 
work  to  embody  or  situate  the  content,  the  learner,  and 
the  experience. 


DESIGNING  EMBODIED  CURRICULA 

Thus  far,  we  have  argued  that  the  meaning  of  a  set  of 
formalisms  changes  when  they  are  situated  within  a 
context  of  use.  A  critical  challenge,  therefore,  for  those 
interested  in  designing  embodied  curricula  is  estab¬ 
lishing  the  context  for  embodiment,  and  this  requires 
establishing  a  curricular  narrative  with  an  element  of 
authenticity;  however,  authenticity  as  a  term  is  widely 
used  but  poorly  defined.  In  designing  an  environment 
for  learning,  one  must  determine  what  is  meant  by 
authentic  — and  to  whom  (Barab  et  al.,  2000).  For 
example,  what  is  authentic  to  the  designer  may  not  be 
to  the  learner,  what  is  authentic  to  the  learner  may  not 
be  to  the  designer,  and  both  or  neither  may  be  consid¬ 
ered  authentic  by  real-world  practitioners  (Brown  et 
al.,  1989). 

Authenticity  lies  in  the  learner-perceived  relations 
between  the  associated  practices  and  one’s  projected 
or  envisioned  use  value  of  those  practices.  With  that 
said,  a  number  of  educators  have  worked  to  develop 
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both  theories  and  learning  environments  that  help  to 
establish  embodiment.  Below  we  review  a  number  of 
these,  first  describing  those  that  borrow  from  a  simu¬ 
lation  model  of  authenticity,  distinguishing  those  that 
are  primarily  under  the  direction  of  the  designer  vs. 
those  in  which  the  to-be-learned  content  is  more  emer¬ 
gent  though  still  not  situated  within  a  real-world  con¬ 
text.  Then,  we  overview  designed  spaces  that  leverage 
a  participatory  model,  prioritizing  the  importance  of 
the  real-world  context  in  the  service  of  real-world 
issues  (Petraglia,  1998;  Radinsky  et  al.,  1998). 

Designed  Simulation  Models 
for  Establishing  Embodiment 

Simulation  models  build  upon  the  assumption  that 
classroom  activity  should  be  made  to  resemble  as 
much  as  possible  the  activities  in  which  real-world 
practitioners  engage.  This  includes  featuring  elements 
of  factual  authenticity,  wherein  the  environmental 
details  of  the  task  are  designed  to  be  similar  to  those 
of  the  real  world;  procedural  or  process  authenticity, 
in  which  learner  practices  are  similar  to  those  that 
would  be  engaged  outside  of  schools;  and  task  authen¬ 
ticity,  in  which  the  tasks  are  similar  to  those  being 
undertaken  by  real-world  practitioners  (Petraglia, 
1998).  Barab  and  Duffy  (2000),  borrowing  from  Senge 
(1994),  referred  to  these  contexts  as  practice  fields 
because  they  do  not  take  place  in  the  real  fields  but, 
rather,  allow  learners  to  practice  the  sorts  of  activities 
that  they  will  later  encounter  outside  of  schools.  In  our 
thinking,  it  is  more  than  entailing  authentic  practice 
that  makes  these  contexts  valuable  ;  they  also  legitimize 
the  worth  and  meaning  of  the  content  being  practiced. 
Although  no  single  set  of  strategies  can  capture  all  of 
the  theories  and  designs  representing  simulation  mod¬ 
els,  we  overview  a  number  of  theories  and  designs  that 
illuminate  what  we  mean  by  practice  fields. 

Anchored  Instruction 

Anchored  instruction  refers  to  instruction  in  which  the 
material  to  be  learned  is  presented  in  the  context  of  a 
specific  topic  or  anchor  that  serves  to  legitimize  the 
material  and,  further,  allows  it  to  be  examined  from 
multiple  perspectives.  The  work  of  the  Cognition  and 
Technology  Group  at  Vanderbilt  (1990,  1991,  1993) 
has  primarily  focused  on  video-based  macrocontexts, 
intended  to  overcome  inert  knowledge  (Whitehead, 
1929)  by  anchoring  learning  — thereby  embodying  the 
to-be-learned  content— within  the  context  of  meaning¬ 
ful  problem-solving  activities.  In  contrast  to  the  dis¬ 
connected  application  problems  found  at  the  end  of 
textbook  chapters,  macrocontexts  refer  to  stories  set  in 


semantically  rich,  open-ended  environments  (CTGV, 
1993).  In  these  anchoring  macrocontexts,  students 
begin  with  a  complex  narrative  introducing  a  problem 
and  then  use  top-down  strategies  to  generate  the  sub¬ 
goals  necessary  to  reach  the  solution.  This  top-down 
processing  helps  students  learn  the  lower-level  skills 
(e.g.,  mathematical  algorithms  and  facts)  in  a  manner 
that  also  gives  them  insights  into  the  relationships 
between  the  skills  being  learned  and  the  reciprocal 
opportunities  for  using  them. 

One  illuminative  set  of  curricula  is  the  Jasper 
Woodbury  Problem  Solving  Series,  designed  for  stu¬ 
dents  in  grades  5  and  up.  Each  videodisc  contains  a 
short  (approximately  17-minute)  video  adventure  that 
ends  with  a  complex  challenge.  Like  good  detective 
novels,  the  adventures  embed  all  of  the  essential  data 
(along  with  additional,  irrelevant  data)  within  the  story. 
In  one  such  story,  students  must  discover  and  satisfy 
17  mathematical  subproblems  necessary  for  determin¬ 
ing  if  the  main  character,  Jasper,  can  reach  home 
(CTGV,  1990).  Much  of  the  challenge  in  these  episodes 
involves  distinguishing  the  noise  in  the  context  from 
those  aspects  relevant  to  the  problem  at  hand.  The  Jas¬ 
per  adventures  also  present  embedded  didactic  scenes 
that  model  particular  approaches  to  solving  problems 
and  that  can  be  revisited  on  a  just-in-time  basis  as 
students  need  them  to  solve  the  challenges.  When  stu¬ 
dents  use  these  curricula,  they  learn  mathematics  as 
they  work  to  solve  the  challenge:  they  develop  concepts 
and  skills  through  just-in-time  instruction. 

Significantly,  these  lessons  do  not  separate  the 
content  (e.g.,  the  distance-rate-time  formula)  from  the 
medium  (e.g.,  the  videodisc  story  in  which  Jasper’s 
problem  is  introduced);  rather,  they  simply  present  a 
context  (i.e.,  determining  whether  or  not  Jasper  can 
reach  home) .  Although  anchored  instruction  may  also 
resemble  problem-based  learning  (PBL;  described 
below),  it  tends  to  be  less  open  ended  than  the  latter. 
Most  anchored  modules  are  designed  for  young  learn¬ 
ers  and  so  embed  all  of  the  data  necessary  to  solve  the 
problem  within  the  modules  themselves  through  an 
embedded-data  design.  Also,  in  contrast  to  PBL,  sub¬ 
stantial  independent  research  and  data  collection  are 
not  usually  required  in  anchored  modules.  As  with 
PBL,  the  goal  is  to  capture  a  real  problem  along  with 
its  real-world  context,  but  in  anchored  instruction  there 
is  no  pretense  that  this  problem  actually  exists  for  the 
students;  instead,  learners  are  invited  to  engage  in  a 
fictitious  problem.  It  is  also  important  to  note  that, 
although  the  first  set  of  anchored  episodes  designed 
by  the  CTGV  focused  on  relatively  constrained 
designed  simulation  problems,  over  time  they  have 
developed  community-linking  scaffolds  (Barron  et  al., 
1995)  and  flexibly  adaptive  designs,  the  outcomes  of 
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which  are  increasingly  determined  by  the  students  and 
teachers  (Schwartz  et  al.,  1999). 

Problem-Based  Learning 

According  to  Finkle  and  Torp  (1995,  p.  1),  problem- 
based  learning  (PBL)  simultaneously  develops  both 
problem-solving  strategies  and  disciplinary  knowledge 
bases  and  skills  by  placing  students  in  the  active  role 
of  problem  solvers  confronted  with  an  ill-structured 
problem  that  mirrors  real-world  problems.  Although 
originally  associated  specifically  with  medical  curric¬ 
ula  and  stipulating  a  sequence  of  rules  to  be  followed 
in  addressing  the  ill- structured  problems  of  that  field 
(Barrows  and  Tamblyn,  1980),  problem-based  learning 
has  been  adopted  in  a  range  of  other  domains,  not 
strictly  as  a  methodology  but  more  broadly,  suggesting 
an  approach  to  instruction.  Although  not  situated  in 
the  real  world  per  se  (participatory  authenticity) ,  learn¬ 
ers  nonetheless  experience  simulatory  authenticity, 
which  affords  both  meaning  and  focus.  Central  to  this 
instructional  strategy  is  the  problematic  context 
(Dewey,  1938/1997)  or  cognitive  puzzlement  (Savery 
and  Duffy,  1995)  that  fosters  learning  through  engage¬ 
ment  in  meaningful  domain-related  tasks. 

In  discussing  PBL,  Savery  and  Duffy  (1995)  artic¬ 
ulated  a  series  of  instructional  principles  that  can  be 
applied  to  a  PBL  environment,  synthesized  as  follows. 
First,  the  learner  must  perceive  the  learning  activities 
as  related  to  a  broader  task  or  problem  and  must  accept 
ownership  of  the  instructional  goals  and  the  problem¬ 
solving  process.  In  addition,  though  embedded  in  a 
tailored  or  sheltered  context,  the  learning  activities 
must  nonetheless  support  and  challenge  the  learner  and 
must  demand  authentic  discourse  to  prepare  the  learner 
to  function  in  the  complexity  of  the  real-world  domain. 
Finally,  the  learner  must  test  emergent  understandings 
against  alternative  views  and  contexts  and  must  reflect 
on  both  what  is  learned  and  the  process  by  which 
learning  occurs.  This  approach  differs  from  such 
related  strategies  as  case-based  reasoning  (Kolodner, 
2006)  in  that  the  students  bear  responsibility  for  devel¬ 
oping  an  original  solution  to  the  problem  rather  than 
studying  someone  else’s  solution. 

As  a  workplace  simulation  situating  grade-school 
children  in  different  positions  at  a  fictitious  monetary 
institution,  the  Chelsea  Bank  program  is  an  example 
of  PBL  that  consists  of  over  a  dozen  computer-based 
episodes  and  engages  middle-school  students  in  a  vari¬ 
ety  of  finance-related  problems  through  the  use  of  a 
range  of  problem-solving  strategies,  including,  for 
example,  making  predictions,  modifying  solutions, 
and  seeking  help  (Duffy  et  al.,  1996).  Indeed,  although 
situated  within  the  domain  of  finance,  the  project  does 


not  serve  as  mere  vocational  training  but  instead  aims 
to  develop  skills  in  basic  literacy  and  problem  solving. 
For  example,  in  assisting  a  Chelsea  customer,  students 
make  use  of  bank  manuals,  account  information,  and 
other  resources  to  decide,  from  multiple-choice 
options,  what  action  to  take;  they  must  select  from  the 
options  and  justify  their  answers.  A  key  challenge  in 
the  successful  implementation  of  the  Chelsea  Bank 
program  and  other  PBL  simulations  is  helping  teachers 
to  act  as  a  guide  that  supports  successful  inquiry 
(Duffy  et  al.,  1996). 

Cognitive  Apprenticeship 

Cognitive  apprenticeship  is  an  approach  to  learning 
developed  by  Collins  et  al.  (1989)  to  take  advantage 
of  the  rich  contexts  through  which  learning  in  tradi¬ 
tional  apprenticeships  occurred.  They  developed  an 
instructional  model  derived  from  the  metaphor  of  the 
apprentice  working  under  the  master  craftsperson  in 
traditional  societies,  as  well  as  from  research  related 
to  how  people  learn  and  participate  in  everyday,  infor¬ 
mal  environments  (Rogoff  and  Lave,  1984).  Tradi¬ 
tional  apprenticeships,  however,  entailed  hands-on 
learning  trajectories,  evidenced  by  the  product  of  the 
craft.  In  contrast,  cognitive  apprenticeship  emphasizes 
cognitive  learning  trajectories  through  making  concep¬ 
tual  processes  visible.  In  this  approach,  the  teacher  or 
more  knowledgeable  other  serves  to  explicate  the 
thinking  underlying  the  teacher’s  actions,  thereby 
modeling  for  students  through  a  form  of  cognitive 
apprenticeship.  The  teacher  then  coaches  and  scaf¬ 
folds.  with  the  goal  being  to  gradually  fade  teacher 
intervention  such  that  the  student  can  engage  in  the 
behavior  without  the  support  of  the  teacher. 

Additional  instructional  strategies  include  articu¬ 
lation  as  learners  are  encouraged  to  verbalize  their 
knowledge  and  thinking,  reflection  on  the  learning 
experience,  and  exploration  as  they  begin  to  pose  prob¬ 
lems  of  their  own  creation.  Central  to  cognitive  appren¬ 
ticeship  is  that,  rather  than  simply  learning  a  particular 
fact  or  concept,  students  engage  in  a  rich  and  mean¬ 
ingful  task.  Collins  et  al.  ( 1989,  p.  487)  suggested  that 
“a  critical  element  in  fostering  learning  is  to  have 
students  carry  out  tasks  and  solve  problems  in  an  envi¬ 
ronment  that  reflects  the  multiple  uses  to  which  their 
knowledge  will  be  put  in  the  future.”  Indeed,  Palincsar 
and  Brown’s  (1984)  reciprocal  teaching  of  reading 
involves  many  of  the  features  of  cognitive  apprentice¬ 
ship  in  that  the  basic  method  centers  on  modeling  and 
coaching  students  in  four  strategic  skills:  formulating 
questions  based  on  the  text,  summarizing  the  text, 
making  predictions  about  what  will  ensue,  and  clari¬ 
fying  difficulties  with  the  text. 
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The  method  has  been  used  with  small  and  large 
groups  as  well  as  with  individual  students.  The  term 
reciprocal  teaching  denotes  that  the  teacher  and  stu¬ 
dents  take  turns  playing  the  role  of  teacher  as  they 
come  to  understand  a  text.  In  describing  its  effective¬ 
ness,  Palincsar  and  Brown  (1984)  argued  that  working 
with  a  text  in  a  discussion  format  differs  greatly  from 
teaching  isolated  comprehension  skills  such  as  identi¬ 
fying  the  main  idea.  When  students  engage  in  recip¬ 
rocal  teaching,  they  learn  strategies  that  serve  a  larger 
goal,  namely  both  to  understand  what  they  are  reading 
and  to  develop  the  critical  ability  to  read  and  learn. 
Another  example  lies  in  the  domain  of  mathematics 
(Schoenfeld,  1996).  In  this  work,  the  expert  thinks 
aloud  as  she  works  through  a  novel  problem  and  then 
reflects  with  other  learners  on  the  strategies  used  and 
the  paths  followed.  Describing  the  embodiment  of  one 
class,  Schoenfeld  (1996,  p.  213)  reported: 

Mathematics  was  the  medium  of  exchange.  We  talked 
about  mathematics,  explained  it  to  each  other,  shared  the 
false  starts,  enjoyed  the  interaction  of  personalities.  In 
short,  we  became  mathematical  people.  It  was  fun,  but 
it  was  also  natural  and  felt  right.  It  wasn’t  a  separate 
“school  experience”  for  a  few  hours  a  week.  By  virtue 
of  this  cultural  immersion,  the  students  experienced 
mathematics  in  a  way  that  made  sense. 

Schoenfeld  (1987)  emphasized  the  importance  of  cre¬ 
ating  a  microcosm  of  a  mathematician’s  culture  to  help 
students  think  like  mathematicians. 

Emergent  Simulation  Models 
for  Establishing  Embodiment 

The  designs  discussed  above  involve  developing  much 
of  the  learning  context  before  the  learners  have  even 
begun.  The  constraints  are  primarily  built  into  the 
resources,  and  the  acceptable  outcomes  are  fairly  well 
defined;  the  designer,  for  the  most  part,  knows  what  a 
correct  solution  is,  even  as  the  design  is  being  created. 
In  contrast,  an  emergent  simulation  model  situates 
much  of  the  interaction  within  the  tasks  but  leaves  the 
participants  to  socially  negotiate  much  of  the  structure 
of  the  activities  and  determine  what  counts  as  reason¬ 
able  evidence  for  success.  In  these  cases,  an  important 
contribution  to  their  authenticity  derives  from  convinc¬ 
ing  others  of  the  meaning  and  credibility  of  one’s 
interpretations.  Further,  such  an  approach  usually 
occurs  over  extended  durations  and  involves  both  less 
knowledgeable  and  more  knowledgeable  others  with 
whom  one  collaborates;  however,  in  contrast  to  partic¬ 
ipation  models,  these  remain  simulations  and  typically 
present  a  task  designed  for  the  learning  situation  as 


opposed  to  a  real-world  problem  that  the  work  may 
impact.  Again,  although  no  single  set  of  strategies  can 
capture  all  of  the  theories  and  designs  consistent  with 
this  perspective,  we  overview  several  theories  and 
designs  that  illuminate  the  category  of  emergent  sim¬ 
ulation  models. 

Case-Based  Reasoning 

Case-based  reasoning  (CBR)  originated  in  the  pursuit 
of  artificial  intelligence  within  the  field  of  computer 
science  (Schank,  1982),  but,  influenced  by  construc¬ 
tivist  conceptualizations  of  learning,  the  approach  now 
guides  a  range  of  educational  endeavors.  Indeed,  as  an 
instructional  strategy,  a  case-based  approach  affords  a 
rich  learning  context  without  necessitating  undue 
design  effort.  Like  PBL,  CBR  relies  on  prior  knowl¬ 
edge  and  experience  to  provide  a  rich  context  for  prob¬ 
lem  solving,  and,  significantly,  this  foundation  extends 
beyond  the  compass  of  the  individual  learner  to  allow 
the  knowledge  and  experience  of  others  in  the  field  to 
constitute  the  case  (Kolodner,  1993,  2006).  As  Jonas- 
sen  (1999,  p.  219)  wrote: 

Students  acquire  knowledge  and  requisite  thinking  skills 
by  studying  cases  ...  and  preparing  case  summaries  or 
diagnoses.  Case  learning  is  anchored  in  authentic  con¬ 
texts;  learners  must  manage  complexity  and  think  like 
practitioners. 

The  ill-structured  quality  of  the  cases,  then,  affords 
a  perceived  authenticity  on  the  part  of  the  learners 
(Jonassen  and  Hemandez-Serrano,  2002).  New  under¬ 
standings  naturally  ensue  from  the  interpretation,  con¬ 
sideration,  and  application  of  the  understandings 
embedded  in  the  cases,  whether  experienced  person¬ 
ally  or  vicariously,  provided  that  the  learner’s  experi¬ 
ence  is  informed  by  “concrete,  authentic,  and  timely 
feedback”  (Kolodner  and  Guzdial,  2000,  p.  220)  so  as 
to  facilitate  an  iterative  refinement  of  and  fluency  with 
the  competencies  germane  to  both  the  present  case  and 
others  that  complement  or  contrast  with  it. 

Case  libraries  bring  together  cases  reflecting  a 
domain  of  interest,  such  as  legal,  scientific,  or  social 
work.  These  cases  may  be  culled  not  only  from  the 
literature  of  the  domain  but  also  from  the  encounters 
of  other  students  who  have  engaged  with  a  domain- 
related  concept  or  skill  (Kolodner,  2006);  that  is,  the 
experience  of  one  student  can  be  presented  to  others 
as  a  case  through  which  they  can  vicariously  undergo 
the  experience,  thus  developing  individual  understand¬ 
ings  that  emerge  from  the  encounter.  In  this  way,  not 
only  case  libraries  but  also  class  discussions  represent 
tools  or  structures  that  can  be  designed  to  facilitate 
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learner  embodiment.  Still,  even  though  cases  advance 
student  understanding  of  domain  formalisms,  they 
nonetheless  do  so  by  embedding  these  formalisms  in 
a  rich  context;  while  scaffolding  how  students  conceive 
of  the  formalisms,  the  cases  achieve  this  through  nar¬ 
rative  rather  than  expository  discourse,  thus  affording 
an  authenticity  despite  the  designed  nature  of  the  expe¬ 
rience. 

Again,  although  case  libraries  demonstrate  the 
potential  for  instructors  to  design  effective  learning 
experiences,  engagement  with  the  cases  themselves 
remains  grounded  in  their  rich  contexts,  and  learning 
emerges  in  a  manner  emphasizing  student  ownership 
of  the  experience  and  ensuing  understandings.  This 
emergent  quality  is  exemplified  by  the  Balloon  Car 
Challenge  discussed  by  Kolodner  (2002).  One  in  a 
series  of  activities  in  a  middle-school  physical  science 
class,  the  Balloon  Car  Challenge  employs  a  number 
of  learning  theories  including  CBR  to  engage  students 
in  science  applications  that,  consistent  with  the  pre¬ 
cepts  discussed  above,  embody  the  development  of 
scientific  practices  within  the  iterative  testing  and 
refinement  of  possible  solutions.  This  unit  on  propul¬ 
sion  requires  students  to  test  different  numbers  of  bal¬ 
loons  as  well  as  different  lengths  and  diameters  of 
straws  connecting  the  balloons  to  the  vehicles.  Ulti¬ 
mately,  based  on  their  tests,  conjectures,  and  findings, 
students  combine  the  various  propulsion  systems  to 
enable  their  vehicles  to  traverse  a  demanding  course. 
Through  these  experiences,  students  develop  skills  in 
scientific  hypothesis,  experimentation,  and  explana¬ 
tion  as  well  as  evidence-based  decision  making.  More¬ 
over,  they  gain  experience  and  fluency  with  scientific 
discourse  — that  is,  “talking  and  doing  science.” 
Through  their  experiences,  they  engage  themselves  in 
authentic  practices  of  the  domain  of  science. 

Project-Based  Learning 

Project-based  learning  involves  posing  nontrivial  prob¬ 
lems,  in  response  to  which  students  formulate  ques¬ 
tions,  predictions,  experiments,  analyses,  conclusions, 
and  artifacts  (Blumenfeld  et  al.,  1991).  Typically  of 
extended  duration  and  collaborative  in  nature  (Brans- 
ford  and  Stein,  1993),  these  investigations  or  projects 
display  an  authenticity  that,  for  the  students,  facilitates 
an  embodied  curricular  experience.  In  contrast  to  PBL, 
which  entails  designed  or  structured  problems  guiding 
student  discovery  of  “correct”  solutions,  the  outcomes 
of  project-based  learning  are  uncertain,  emerging  from 
the  practices  of  the  students  themselves  as  they  create 
links  between  the  classroom  and  the  real  world  and 
across  diverse  academic  disciplines.  In  short,  although 
the  teacher  may  establish  the  learning  environment,  it 


is  the  student  motivation,  direction,  and  achievement 
that  characterize  the  learning  process.  Still,  though,  the 
teacher  must  also  support  this  process  due  to  its  pro¬ 
tracted  and  often  ambiguous  nature  (Blumenfeld  et  al., 
2006).  Two  essential  aspects  of  project-based  learning 
deserve  reiteration:  First,  because  student-formulated 
questions  drive  the  inquiry  projects,  the  students  con¬ 
struct  their  own  understandings  of  domain  formalisms , 
and,  second,  because  the  projects  culminate  in  con¬ 
crete,  explicit  artifacts,  the  students  can  reflect  on  these 
understandings  and  revise  them  based  on  feedback 
from  others. 

In  the  early  1990s,  researchers  found  middle- 
school  and  high-school  science  education  to  be  prob¬ 
lematic:  superficial  in  addressing  a  broad  range  of  top¬ 
ics  at  only  shallow  levels  of  complexity  and  artificial 
in  structuring  student  experience  in  generic  and  pre¬ 
dictable  ways.  Further,  not  only  textbooks  but  also 
teacher  practices  failed  to  recognize  the  need  for  stu¬ 
dents  to  bring  their  prior  knowledge  to  bear  upon  their 
personal  construction  of  scientific  understandings.  In 
response,  building  on  the  precepts  of  project-based 
learning,  several  domain- specific  projects  emerged  that 
together  characterized  project-based  science  (Krajcik 
and  Blumenfeld,  2006),  including  the  Center  for 
Fearning  Technologies  in  Urban  Schools  (FeTUS)  and 
the  Investigating  and  Questioning  our  World  through 
Science  and  Technology  (IQWST)  project,  both  of 
which  developed  out  of  school-university  collabora¬ 
tions.  Central  to  the  current  discussion,  these  efforts 
embodied  the  learning  of  science  practices  and  con¬ 
cepts  within  the  context  of  a  larger  project. 

As  Krajcik  and  Blumenfeld  (2006)  explained,  these 
projects  were  driven  by  authentic  questions— meaning¬ 
ful  and  contextualized  questions  that  structured  student 
activity  and  aligned  that  activity  with  academic  stan¬ 
dards.  Second,  student  activity  bore  authenticity  in  that 
it  was  situated  in  the  practices  and  formalisms  of  the 
scientific  domain;  again,  students  engaged  in  sustained 
inquiry  resulting  in  scientific  explanations.  Third,  stu¬ 
dent  work  was  dependent  upon  collaboration— not  only 
with  other  students,  who  corroborated  and  critiqued 
emergent  understandings,  and  not  only  with  the  teacher, 
who  helped  to  structure  and  support  the  inquiry  process, 
but  also  with  members  of  the  community  beyond  the 
classroom,  who  helped  the  students  become  more  fluent 
with  the  discourse  and  practices  of  the  domain.  Fourth, 
these  projects  leveraged  technology  tools  to  not  only 
align  student  activity  with  authentic  scientific  practices 
but  also  facilitate  student-constructed  understandings 
and  artifacts.  Indeed,  this  relates  closely  to  the  fifth 
feature  of  the  project-based  science  projects  discussed 
by  Krajcik  and  Blumenfeld,  namely  that  the  instantia¬ 
tion  of  student  understanding  in  the  creation  of  artifacts 
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served  to  advance  not  only  the  understandings  of  other 
students  but  also,  through  feedback  and  iterative  revi¬ 
sion,  the  students’  own  emergent  understandings. 

Classroom  Learning  Communities 

As  argued  above  with  respect  to  cognitive  apprentice¬ 
ship,  when  one  employs  a  situative  perspective  to 
inform  classroom  practice,  the  culture  of  the  class¬ 
room  will— indeed,  must— be  transformed  (Brown, 
1992;  Schoenfeld,  1988).  As  Driscoll  (2000,  p.  175) 
explained,  “The  traditional  social  structure  of  schools 
is  one  in  which  teachers  dispense  knowledge  to  stu¬ 
dents  through  classroom  activities,  textbooks,  and 
possibly  other  media.”  Such  an  arrangement  privi¬ 
leges  the  teacher  as  controlling  student  learning  and 
portrays  the  teacher  and  the  textbook  alone  as  bearing 
legitimate  knowledge.  "When  a  classroom  becomes 
a  learning  community,  however,  the  social  structure 
transforms  into  one  in  which  teacher  and  learners 
work  collaboratively  to  achieve  important  goals,  goals 
that  may  well  have  been  established  jointly”  (Driscoll, 
2000,  p.  175).  In  this  way,  learning  communities 
instantiate  distributed  expertise,  collaboration,  and 
emergent  ownership. 

For  over  a  decade.  Brown  and  Campione  (Brown, 
1992;  Brown  and  Campione,  1990;  Brown  et  al.,  1994) 
have  engineered  and  researched  communities  of  learn¬ 
ers.  In  Brown’s  (1992)  seminal  work  on  design  exper¬ 
iments,  she  discussed  the  process  of  building  a  learning 
community  among  fifth  graders.  Their  classrooms 
showed  an  atmosphere  of  respect  where  students’ 
statements  were  regarded  earnestly  and  their  questions 
were  considered  seriously;  moreover,  students  devel¬ 
oped  a  community  of  discourse  in  which  “meaning  is 
negotiated  and  renegotiated  as  members  of  the  com¬ 
munity  develop  and  share  expertise”  (Brown  et  al., 
1994,  p.  200).  Central  to  these  efforts  was  the  use  of 
reciprocal  teaching  and  jigsaw  methods  to  engage  stu¬ 
dents  in  collaboration.  Because  reciprocal  teaching  has 
been  discussed  above,  the  following  focuses  on  the 
jigsaw  method. 

This  instructional  strategy  involves  students  work¬ 
ing  collaboratively  to  develop  expertise  with  one  com¬ 
ponent  of  a  larger  task;  once  they  have  mastered  the 
components  for  which  they  are  responsible,  students 
engage  in  reciprocal  teaching  to  share  what  they  have 
learned  with  other  members  of  the  group.  Through 
these  techniques,  educators  can  develop  repetitive 
structures,  called  cycles,  to  help  students  gain  facility 
with  the  various  components  and  recognize  their  devel¬ 
oping  mastery  over  time.  Lasting  approximately  10 
weeks,  these  cycles  begin  with  a  teacher  or  visiting 
expert  introducing  a  unit  and  benchmark  lesson  and 


emphasizing  a  holistic  view  of  the  task  as  well  as  an 
analytic  perspective  of  how  the  various  topics  interre¬ 
late.  Students  then  concentrate  on  researching  their 
pieces  of  the  puzzle  and  teaching  them  to  others  in 
their  groups,  with  the  distributed  expertise  emerging 
as  students  develop  competence  in  the  various  aspects 
of  their  tasks.  The  teacher  models  the  jigsaw  practice 
throughout  the  research  cycle,  and  at  its  completion 
the  groups  engage  in  full  reciprocal  teaching,  with  each 
student  participating  as  an  expert  of  his  or  her  own 
facet  of  the  topic  material. 

Significantly,  with  technology-mediated  commu¬ 
nication  being  employed  increasingly  in  educational 
settings,  such  structures  as  e-mail  and  teleconferencing 
allow  students  to  collaborate  not  only  face  to  face  but 
also  distributed  across  space  and  time  and  to  commu¬ 
nicate  with  not  only  each  other  but  the  wider  commu¬ 
nity  as  well.  Indeed,  Scardamalia  and  Bereiter  (Bere¬ 
iter  and  Scardamalia,  1996;  Scardamalia  and  Bereiter, 
1994, 1996)  presented  another  interesting  example  that 
leverages  technological  resources  for  supporting  col¬ 
laboration.  One  exciting  aspect  of  their  work  is  that 
the  community  builds  a  communal  database  that  each 
new  class  engages,  embodying  the  learning  not  only 
in  the  immediate  classroom  rituals  but  also  as  part  of 
an  extended  community  evolving  over  time. 

Participation  Models 

for  Establishing  Embodiment 

Participation  models  establish  a  sense  of  authenticity 
by  engaging  learners  in  the  authentic  practices  as  they 
work  on  real  tasks,  frequently  as  a  member  of  a  com¬ 
munity  including  newcomers  and  more  established  old 
timers  (Lave  and  Wenger,  1991).  As  with  both 
designed  and  emergent  simulations,  participation  mod¬ 
els  entail  factual,  process,  and  task  authenticity  but 
additionally  introduce  ecological  authenticity  in  that 
the  learner  engages  real-world  tasks  within  the  ecolog¬ 
ical  niche  where  the  tasks  bear  value.  In  addition  to 
real-world  immersion,  these  designs  also  integrate 
considerable  pedagogical  scaffolding,  including  prac¬ 
tice  scenarios  occurring  in  the  classroom  as  well  as 
simple  just-in-time  lectures  tailored  to  help  learners 
understand  the  necessary  content  and  participate  more 
meaningfully  in  related  tasks.  In  fact,  for  these  types 
of  spaces,  an  important  contingency  of  their  success 
as  contexts  for  learning  is  that  even  such  real-world 
participation  is  structured  so  non-experts  can  succeed, 
in  contrast  to  contexts  that  simply  immerse  learners  in 
a  sink-or-swim  scenario.  Still,  all  pedagogically  struc¬ 
tured  activities,  whether  classroom  based  or  not,  are 
embodied  in  the  participatory  context  of  a  real-world 
task  with  real-world  implications. 
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Participatory  Simulations 

The  participation  model  of  authenticity  is  founded  on 
the  assumption  that  the  authenticity  of  a  learning  activ¬ 
ity  derives  from  learners  participating  in  a  real-world 
problem  in  an  authentic  context.  Participation  models 
designed  for  school  contexts  pose  challenges  because 
they  require  that  students  leave  the  classroom  environ¬ 
ment  and  participate  directly  with  real-world  practitio¬ 
ners  as  members  of  their  communities.  One  means  of 
establishing  participatory  authenticity  entails  fostering 
student-expert  partnerships  (SEPs)  as  part  of  the  edu¬ 
cational  experience.  Central  to  the  SEP  model  is  adopt¬ 
ing— that  is,  learning  and  doing— the  particular  con¬ 
tent  through  hands-on,  minds-on  activities  that  afford 
students  an  opportunity  to  participate  in  a  project  with 
real-world  impact  and  with  significance  to  both  the 
students  and  real-world  practitioners.  Examples  of 
such  partnerships  include  the  Global  Learning  and 
Observations  to  Benefit  the  Environment  (GLOBE) 
project  (Finarelli,  1998;  Rock  and  Lawless,  1997),  the 
Global  Rivers  Environmental  Education  Network 
(GREEN)  project  (Donahue  et  al.,  1998),  and  Forest 
Watch  (Rock  and  Lauten,  1996).  Central  to  all  these 
projects  is  that  the  practices  and  outcomes  are  deemed 
authentic  and  owned  by  the  learner  as  well  as  the 
community  of  practice  and  that  they  respond  to  real- 
world  issues. 

Another  example,  the  National  Geographic  Soci¬ 
ety’s  Kids  Network  curriculum  (Bradsher  and  Elogan, 
1995;  Karlan  et  al.,  1997),  has  engaged  over  1.5  mil¬ 
lion  children  from  50  countries  in  collaborative  inves¬ 
tigations  of  real-world  issues.  The  curriculum  consists 
of  8-week  curricular  units  targeted  toward  fourth 
through  sixth  graders.  During  implementation,  five  to 
ten  geographically  dispersed  classrooms  connect 
through  the  Kids  Network  personnel  to  form  a  research 
team.  The  students  begin  by  studying  the  curriculum 
area  (e.g.,  global  warming,  stream  erosion)  and  dis¬ 
cussing  the  general  issue  in  relation  to  their  specific 
community;  the  distributed  classes  work  as  a  team, 
negotiating  how  to  approach  the  issues  to  reflect  the 
local  interests  of  each  group.  This  process  of  collabo- 
ratively  determining  the  particular  research  issues  bal¬ 
ances  different  groups’  interests,  allows  for  ownership 
and  legitimacy,  and  makes  possible  the  global  compar¬ 
isons.  Once  the  research  issues  have  been  determined, 
the  groups  develop  data  collection  tools  and  gather 
samples  from  their  communities,  with  Kids  Network 
experts  helping  to  discuss  issues  and  offer  guidance. 

The  Kids  Network  staff  receives  and  integrates 
data  from  the  different  locations.  Data  summaries  are 
then  prepared  and  sent  to  participating  classrooms;  in 
addition,  a  practicing  scientist  sends  initial  interpreta¬ 


tions  of  the  data  to  model  scientific  thinking  for  the 
students  (Bradsher  and  Hogan,  1995).  The  students 
complete  the  lesson  by  forming  original  interpretations 
of  the  data,  drawing  conclusions  relevant  to  their  com¬ 
munities,  and  preparing  presentations  of  the  findings 
for  a  community  audience.  These  findings  and  presen¬ 
tations  also  become  incorporated  within  the  network 
such  that  subsequent  groups  can  learn  from  previous 
efforts.  Bradsher  and  Hogan  (1995,  p.  39),  two 
National  Geographic  Society  project  personnel,  char¬ 
acterized  the  Kids  Network  curriculum: 

Students  pose  and  research  questions  about  their  local 
community,  form  hypotheses,  collect  data  through  exper¬ 
iments,  and  analyze  results.  The  answers  are  largely 
unknown  in  advance,  and  the  findings  are  of  interest 
beyond  the  classroom. 

Although  the  curriculum  is  considerably  more  struc¬ 
tured  than  described  here  and  the  findings  more  pre¬ 
scriptive  (Hunter,  1990;  Karlan  et  al.,  1997),  the 
approach  bears  potential  for  engaging  students  in 
authentic  scientific  problems  and  discourse  alongside 
other  students  and  scientists.  Central  to  the  embodi¬ 
ment  is  that,  through  their  work,  students  make  signif¬ 
icant  contributions  to  their  real  world  communities. 

Academic  Play  Spaces 

Academic  play  spaces  are  educational,  entertaining,  and 
personally  transformative  curricular  contexts  designed 
according  to  contemporary  theories  of  learning  (Brans- 
ford  et  al.,  2000),  narrative  (Bruner,  2002;  Rosenblatt, 
1938/1995),  interactive  narrative  (Murray,  1997),  and 
game  design  (Gee,  2003;  Salen  and  Zimmerman,  2004). 
Significantly,  the  structure  and  content  of  these  spaces 
align  with  not  only  academic  standards  and  domain 
formalisms  but  also  real-world  issues  and  authentic 
practices.  Certainly,  games  can  establish  immersive 
“worlds”  affording  a  sense  of  agency  and  consequenti- 
ality,  but  academic  play  spaces  additionally  involve  cur¬ 
ricular  tasks  and  pedagogical  elements  to  provide  for 
academic  learning.  These  environments  support  com¬ 
plex  forms  of  participation  within  interactive  narrative 
frames,  wherein  learners  have  agency  in  determining 
how  these  narratives  unfold.  Central  challenges  in  devel¬ 
oping  such  narrative-based  curricula  include  not  only 
meaningfully  situating  academic  content  but  also  situ¬ 
ating  the  learner  in  such  a  way  that  knowing  becomes 
a  process  of  embodiment  and  participation.  Further,  the 
design  should  position  the  disciplinary  content  so  it  both 
yields  outcomes  in  the  immediate  context  and  transfers 
to— that  is,  relates  and  applies  to— other,  future  narra¬ 
tive  experiences  (Bereiter,  1997).  In  this  way,  academic 
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play  spaces  necessarily  blur  the  lines  of  the  magic  circle, 
the  imaginary  boundary  between  the  game  world  and 
other  non-game  worlds  (Murray,  1997;  Salen  and  Zim¬ 
merman,  2004). 

The  process  of  designing  academic  play  spaces  to 
support  embodied  experiences  may  be  illustrated  by 
the  case  of  Quest  Atlantis,  a  learning  and  teaching 
project  employing  a  virtual  multi-user  environment  to 
immerse  children  ages  9  to  12  in  educational  chal¬ 
lenges  (Barab  et  al.,  2005b).  Leveraging  strategies 
from  commercial  games  as  well  as  research  on  learn¬ 
ing  and  motivation.  Quest  Atlantis  allows  children  to 
travel  through  virtual  spaces  to  perform  entertaining, 
academic  activities;  they  can  also  text  chat  with  other 
children  and  mentors  and,  through  customizing  their 
avatars  and  project  homepages,  develop  their  virtual 
personae.  Additionally,  the  program  advocates  a  series 
of  pro-social  life  commitments  (e.g.,  environmental 
awareness,  creative  expression,  diversity  affirmation) 
to  foster  student  awareness  and  empowerment  (Gard¬ 
ner  et  al.,  2001).  Children  engage  with  these  commit¬ 
ments  throughout  their  participation  in  the  project  as 
well  as  through  the  fictional  meta-narrative  featuring 
seven  teenage  Council  members  on  the  planet  Atlantis. 
To  blur  the  magic  circle  of  the  game,  children’s  in¬ 
game  identities  depend  on  their  successful  completion 
of  tasks  with  their  local  community,  as  part  of  their 
out-of-school  life  worlds  (Barab  and  Roth,  2006). 

Each  Council  member  champions  one  of  the  com¬ 
mitments  so  children  develop  rich  understandings  and 
ecological  perspectives  regarding  the  commitments. 
Significantly,  the  curricular  tasks  often  involve  engag¬ 
ing  in  not  simply  the  virtual  but  the  real  world  as  well , 
such  as  collecting  data  about  the  water  quality  of 
nearby  rivers  to  inform  others  about  the  process.  In 
this  project,  children  are  not  simply  audience  to  the 
underlying  narrative  as  it  is  advanced  through  novels, 
comic  books,  and  other  regalia,  although  they  find 
these  play  structures  to  be  immersive  and  compelling 
(Barab  et  al.,  2005a).  Rather,  they  are  participants  in 
the  narrative  as  they  engage  in  activities,  acquiring  new 
potentialities  and  even  contributing  to  the  evolution  of 
the  narrative  itself  through,  for  example,  interacting 
with  the  Council  members  and  voting  on  pertinent 
issues.  Indeed,  children’s  transactions  with  the  narra¬ 
tive,  the  virtual  space,  and  their  out-of-game  commu¬ 
nities  collectively  come  to  afford  an  embodied  expe¬ 
rience  that  is  simultaneously  virtual  and  real. 

Communities  of  Practice 

Communities  of  practice  demonstrate  many  of  the 
aspects  of  learning  communities  discussed  above  but 
also  involve  more  extended  time  frames  and  evolve  in 


authentic  contexts.  Lave  and  Wenger  (1991)  advanced 
the  term  community  of  practice  to  capture  the  impor¬ 
tance  of  activity  in  fusing  individuals  to  communities 
and  of  communities  in  legitimizing  individual  prac¬ 
tices.  Within  this  context,  learning  is  regarded  as  a 
trajectory  along  which  learners  move  from  being  legit¬ 
imate  peripheral  participants  to  core  participants  in  the 
community  of  practice.  Building  on  prior  conceptual¬ 
izations  (Barab  and  Duffy,  2000),  Barab  et  al.  (2004, 
p.  55)  defined  a  community  of  practice  as  “a  persistent, 
sustained  social  network  of  individuals”  that  demon¬ 
strates  the  following  characteristics:  (1)  shared  knowl¬ 
edge,  values,  and  beliefs;  (2)  overlapping  histories 
among  members;  (3)  mutual  interdependence; 
(4)  mechanisms  for  reproduction;  (5)  a  common  prac¬ 
tice  and/or  mutual  enterprise;  (6)  opportunities  for 
interactions  and  participation;  (7)  meaningful  relation¬ 
ships;  and  (8)  respect  for  diverse  perspectives  and 
minority  views.  Significantly,  communities  of  practice 
are  nested  in,  and  overlap  with,  other  communities, 
and  they  can  exist  solely  in  the  real  world  or  be  sup¬ 
ported  in  part  or  whole  by  online  sociotechnical  struc¬ 
tures. 

One  example  of  an  educational  community  of 
practice  is  the  Community  of  Teachers,  a  university- 
based  professional  development  program  for  pre-ser¬ 
vice  teachers  (Barab  et  al.,  2002).  The  program  empha¬ 
sized  fieldwork,  with  each  participant  committing  to  a 
particular  school.  They  were  not  assigned  to  a  specific 
teacher;  instead,  an  apprenticeship  relation  would 
develop  with  a  teacher  based  on  social  negotiation  and 
mutual  recognition  of  the  benefit  of  a  relationship.  In 
this  way,  students  became  paired  with  “old  timers” 
during  their  first  year  working  toward  certification,  and 
they  continued  to  work  with  these  teachers  for  their 
duration  in  the  program.  Further,  each  student  negoti¬ 
ated  membership  in  a  community  of  students  studying 
to  be  teachers  and,  again,  remained  a  part  of  that  com¬ 
munity  for  the  duration  of  their  study,  sharing  their 
developing  understandings  and  skills  and  continually 
negotiating  the  goals  and  meanings  of  both  the  com¬ 
munity  and  the  profession  (Barab  et  al.,  2002).  As  with 
any  community,  the  Community  of  Teachers  included 
wizened  old  timers  (students  with  teaching  experi¬ 
ence),  newcomers  (sophomore  students),  and  the  levels 
between,  all  engaged  in  a  common  enterprise. 

Further,  the  program  codified  a  list  of  23  Program 
Expectations  that  each  student  satisfied  through  the 
apprenticeship  (Gregory,  1993),  and,  indeed,  these 
expectations  were  realized  through  embodied  prac¬ 
tices  rather  than  through  lectures  or  textbooks.  Stu¬ 
dents  amassed  evidence  of  their  teaching  in  personal 
portfolios  that  demonstrated  how,  through  their  expe¬ 
riences,  they  satisfied  the  expectations.  Because  it  was 
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Figure  9.1  Core  tensions  in  designing  for  embodiment. 


the  portfolio  that  supported  the  case  that  a  student  was 
ready  to  enter  the  profession,  over  time  there  grew  a 
collection  of  personal  narratives  that  came  to  embody 
the  canonical  practices  of  the  community,  helping  the 
students  to  develop  a  shared  language  to  describe  par¬ 
ticular  group  practices.  The  community  fostered  a  tra¬ 
dition  and  heritage  that  was  continually  inherited  and 
revised  by  members  as  they  became  a  part  of  the  pro¬ 
gram.  Moreover,  the  community  trajectory  extended 
across  different  classrooms  and  occasions  in  such  a 
way  that  individuals  came  to  regard  themselves  as 
becoming  increasingly  central  within  the  nested  com¬ 
munities  of  preservice,  practicing,  and  seasoned  teach¬ 
ers.  An  important  component  of  this  program  and, 
more  broadly,  the  community  of  practice  model  for 
establishing  embodiment  is  that  the  work  or  practice 
of  the  members  is  situated  in  both  real-world  and 
reflective  contexts  and  that  becoming  a  member  of  the 
community  and  becoming  knowledgeably  skillful  go 
hand  in  hand  (Wenger,  1998). 


CONCLUDING  REMARKS 

If  situativity  theorists  are  correct  and  the  context  in 
which  one  learns  influences  one’s  coming  to  know  the 
content  to  be  learned,  then  understanding  strategies  for 
situating  content  becomes  critical.  In  this  chapter,  we 
have  provided  a  conceptual  framework  and  examples 
from  the  literature  related  to  designing  for  situational 
embodiment.  This  was  not  meant  to  represent  an 
exhaustive  list  of  examples  but,  rather,  an  illuminative 
one.  Additionally,  we  have  discussed  three  types  of 


curricular  designs  for  embodiment,  ranging  from 
designed  and  emergent  simulation  models  to  partici¬ 
pation  models.  It  is  our  conviction  that  each  of  these 
bears  advantages  and  disadvantages.  The  control  and 
ease  of  implementation  for  designed  simulations 
makes  them  more  likely  to  be  adopted  in  the  class¬ 
room.  Because  of  the  rich  dialog  and  potential  agency 
they  afford  learners,  emergent  simulations  entail  an 
additional  level  of  authenticity  and  ownership,  but  this 
also  complicates  their  use  within  K-12  classrooms. 
Finally,  participation  models  do  not  require  specula¬ 
tion  regarding  real-world  scenarios  but  actually  take 
place  within  authentic  contexts;  yet,  this  type  of 
embodiment  proves  to  be  time  consuming,  difficult  to 
support,  and,  indeed,  although  rich  in  experience, 
potentially  pedagogically  impoverished. 

Looking  across  the  different  examples,  we  also  dis¬ 
cern  tensions  that  emerge  in  working  toward  curricular 
embodiment— namely,  tensions  concerning  the  quality 
of  the  context  (noisy  vs.  tailored)  and  the  quality  of  the 
formalisms  (explicit  vs.  implicit).  Indeed, balancing  the 
curricular  experience  across  the  four  quadrants  illus¬ 
trated  in  Figure  9.1  poses  a  key  challenge  for  instruc¬ 
tional  design  and  represents  an  important  matter  to 
examine  in  future  research.  Specifically,  curricula 
emphasizing  explicit  formalism  risks  occasioning  an 
overly  school-like  experience,  while  excessive  focus  on 
context  may  be  inefficient— especially  given  the  con¬ 
temporary  preoccupation  with  facilitating  students  in 
accumulating  standards.  Further,  prior  experience  and 
knowledge  likely  affect  the  extent  to  which  narrative  is 
necessary  to  establish  embodiment.  For  students  who 
already  have  a  rich  appreciation  of  contexts  of  use. 
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establishing  an  elaborate  narrative  context  for  learning 
might  be  ineffectual  and  potentially  counterproductive; 
instead,  attending  to  explicitly  tailored  instruction 
through  a  lecture  format  might  be  optimal. 

In  general,  we  posit  that  a  situationally  embodied 
curriculum  is  not  simply  a  sophisticated  word  problem 
such  as  may  be  found  in  a  conventional  textbook; 
instead,  the  design  theories  described  above  target  a 
qualitatively  different  type  of  curriculum.  It  is  our  core 
argument  that,  when  domain  formalisms  are  situated 
within  a  context  of  use,  their  meanings  and  the  learner 
relations  with  these  meanings  are  usefully  trans¬ 
formed,  and  it  is  establishing  such  meaningful  rela¬ 
tion— not  inert  content  acquisition— that  education 
should  support  (Dewey,  1938/1997).  To  situate  this 
argument,  we  have  presented  a  number  of  design  the¬ 
ories  that,  in  different  ways,  all  embody  content  within 
a  rich  context.  Further,  they  exist  not  simply  as  design 
theories  but  have  been  put  into  practice  to  produce 
actual  designs,  enlist  empirical  research  into  their 
effectiveness,  and  advance  theoretical  approaches  to 
instructional  design.  Within  the  allotted  space,  we  have 
not  been  able  to  provide  enough  detail  to  embody  the 
learner  within  any  of  these  approaches.  Instead,  our 
goals  were  more  modest:  to  provide  sketches  of  and 
connections  among  them  so  the  interested  reader  may 
pursue  them  further.  Our  main  goal,  however,  was  to 
highlight  a  theory  of  embodiment  and  provide  an  over¬ 
view  of  approaches  for  conceptualizing  and  possibly 
even  designing  embodied  curriculum. 
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ABSTRACT 

The  first  and  second  edition  of  this  chapter  defined 
generative  learning  and  its  foundations  and  presented 
relevant  research  that  tested  the  theory.  The  primary 
goal  of  this  edition  is  to  reconceptualize  the  processes 
for  making  meaning  by  synthesizing  theoretical  foun¬ 
dations  of  generative  learning  and  exploring  generative 


learning  effects  by  different  types  of  learning  out¬ 
comes.  The  essence  of  this  model  of  generative  learn¬ 
ing  is  knowledge  generation.  Only  through  learners’ 
generation  of  relationships  and  meaning  themselves 
can  knowledge  be  generated  that  is  sustainable— this 
is  the  essential  process  of  meaning  making  by  the 
learner.  Likewise,  only  those  activities  that  involve  the 
actual  creation  of  relationships  and  meaning  would  be 
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classified  as  examples  of  generative  learning  strate¬ 
gies.  A  variety  of  studies  reporting  on  results  of  gen¬ 
erative  strategies  have  shown  that,  in  most  cases,  active 
learner  involvement  produced  increased  gains  in  recall, 
comprehension,  and  higher  order  thinking  or  improve¬ 
ment  in  self-regulated  learning  skill.  Misconception, 
providing  feedback,  and  developmental  appropriate¬ 
ness  are  issues  that  have  emerged  as  unresolved.  As 
such,  there  is  much  research  left  to  do  to  establish 
specific  guidelines  that  help  the  designer  create  a  learn¬ 
ing  environment  that  stimulates  attention  and  inten¬ 
tion,  promotes  active  mental  processing  at  all  stages 
and  levels  of  learning,  and  provides  the  learner  with 
appropriate  help  in  the  generation  process. 


KEYWORDS 

Attention:  Arousal  and  intention  in  the  brain  that  influ¬ 
ence  an  individual’s  learning  processes.  Without 
active,  dynamic,  and  selective  attending  of  envi¬ 
ronmental  stimuli,  it  follows  that  meaning  genera¬ 
tion  cannot  occur. 

Knowledge  generation:  Generation  of  understanding 
through  developing  relationships  between  and 
among  ideas. 

Meaning  making:  The  process  of  connecting  new  infor¬ 
mation  with  prior  knowledge,  affected  by  one’s 
intention,  motivation,  and  strategies  employed. 

Motivation  processes:  Wittrock  (1991)  specified  inter¬ 
est  and  attribution  as  being  the  two  essential  and 
linked  components  of  motivation  processes  acti¬ 
vated  by  arousal  and  intention  through  the  descend¬ 
ing  reticular  activation  system. 

Self-regulation:  Active  participation  in  terms  of  behav¬ 
ior,  motivation,  and  metacognition  in  one’s  own 
learning  process  (Zimmerman,  1986). 


INTRODUCTION 

The  internal  processes  of  learning  and  how  they  are 
stimulated  are  of  prime  importance  to  instructional 
designers.  These  processes  are  understood  through 
extensive  theoretical  conceptions  and  predictions  and 
empirical  evidence  about  cognitive  functioning,  pro¬ 
cesses,  and  the  structure  of  memory.  Using  theoretical 
foundations  about  learning,  designers  develop  a  con¬ 
ception  of  the  thinking  that  occurs  within  the  learner, 
and  they  use  this  conception  to  guide  the  design  of 
learning  environments.  One  important  concept  explain¬ 
ing  these  processes  is  generative  learning  theory.  Gen¬ 
erative  learning  theory,  with  its  companion  model,  gen¬ 
erative  teaching,  has  a  foundation  in  neural  research  as 


well  as  research  regarding  the  structure  of  knowledge 
and  cognitive  development.  Generative  learning  theory 
focuses  on  selecting  appropriate,  learner-centric 
instructional  activities  for  the  learner.  This  theory  is  one 
that  combines  the  importance  of  learner  and  instruc¬ 
tional  intentionality.  Because  of  this  blending,  Bonn 
and  Grabowski  (2001)  call  generative  learning  theory 
a  practical  cousin  of  constructivism.  Perhaps  it  also 
provides  a  more  complete  perspective  about  learning, 
making  it  a  second  cousin  to  behaviorism.  The  theory 
brings  together  our  understanding  of  learning  processes 
and  the  design  of  external  stimuli  or  instruction. 
Describing  how  meaning  is  made  for  different  types  of 
learning  outcomes  is  the  challenge  of  this  chapter. 

MAKING  MEANING  IN 
GENERATIVE  LEARNING 

Generative  Learning  Foundations 

Wittrock  (1974a, b)  was  the  founder  of  generative 
learning  theory.  His  beliefs  about  learning  were  influ¬ 
enced  by  research  in  several  areas  of  cognitive  psy¬ 
chology,  including  cognitive  development,  human 
learning,  human  abilities,  information  processing,  and 
aptitude  treatment  interactions.  His  work  explains  and 
prescribes  teaching  strategies  to  maximize  reading 
comprehension.  In  his  theory,  Wittrock  emphasized 
one  very  significant  and  basic  assumption:  The  learner 
is  not  a  passive  recipient  of  information;  rather,  he  or 
she  is  an  active  participant  in  the  learning  process, 
working  to  construct  meaningful  understanding  of 
information  found  in  the  environment.  Wittrock 
(1974a,  p.  182)  stated  that,  “although  a  student  may 
not  understand  sentences  spoken  to  him  by  his  teacher, 
it  is  highly  likely  that  a  student  understands  sentences 
that  he  generates  himself.”  It  is,  as  Harlen  and  Osborne 
(1985,  p.  137)  call  it,  “learning  through  the  person.” 

Designing  instruction  based  on  this  basic  assump¬ 
tion,  however,  is  not  as  simple  or  straightforward  as  it 
may  first  appear.  Wittrock  built  his  model  around  four 
parts  based  on  a  neural  model  of  brain  functioning 
(Luria,  1973)  and  cognitive  research  on  the  process  of 
knowing  (Wittrock,  1992).  As  is  evident  from  the 
empirical  results  that  are  reported  later  in  this  chapter, 
those  studies  that  apply  a  simplistic  rather  than  a  holis¬ 
tic  perspective  result  in  mixed  or  unpredicted  findings. 
It  is  important,  therefore,  to  elaborate  on  the  interre¬ 
lationship  of  the  four  parts  to  Wittrock’s  model:  gen¬ 
eration,  motivation,  learning,  and  knowledge  creation. 
Metacognitive  processes  also  play  a  key  role  in  this 
model,  although  in  most  cases  Wittrock  folds  meta¬ 
cognition  into  the  knowledge  creation  process. 
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The  Process  of 


Figure  10.1  Conceptual  understanding  of  generative  learning. 


Interrelationship  of  the  Components 
of  Generative  Learning 

The  concept  map  in  Figure  10.1  is  illustrative  of  our 
conceptualization  of  the  ideas  presented  in  Wittrock’s 
writings  (1974a,b,  1985,  1990,  1991,  1992)  and  in 
Grabowski’s  (2004)  concept  map  regarding  the  pro¬ 
gression  of  generative  learning.  As  shown  inside  the 
dotted  rectangle  in  Figure  10.1,  the  essence  of  gener¬ 
ative  learning  is  knowledge  generation.  Only  through 
the  learner’s  self-generation  of  relationships  and 
understanding  can  knowledge  be  generated  meaning¬ 
fully.  Only  those  activities  that  involve  the  actual  cre¬ 
ation  of  relationships  and  meaning  would  be  classified 
as  examples  of  generative  learning  strategies.  Restruc¬ 
turing  of  environmental  information  by  definition 
requires  the  learner  to  generate  either  organizational 
or  integrated  relationships  and  construct  personal 
meaning.  One  part  of  meaning  making  results  from  the 
processes  of  generating  relations  between  memory 
including  preconceptions,  abstract  knowledge,  every¬ 
day  experience,  domain-specific  knowledge,  and  new 


information  (Wittrock,  1974b).  This  connection  is 
shown  by  the  connecting  arrows.  Generation  can  result 
in  schema  fitting  (Rummelhart,  1981;  Rummelhart  and 
Ortony,  1977). 

Basically,  data  points  or  schemata  form  the  knowl¬ 
edge  units  that  are  manipulated  in  generative  learning 
theory.  Because  of  the  way  knowledge  is  stored, 
instructional  and  learning  activities  must  connect  new 
to  existing  knowledge  so  it  is  easily  retrievable.  This 
connection  is  made  by  adding  information  to  schema, 
restructuring,  or  tuning  it.  Although  connections  are 
made  by  links,  those  linkages  are  not  defined  or  labeled, 
as  in  creating  a  pattern  note  without  labeling  the  lines. 
Generative  learning  theory,  on  the  other  hand,  is  similar 
in  concept  to  creating  a  pattern  note  with  all  the  links 
labeled.  Activities  designed  by  schema  theorists  would 
include  those  that  remind  learners  of  prior  knowledge 
and  relate  the  information  to  what  the  learners  already 
know.  It  is  less  relevant  who  selects  those  connection 
points  compared  to  the  fact  that  they  are  made. 

By  definition,  learners  should  become  account¬ 
able  and  responsible  in  learning  and  mentally  active 
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in  constructing  relations  between  what  they  know  and 
what  they  are  learning  (Wittrock,  1990).  Motivation, 
the  second  of  Wittrock’s  components  shown  inside 
the  dotted  rectangle,  promotes  the  impulse  or  inten¬ 
tion  to  learn  or  carry  out  a  task  (Como,  2001).  Per¬ 
sistence  and  sustained  interest  in  knowledge  genera¬ 
tion  process  is  one  essential  component  of  the 
motivation  process  of  this  model.  Interest  can  be 
enhanced  only  when  the  learners  attribute  successful 
comprehension  to  their  own  effort  at  knowledge  gen¬ 
eration  (Wittrock,  1991). 

Learners  who  are  motivated  to  generate  meaning 
between  their  memory  and  new  information  need  to 
use  various  learning  strategies  from  simple  coding  to 
integration  strategies.  Depending  on  their  motivation 
level  or  memory,  such  as  prior  learning  on  domain  or 
learning  strategy,  or  learners’  preference,  learners 
employ  different  learning  strategies  in  knowledge  gen¬ 
eration.  This  process  can  be  explained  by  information¬ 
processing  theory:  the  process  of  thinking  and  memory 
storage  — in  other  words,  the  stages  and  levels  of  pro¬ 
cessing.  What  we  take  from  information-processing 
theory  is  an  emphasis  on  how  we  think,  rather  than  on 
what  we  think  or  that  we  think.  Its  focus  is  on  that 
process  of  transforming  external  stimuli  into  some 
recallable  form  to  be  stored  in  memory.  The  emphasis 
of  generative  learning  theory  is  on  the  generation  of 
new  conceptual  understandings,  not  just  on  transfer¬ 
ring  information. 

Finally,  Wittrock  (1991)  emphasized  that  learners 
should  control  their  own  generative  processes.  Meta¬ 
cognition  regulates  one’s  cognitive  activities  in  learn¬ 
ing  processes  and  therefore  surrounds  the  three  gen¬ 
erative  learning  processes  shown  in  Figure  10.1 
(Brown,  1978;  Flavell,  1979).  Self-monitoring  is  a 
vital  process  here  because  it  informs  learners  about 
their  progress  (Zimmerman,  1998).  Based  on  self¬ 
monitoring,  learners  manage  their  effort  and  available 
resources  and  change  their  learning  strategies  to  gen¬ 
erate  meaning. 

In  summary,  to  make  meaning,  learners  actually 
create  relationships  among  or  between  their  memory 
and  new  information.  Learners  are  mentally  active 
and  use  various  learning  strategies  in  this  knowledge 
generation  process.  Also,  learners  metacognitively 
self-regulate  this  process.  The  outcome  of  this  knowl¬ 
edge  generation  was  originally  investigated  in  reading 
comprehension,  and  many  researchers  extended  this 
model  to  investigate  a  variety  of  generative  learning 
strategies  that  were  expected  to  promote  different  lev¬ 
els  of  learning  in  a  variety  of  domains.  Recently, 
researchers  have  explored  higher  order  thinking  and 
self-regulated  learning  skills  as  outcomes  of  genera¬ 
tive  learning. 


APPLIED  RESEARCH 

Synthesis  from  the 
Learning  Process  Perspective 

Learning  strategies,  as  illustrated  in  Figure  10.1,  that 
are  employed  to  generate  knowledge  and  meaning  dur¬ 
ing  the  learning  process  are  the  most  frequently  studied 
component  of  generative  learning  theory.  For  example, 
many  research  studies  have  tested  the  effects  of  simple 
coding  strategies  such  as  underlining,  note  taking,  and 
adjunct  or  inserted  questions;  more  complex  coding 
organizational  strategies,  such  as  the  creation  of  hier¬ 
archies,  headings,  summaries,  and  concept  maps  or 
manipulation  of  objects;  and,  finally,  elaborative  inte¬ 
gration  strategies  such  as  imaging  and  creation  of 
examples,  interpretations,  or  analogies  (Table  10.1). 

An  analysis  was  conducted  from  this  strategy  per¬ 
spective  in  the  first  two  editions  of  this  chapter.  From 
this  perspective,  three  conclusions  could  be  drawn. 
First,  when  comparing  generative  strategies  treatment 
and  control  groups,  researchers  often  found  no  signif¬ 
icant  differences,  indicating  that  mental  activity  cannot 
be  strictly  controlled  by  instruction,  and  this  finding 
raised  the  issue  that  requiring  an  overt  response  may 
be  more  effective  in  encouraging  the  desired  result 
(Anderson  and  Kulhavey,  1972;  Peper  and  Mayer, 
1986;  Shrager  and  Mayer,  1989). 

Second,  investigation  of  the  subject-generated  vs. 
experimenter-provided  activity  is  a  relatively  common 
theme  in  generative  learning  studies  (see,  for  example, 
Barnett  et  al.,  1981;  Bull  and  Wittrock,  1973;  Doc- 
torow  et  al.,  1978;  Rickards  and  August,  1975;  Smith 
and  Dwyer,  1995;  Stein  and  Bransford,  1979).  These 
research  results  are  mixed  with  learning  gains  not  con¬ 
sistently  predicted  by  this  factor. 

Third,  comparing  the  various  types  of  generative 
strategies  is  another  major  research  theme  (Hooper  et 
al.,  1994;  McKeague  and  DiVesta,  1996;  Wittrock  and 
Alesandrini,  1990).  Again,  looking  at  generative  learn¬ 
ing  from  this  perspective  did  not  result  in  consistent 
or  definitive  guidelines  for  its  use.  Although  some  of 
the  research  has  found  some  generative  learning  strat¬ 
egies  to  be  effective,  the  results  are  mixed  when  exam¬ 
ining  them  by  learning  strategy  type,  and  they  warrant 
a  different  type  of  analysis. 

Synthesis  from  a  Learning 
Outcomes  Perspective 

Four  categories  of  criterion  measures  emerged  when 
examining  research  on  generative  learning  strategies 
from  a  learning  outcomes  perspective:  recall,  compre¬ 
hension,  higher  order  thinking,  and  self-regulation  skill. 
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Recall 

According  to  the  research  summarized  in  Table  10.1, 
recall  is  the  most  frequently  studied  criterion  measure. 
Wittrock  and  Carter  (1975)  conducted  a  study  exam¬ 
ining  free  recall  in  generative  vs.  reproductive  treat¬ 
ments.  The  generative  group  was  directed  to  organize 
the  hierarchies,  whereas  the  reproductive  group  was 
directed  to  simply  copy  them.  The  results  showed  bet¬ 
ter  performance  for  the  generative  treatment  groups 
than  for  the  reproductive  groups  for  the  disorganized 
and  randomly  organized  hierarchies;  however,  the 
organized  reproductive  group  performed  better  on  free 
recall  than  the  unrelated  generative  group.  Burton  et 
al.  (1986)  investigated  the  effect  of  superordinate  and 
subordinate  questions  by  using  a  secondary  task  probe 
technique  and  a  passage  about  a  mythical  country.  The 
overall  results  indicated  that  more  main  ideas  were 
recalled  than  details. 

Davis  and  Hult  (1997)  studied  immediate  and 
delayed  free  recall  in  note  taking  and  writing  summa¬ 
ries  in  introductory  psychology  classes.  Their  results 
support  the  findings  of  Barnett  et  al.  (1981)  that  writ¬ 
ing  summaries  during  pauses  in  the  lecture  note-taking 
activity  significantly  improved  free  recall  and  delayed 
retention.  King  (1992)  examined  the  effect  of  self¬ 
questioning,  summarizing,  and  note  taking  on  imme¬ 
diate  and  delayed  recall  of  under-prepared  college  stu¬ 
dents.  On  immediate  recall,  summarizers  performed 
better  than  self-questioners,  who  were  better  than  note 
takers,  indicating  a  progressive  generative  effect.  Self¬ 
questioners  performed  best  on  the  delayed  tests,  indi¬ 
cating  that  deeper  processing  may  occur  in  more  gen¬ 
erative  tasks  such  as  self-questioning.  Woods  and  Ber¬ 
nard  (1987)  also  found  effects  of  adjunct  conceptual 
post-questions  on  encouraging  a  greater  depth  of  pro¬ 
cessing  of  verbal  information  among  adults  60  years 
of  age  and  older.  From  results  on  intentional  and  inci¬ 
dental  free -recall  tests,  they  found  that  adjunct  ques¬ 
tions  helped  older  learners  process  only  intentional 
text  at  a  greater  depth.  Shrager  and  Mayer  (1989) 
reported  interesting  results  regarding  aptitude-treat¬ 
ment  interaction  research.  Note  takers  recalled  more 
than  non-note  takers  for  students  with  low  levels  of 
prior  knowledge  but  not  for  those  with  high  levels  of 
prior  knowledge. 

To  summarize  these  findings,  recall  has  been  often 
used  as  dependent  variable  in  the  studies  for  simple 
coding  strategies.  In  addition,  some  researchers 
reported  that  the  higher  the  level  of  generativity,  the 
higher  the  score  on  delayed  recall.  Research  findings, 
however,  are  mixed  with  regard  to  the  type  of  strate¬ 
gies,  measurement  instrument,  and  the  interaction  with 
individual  differences. 


Comprehension 

In  two  experiments  with  elementary-school  children, 
Doctorow  et  al.  (1978)  studied  the  effect  of  learner-gen¬ 
erated  vs.  experimenter-provided  paragraph  headings 
and  sentence  meanings  on  comprehension.  The  combi¬ 
nation  of  text  organized  through  the  use  of  headings  plus 
learner-generated  sentences  about  the  paragraphs  pro¬ 
duced  dramatic  gains  in  reading  comprehension.  This 
strategy  also  increased  comprehension  more  for  high- 
ability  students  than  for  low-ability  students,  perhaps 
because  high-ability  students  have  better  organizational 
cognitive  abilities  to  make  sense  out  of  disorganized 
information.  Two  studies  tested  the  effect  of  a  combina¬ 
tion  of  generative  strategies  on  comprehension.  The 
treatments  of  Camine  and  Kinder  (1985)  study  were 
images, corrective  feedback,  and  summary  to  study  read¬ 
ing  comprehension.  They  found  significant  gains  from 
pre-  to  post-tests  for  both  narrative  and  expository  text. 
Linden  and  Wittrock  (1981)  also  tested  the  effect  of 
images,  summary  sentences,  and  analogies,  all  of  which 
increased  and  correlated  with  comprehension. 

Several  studies  have  tested  the  effect  of  concept 
mapping  on  comprehension.  Smith  and  Dwyer  (1995) 
found  a  significant  difference  only  on  lower  level  ter¬ 
minology  tasks,  not  on  a  comprehension  task,  in  favor 
of  instructor-provided  maps .  Similarly,  two  other  studies 
(Kenny,  1995;  Taricani,  2002)  found  no  effects  of 
learner-generated  graphic  organizers  or  concept  maps 
on  comprehension.  In  the  Taricani  (2002)  study,  feed¬ 
back  was  also  tested  against  generativity  to  evaluate  the 
notion  that  learner  generation  can  create  misconceptions 
that  are  corrected  before  testing;  however,  the  blending 
of  concept  mapping  and  feedback  did  not  assist  in  over¬ 
all  learning  of  terminology  and  comprehension.  Wang 
(2003)  also  tested  the  effect  of  different  types  of  concept 
mapping  strategies  on  terminology  and  comprehension. 
Students  who  were  given  partially  predeveloped  concept 
maps  performed  better  on  all  levels  of  learning  outcome, 
but  students  who  created  their  own  concept  map  groups 
performed  better  only  on  terminology. 

In  summary,  the  effect  of  complex  coding  strate¬ 
gies  on  comprehension  is  still  mixed,  but  simple  cod¬ 
ing  strategies  are  consistently  reported  to  be  effective 
for  aiding  comprehension.  Higher  levels  of  generativ¬ 
ity  when  using  complex  coding  strategies  (e.g., 
learner-generated  concept  map)  did  not  seem  to  be 
effective  for  improving  comprehension. 

Higher  Order  Thinking 

Research  has  also  investigated  problem  solving,  rea¬ 
soning,  inference,  and  application.  Among  the 
research,  problem  solving  has  been  tested  by  a  few 
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significantly  better  than  those  who  used  the  manipulatives 
in  teams.  The  opposite  effect  was  found  for  intermediate 
recall. 


Combination  of  coding  and  integration 
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Comprehension  and  Organizational  strategies 

higher-order  Concept  vs.  semantic  maps  Beissner  et  at.  (1993)  Human  heart  Undergraduates  Learner-generated  concept  maps  were  the  better  strategy 

thinking  skill  for  holists. 

Learner-generated  semantic  maps  were  better  for  serialists 
for  problem-solving  learning  only. 
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studies.  In  research  comparing  note  takers  and  non¬ 
note  takers  who  were  learning  how  to  use  a  camera, 
Shrager  and  Mayer  (1989)  concluded  that  note  takers 
with  low  prior  knowledge  solved  transfer  problems 
better  than  non-note  takers  with  low  prior  knowledge. 
Barab  et  al.  (1999)  also  reported  that  students  in  the 
generative  activity  group  performed  better  in  problem 
solving  in  a  computer-based  learning  environment. 
Regarding  organizational  strategy,  Beissner  et  al. 
(1993)  compared  the  effects  of  learner-generated  con¬ 
cept  maps  and  serial  maps  with  learner  differences  at 
four  levels  of  learning.  Their  findings  showed  an  inter¬ 
action  between  concept  vs.  semantic  maps  and  seri- 
alist  or  holist  learners  on  the  problem-solving  ques¬ 
tions  only,  with  serialists  performing  better  with 
semantic  maps  and  holists  performing  better  with  con¬ 
cept  maps.  In  a  more  recent  study,  Lee  and  Nelson 
(2005)  compared  the  effect  of  learner- generated  con¬ 
cept  maps  and  instructor-generated  concept  maps  on 
problem  solving.  Research  results  showed  that  the 
learner-generated  concept  map  group  outperformed 
the  instructor-generated  map  group  in  well-structured 
problem  solving.  They  also  reported  an  interaction 
effect  with  learners’  prior  knowledge:  The  high  prior 
knowledge  group  benefited  more  from  the  activity  of 
concept  map  generation  than  the  low  prior  knowledge 
group. 

On  the  other  hand,  Laney  (1990)  and  Sutliff  (1986) 
examined  the  effect  of  generative  strategies  on  reason¬ 
ing  and  inference  respectively.  In  Laney ’s  study,  the 
verbal-only  strategy  and  the  integration  of  verbal  and 
imaging  strategy  were  more  effective  than  the  imag- 
ing-only  strategy  on  the  economic  reasoning  of  third- 
graders.  He  felt  that  his  results  were  consistent  with 
Wittrock’s  notion  that  the  effective  use  of  imaging  is 
developmental.  Laney’s  third-grade  subjects  had  not 
yet  developed  this  ability  and  were  more  familiar  with 
verbal  instruction.  Sutliff  investigated  the  effect  of 
inserted  questions  on  reducing  passivity  in  a  self- 
instructional  slide-tape  presentation  as  evidenced  by 
increased  learning  of  facts  and  inference.  He  reported 
that  there  were  no  significant  differences  between 
groups,  which  is  in  contrast  to  prior  research  reporting 
significantly  different  effects  of  questions. 

In  sum,  most  of  the  studies  investigating  higher 
order  thinking  skills  dealt  with  complex  coding  strat¬ 
egies  such  as  organization.  Because  it  has  been  only 
a  decade  since  we  have  begun  investigating  this  topic 
with  regard  to  generative  learning,  the  research  find¬ 
ings  have  not  collectively  supported  each  other; 
however,  relatively  more  effort  has  been  directed 
toward  aptitude-treatment  interaction  research  by 
considering  learners’  prior  knowledge  level  or  learn¬ 
ing  style. 


Self-Regulation  Skill 

In  recent  years,  there  has  been  a  resurgence  of  interest 
in  self-regulated  learning  (SRL).  This  phenomenon  can 
be  explained  from  two  perspectives.  First,  recent 
research  findings  support  a  positive  relationship 
between  students’  self-regulating  skill  and  their  aca¬ 
demic  achievement  (Azevedo  and  Cromley,  2004;  Kra- 
marski  and  Gutman,  2006;  Pintrich  and  De  Groot, 
1990;  Zimmerman,  1998;  Zimmerman  and  Schunk, 
2001).  Second,  a  major  goal  of  education  is  to  train 
students  to  be  skillful  self-regulators  to  enable  them  to 
be  life-long  learners  (Boekaerts,  1997;  Boekaerts  and 
Corno,  2005).  “Self-regulated  learners  are  behavior- 
ally,  motivationally,  and  metacognitively  active  partic¬ 
ipants  in  their  own  learning  process”  (Zimmerman, 
1986).  This  interpretation  is  in  line  with  Wittrock’s 
emphasis  on  learners’  motivation,  cognitive  learning 
strategy,  and  metacognitive  process  in  the  knowledge 
generation  processes.  In  other  words,  self-regulation 
skills  may  play  a  vital  role  in  generative  learning,  but 
learners  can  also  develop  their  self-regulation  skills 
through  generative  learning  activities.  Chularut  and 
DeBacker  (2004)  investigated  the  effect  of  a  generative 
learning  strategy  — concept  mapping— on  students’ 
achievement  and  self-regulation  and  self-efficacy  in 
learning  English  as  a  second  language.  They  found  that 
students  who  were  using  concept  mapping  had  signif¬ 
icantly  greater  achievement  gains  at  post-test  compared 
to  pre-test.  In  addition  to  the  benefit  of  achievement, 
their  results  showed  that  a  positive  effect  of  engaging 
in  concept  mapping  increased  students’  self-regulation 
and  self-efficacy  relative  to  the  control  group. 

Summary 

A  variety  of  studies  reporting  on  results  of  generative 
strategies  have  been  summarized  here.  This  section  is 
not  intended  to  be  exhaustive;  rather,  the  studies  have 
been  selected  as  representative  of  the  kind  of  research 
that  has  been  conducted  across  content  areas ,  learning 
types,  and  age  levels;  however,  all  articles  that  could 
be  found  that  specify  generative  learning  as  the  theory 
being  tested  are  included.  In  general,  results  have 
shown  some  increased  gains  in  recall,  comprehension, 
and  higher  order  thinking  skill  as  well  as  improvement 
in  self-regulated  learning  skill  when  the  learner  is  an 
active  partner  vs.  a  passive  participant  in  the  learning 
process  and  when  instruction  includes  activities  that 
relate  new  information  together  and  new  information 
to  prior  knowledge.  These  studies  on  generative  learn¬ 
ing  have  shown  that,  in  many  cases,  active  learner 
involvement  produced  increased  learning;  that  is, 
learner-generated  activities  have  resulted  in  significant 
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gains  in  learning,  although  misconceptions,  feedback, 
and  developmental  appropriateness  require  further 
investigation,  and  there  remain  some  mixed  findings 
based  on  generative  learning  strategies. 


IMPLICATIONS  FOR 
FURTHER  RESEARCH 

Past  research  verifies  Wittrock’s  basic  premise  of 
active  learner  engagement;  however,  further  research 
should  explore  the  interrelationship  of  the  four  com¬ 
ponents  as  shown  in  Figure  10.1  (motivation,  learning 
strategy,  knowledge  generation,  metacognition)  in 
making  and  applying  meaning  to  predict  higher  recall, 
greater  comprehension,  better  higher  order  thinking, 
and  more  controlled  self-regulation.  What  appear  to  be 
weak  or  inconsistent  results  in  previous  research  may 
be  strengthened  if  all  components  are  taken  into 
account.  Two  such  areas  of  research  are  proposed. 

Motivation,  Learner,  and 
Knowledge  Creation  Processes 

Identifying  strategies  that  will  enhance  the  perception 
of  learner  responsibility  is  one  example  of  studying  the 
interrelationships  of  the  components.  This  indicates  a 
need  to  merge  the  learner  control  research  with  that  of 
generative  learning  to  address  such  questions  as: 

•  What  are  the  best  methods  for  providing 
advisory  feedback  on  learner-generated  con¬ 
ceptions  of  the  instruction  content,  and  what 
are  their  effects? 

•  What  is  the  effect  on  learning  of  directive, 
embedded,  or  inductive  control  when  moti¬ 
vation  level  varies? 

Instructor-Provided  or  Learner- 
Generated  and  Self-Regulation? 

In  addition,  a  question  remains  as  to  the  relationship 
between  generative  learning  and  learners’  self-regula- 
tion.  From  Wittrock’s  definition,  learners  need  to  use 
their  self-regulation  skill  during  the  knowledge  gener¬ 
ation  process,  and  consequently  the  learners  may 
increase  not  only  their  comprehension  but  also  their 
self-regulation  skill.  Self-regulation  seems  to  be  of  crit¬ 
ical  importance  when  learners  are  faced  with  generating 
their  own  understanding  rather  than  having  the  instruc¬ 
tor  providing  understanding,  especially  where  miscon¬ 
ceptions  are  possible.  These  relationships  should  be 
empirically  tested  by  addressing  such  questions  as: 


•  Is  there  any  interaction  between  a  learner’s 
level  of  self-regulation  skill  and  generative 
activities? 

•  What  is  the  effect  of  generative  learning 
strategies  on  a  learner’s  self-regulated  learn¬ 
ing  skill? 

FINAL  THOUGHTS 

The  principles  behind  generative  learning  offer  the 
instructional  designer  much  guidance  for  developing 
environments  that  engage  the  learner  in  active  pro¬ 
cessing  of  the  information  in  face-to-face,  e-learning, 
or  even  informal  learning  environments.  Following 
Wittrock’s  principles,  one  should  put  the  control  of 
learning  in  the  hands  of  the  learner  by  creating  an 
advisory  environment  in  which  learners  manipulate 
information  by  moving  text,  graphics,  and  media 
around  mentally  or  physically,  testing  their  own 
understanding  of  the  relationships  they  are  building. 
This  means  putting  learners  in  an  environment  in 
which  success  can  be  guided,  rewarded,  and  rein¬ 
forced.  Generative  learning  theory  is  not  discovery 
learning  but  student-centric  learning  with  specified 
activities  for  actively  constructing  meaning.  Genera¬ 
tive  learning  activities  require  internal  processing  of 
external  stimuli.  A  generative  learning  environment 
is  not  limited  to  open-ended  resources,  although  it 
could  engender  those,  and  it  includes  carefully 
crafted  external  stimuli  that  are  ready  for  individual 
processing.  Generative  activities  are  what  exist 
between  the  external  stimuli  and  the  learner.  Gener¬ 
ative  learning  theory  does  not  assume  dominance  of 
the  role  of  the  learner  or  instructor  or  instruction  but 
rather  a  partnership  in  the  process. 

As  a  practical  cousin  to  constructivism  and  a  more 
complete  second  cousin  to  behaviorism,  generative 
learning  theory  is  easily  applied  to  any  learning  or 
instructional  setting.  The  subtle  differences  between 
this  theory  and  other  theories  account  for  differences 
in  instructor  roles  (the  role  of  the  sage).  For  behavior¬ 
ism,  the  sage  is  on  the  stage.  For  constructivism,  the 
sage  is  viewed  as  a  guide.  For  generative  learning  the¬ 
ory,  the  sage,  guide ,  and  learner  are  in  the  center. 
Content,  instructional  expertise,  and  instructional  inten¬ 
tion  are  expected  of  the  sage  and  guide.  Active  engage¬ 
ment,  attention,  and  learning  intention  are  also  expected 
of  the  learner.  Much  research  must  yet  be  done  to 
support  this  position,  and  there  is  much  research  left  to 
do  to  establish  specific  guidelines  that  help  the  designer 
create  a  learning  environment  that  stimulates  attention 
and  intention,  promotes  active  mental  processing  at  all 
stages  and  levels  of  learning,  and  provides  appropriate 
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support  in  the  generation  process.  The  evidence  indi¬ 
cates  that  generative  learning  theory  is  very  applicable 
to  instructional  design  and  that  research  defining  types 
of  processing  should  continue. 
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ABSTRACT 

Modem  information  technologies  increase  the  range 
of  feedback  strategies  that  can  be  implemented  in  com¬ 
puter-based  learning  environments;  however,  the 
design  and  implementation  of  feedback  strategies  are 
very  complex  tasks  that  are  often  based  more  on  intu¬ 
ition  than  on  psychologically  sound  design  principles. 
The  purpose  of  this  chapter  is  to  present  theoretically 
and  empirically  based  guidelines  for  the  design  and 
evaluation  of  feedback  strategies.  To  this  end,  this 
chapter  describes  an  interactive,  two-feedback- loop 
model  that  explains  core  factors  and  effects  of  feed¬ 
back  in  interactive  instruction  (Narciss,  2006).  Based 
on  these  theoretical  considerations,  a  multidimensional 
view  of  designing  and  evaluating  multiples  feedback 
strategies  under  multiple  individual  and  situational 
conditions  is  presented.  This  multidimensional  view 
integrates  recommendations  of  prior  research  on  elab¬ 
orated  feedback  (Schimmel,  1988;  Smith  and  Ragan, 
1993),  task  analyses  (Jonassen  et  al.,  1999),  error  anal¬ 
yses  (VanLehn,  1990),  and  tutoring  techniques  (Mc- 
Kendree,  1990;  Merrill  et  al.,  1992). 


KEYWORDS 

Cybernetics:  System  theory  concerned  with  the  issues 
of  regulation,  order,  and  stability  confronting  us  in 
the  treatment  of  complex  systems  and  processes. 

Feedback:  Output  of  a  system  that  is  fed  back  to  the 
controller  of  the  system  as  an  input  signal  to  reg¬ 
ulate  the  system  with  regard  to  a  reference  value 
(cybernetic  definition);  post-response  information 
that  is  provided  to  learners  to  inform  them  of  their 
actual  state  of  learning  or  performance  (instruc¬ 
tional  context). 

Informative  tutoring  feedback:  Multiple-try  feedback 
strategies  providing  elaborated  feedback  compo¬ 
nents  that  guide  the  learner  toward  successful  task 
completion  without  offering  immediately  the  cor¬ 
rect  response. 

Interactive  learning  task:  Tasks  providing  multiple 
response  steps  or  tries  and  instructional  compo¬ 
nents  such  as  feedback,  guiding  questions, 
prompts,  simulation  facilities,  and  so  on. 


INTRODUCTION 

For  almost  a  century  researchers  have  investigated  the 
factors  and  effects  of  feedback  involved  in  instruc¬ 
tional  contexts;  consequently,  the  body  of  feedback 
research  is  very  large.  This  large  body  of  feedback 


research  has  been  examined  and  revisited  extensively 
by  Edna  Mory  in  previous  editions  of  this  Handbook 
(Mory,  1996,  2004).  As  space  is  restricted,  the  body 
of  feedback  research  that  was  included  in  these  previ¬ 
ous  reviews  will  be  not  revisited  in  detail  here,  but  the 
insights  of  this  research  will  be  organized  and  outlined 
on  the  basis  of  a  conceptual  framework  for  designing 
and  evaluating  feedback  for  interactive  learning  tasks. 
To  introduce  this  conceptual  framework,  definitions  of 
the  term  feedback  will  be  discussed  first. 

FEEDBACK  IN  INSTRUCTIONAL 
CONTEXTS:  DEFINITION 

The  term  feedback  is  a  widely  used  concept  in  many 
technical  and  scientific  domains  (e.g.,  economics,  elec¬ 
tronics,  biology,  medicine,  psychology).  The  concept 
of  feedback  is  derived  from  cybernetics  (Wiener, 
1954),  which  is  concerned  with  the  control  of  sys¬ 
tems— that  is,  with  issues  of  regulation,  order,  and 
stability  that  arise  in  the  context  of  complex  systems 
and  processes.  In  cybernetics,  feedback  refers  to  the 
output  of  a  system  that  is  fed  back  to  the  controller  of 
the  system  as  an  input  signal.  This  input/feedback  sig¬ 
nal  closes  the  feedback  loop  and,  in  combination  with 
an  externally  defined  reference  value,  controls  the  sys¬ 
tem.  In  addition  to  the  reference  value  and  the  feedback 
signal,  the  controller  and  the  variable  to  be  controlled 
are  key  elements.  The  controller  stores  the  reference 
value,  compares  it  with  the  current  actual  value,  and, 
on  the  basis  of  this  comparison,  assesses  what  correc¬ 
tion  is  required;  hence,  the  effects  of  a  feedback  signal 
depend  not  only  on  this  feedback  signal  but  also  all 
the  other  functional  elements  of  the  causal  loop. 

Since  the  development  of  Thorndike’s  (1913)  law 
of  effect,  it  has  become  well  established  in  psychology 
that  the  consequences  of  a  behavior  may  influence  the 
rate  and  intensity  of  that  behavior  in  future  situations. 
In  the  domain  of  learning  and  instruction,  feedback 
has  been  considered  to  be  either  a  fundamental  prin¬ 
ciple  for  efficient  learning  (Andre,  1997;  Bilodeau, 
1969;  Bloom,  1976;  Fitts,  1962;  Taylor,  1987)  or  at 
least  as  an  important  element  of  instruction  (Collies  et 
al.,  2001;  Dick  et  al.,2001). 

Some  instructional  researchers  consider  feedback 
in  instructional  contexts  to  be  any  type  of  information 
that  is  provided  to  learners  after  they  have  responded 
to  a  learning  task  (Wager  and  Wager,  1985).  This 
notion  of  feedback  is  far  too  large  because  of  the  large 
variety  of  post-response  information,  and  it  does  not 
include  the  idea  that  the  information  is  presented  with 
the  purpose  of  allowing  the  learner  to  compare  his  or 
her  actual  outcome  with  a  desired  outcome  to  regulate 
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or  control  the  next  attempt  with  this  learning  task. 
Experimental  researchers  thus  use  a  more  limited 
notion  of  feedback.  They  use  the  term  informative 
feedback  to  refer  to  all  post-response  stimuli  that  are 
provided  to  a  learner  by  an  external  source  of  infor¬ 
mation,  according  to  experimentally  defined  rules  and 
conditions,  to  inform  the  learner  on  his  or  her  actual 
state  of  learning  or  performance  (Annett,  1969;  Bilo¬ 
deau,  1969;  Holding,  1965). 

According  to  the  cybernetic  and  experimental  defi¬ 
nitions,  a  general  definition  for  feedback  in  instructional 
contexts  might  be  as  follows:  Feedback  is  all  post¬ 
response  information  that  is  provided  to  a  learner  to 
inform  the  learner  on  his  or  her  actual  state  of  learning 
or  performance.  In  instructional  contexts,  this  definition 
of  feedback  requires  the  differentiation  among  feedback 
presented  by  an  external  source  of  information  and  feed¬ 
back  provided  by  internal  sources  of  information  (i.e., 
information  directly  perceivable  by  the  learner  while 
task  processing,  such  as  proprioceptive  information 
when  performing  a  pointing  task).  This  differentiation 
is  particularly  important  from  a  methodological  point 
of  view;  consequently,  in  early  experimental  feedback 
studies  researchers  tried  to  eliminate  or  control  internal 
sources  of  feedback  to  investigate  the  effects  of  external 
feedback  on  learning  and  performance  (for  a  review,  see 
Bilodeau,  1969).  The  differentiation  among  external  and 
internal  feedback  is  also  crucial  if  one  investigates  the 
effects  of  feedback  on  the  basis  of  recent  instructional 
models  viewing  the  process  of  knowledge  acquisition 
as  a  process  of  active  knowledge  construction  and  com¬ 
munication  (Jonassen,  1999)  or  as  a  self-regulated  learn¬ 
ing  process  (Butler  and  Winne,  1995).  This  differentia¬ 
tion  should  be  kept  in  mind  when  revisiting  feedback 
research  and  considering  feedback  strategies. 

External  feedback  may  confirm  or  complement  the 
internal  feedback,  or  it  may  contradict  the  internal 
feedback.  The  latter  case  raises  at  least  three  questions: 

•  How  do  learners  treat  or  cope  with  the  dis¬ 
crepancy  between  internal  and  external  feed¬ 
back? 

•  What  individual  and  situational  factors  con¬ 
tribute  to  a  discrepancy  between  external  and 
internal  feedback? 

•  How  can  we  design  and  evaluate  feedback 
strategies  that  support  learners  regulating 
their  learning  process  successfully  if  there  is 
a  discrepancy  between  internal  and  external 
feedback? 

The  first  question  has  been  addressed  implicitly  by 
the  response  certitude  model  of  Kulhavy  and  his  col¬ 
laborators  (Kulhavy  and  Stock,  1989;  Kulhavy  et  al.. 


1990a,b;  Stock  et  al.,  1992)  and  by  the  five-stage 
model  of  mindful  feedback  processing  (Bangert- 
Drowns  et  al.,  1991).  Furthermore,  it  was  explicitly 
the  focus  of  Butler  and  Winne’s  theoretical  synthesis 
regarding  feedback  and  self-regulated  learning  (Butler 
and  Winne,  1995).  These  models  have  been  described 
and  discussed  in  detail  in  Mory’s  prior  reviews  (Mory, 
1996,2004). 

The  second  question  has  been  answered  indirectly 
as  a  result  of  meta-analyses  that  showed  that  external 
feedback  effects  are  not  always  positive  and  thus  tried 
to  identify  possible  moderators  for  the  efficiency  of 
external  feedback  (Bangert-Drowns  et  al.,  1991; 
Kluger  and  DeNisi,  1996).  The  insights  of  these  meta¬ 
analyses  are  integrated  in  the  conceptual  framework 
elaborated  below. 

The  third  question,  one  of  the  most  crucial  ques¬ 
tions  for  instructional  design  and  practice,  has  been  in 
part  addressed  by  researchers  developing  and  evaluat¬ 
ing  intelligent  tutoring  systems  (ITSs).  Detailed  reviews 
of  the  insights  of  ITS  research  are  provided  by  Ander¬ 
son  et  al.  (1995)  and  VanLehn  et  al.  (2005);  see  also 
Chapters  24  and  27  in  this  Handbook.  Core  issues  and 
insights  from  prior  research  with  regard  to  this  ques¬ 
tion  are  discussed  below. 

A  CONCEPTUAL  FRAMEWORK 
FOR  FEEDBACK  IN 
INTERACTIVE  INSTRUCTION 

This  section  focuses  on  feedback  for  interactive  (com¬ 
puter-based)  learning  tasks  that  is  provided  by  an  exter¬ 
nal  source  of  information  (e.g.,  an  instructional  pro¬ 
gram,  a  teacher)  to  contribute  to  the  regulation  of  the 
learning  process  in  such  a  way  that  learners  acquire  the 
knowledge  and  skills  required  to  master  these  tasks.  As 
elaborated  in  the  next  sections,  internal  feedback  is 
considered  an  important  factor  for  treating  the  infor¬ 
mation  provided  by  the  external  feedback.  Conceptu¬ 
alizing  feedback  as  an  instructional  activity  that  aims 
at  contributing  to  the  regulation  of  a  learning  process 
makes  it  possible  to  use  the  core  insights  provided  by 
models  of  instruction  and  self-regulated  learning 
(Bloom,  1976;  Boekaerts,  1996;  Carroll,  1963)  to  ana¬ 
lyze  possible  factors  and  effects  of  informative  feed¬ 
back.  Instructional  models  are  based  on  the  assumption 
that  the  effects  an  instructional  activity  can  have  are 
determined  by  the  quality  of  the  instructional  activity 
(e.g.,  scope,  nature,  and  structure  of  the  information 
provided  and  form  of  presentation),  individual  learning 
prerequisites  (e.g.,  previous  knowledge,  metacognitive 
strategies,  motivational  dispositions,  and  strategies), 
and  situational  factors  in  the  instructional  setting 
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Figure  11.1  Overview  of  the  components  of  the  ITFL  model.  (From  Narciss,  S.,  Informatives  tutorielles  Feedback.  Entwicklungs- 
und  Evaluationsprinzipien  auf  der  Basis  instruktionspsychologischer  Erkenntnisse  (Informative  Tutoring  Feedback ),  Waxmann, 
Munster,  2006.  With  permission.) 


(instructional  goals,  learning  content,  and  tasks).  The 
current  conceptual  framework  links  these  issues  with 
systems  theory  and  attempts  to  integrate  findings  from 
systems  theory  with  recommendations  from  prior 
research  on  elaborated  feedback  (Schimmel,  1988; 
Smith  and  Ragan,  1993),  on  task  analysis  (Jonassen  et 
al.,  1999),  on  error  analysis  (VanLehn,  1990),  and  on 
tutoring  techniques  (Anderson  et  al.,  1995;  McKendree, 
1990;  Merrill  et  al.,  1992;  VanLehn  et  al.,  2005). 

Basic  Assumptions 

The  basic  components  of  a  generic  feedback  loop  serve 
as  the  starting  point  for  formulating  a  feedback  model 
with  two  interacting  feedback  loops:  the  interactive, 
two-feedback-loop  (ITFL)  model: 

•  Identification  or  definition  of  the  variables 
that  should  be  controlled 

•  Continuous  measurement  of  these  controlled 
variables  by  a  sensor 

•  Feedback  of  the  actual  values  of  the  con¬ 
trolled  variables  to  a  controller 

•  Reference  value  for  each  controlled  variable 
that  is  predefined  and  stored  in  the  controller 

•  Comparison  of  the  actual  values  of  the  con¬ 
trolled  variables  with  (predefined)  reference 
values  by  the  controller 


Note:  If  there  is  a  discrepancy  between  the  actual  and 
the  reference  value,  the  controller  must  transform  this 
discrepancy  into  a  control  action. 

•  Transmission  of  this  control  action  to  a  con¬ 
trol  element  (control  actuator) 

•  Execution  of  the  control  action  by  a  control 
actuator 

According  to  systems  theory,  the  control  actuator 
that  carries  out  the  control  actions,  the  controlled  vari¬ 
ables,  and  a  sensor  that  measures  the  controlled  vari¬ 
able  are  key  elements  of  the  controlled  process.  To 
regulate  the  controlled  process,  the  controller  requires 
the  reference  value,  the  actual  value  provided  by  feed¬ 
back,  and  comparison  and  transformation  procedures 
for  generating  the  control  actions . 

In  the  ITFL  model,  the  controlled  process  is 
defined  as  the  carrying  out  of  learning  tasks  or  the 
mastering  of  the  demands  associated  with  these  tasks. 
Building  on  models  of  self-regulated  learning  (Boe- 
kaerts,  1996)  as  well  as  the  approach  of  Butler  and 
Winne  (1995),  this  model  distinguishes  among  cogni¬ 
tive,  motivational,  and  metacognitive  demands  (see 
Figure  11.1).  Quantifiable  controlled  variables  for 
these  criteria  could  include  carefully  defined  and  oper¬ 
ationalized  cognitive,  metacognitive,  or  motivational 
indicators  of  mastery  of  particular  task  requirements . 
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When  regulatory  paradigms  from  systems  theory 
are  applied  to  an  instructional  context  containing  exter¬ 
nal  feedback,  two  interacting  feedback  loops  must  be 
considered:  (1)  an  internal  feedback  loop  that  pro¬ 
cesses  internal  feedback,  or  the  actual  values  to  which 
the  learner  has  direct  access  (e.g.,  confidence  in 
answers,  perceived  effort);  and  (2)  an  external  feed¬ 
back  loop  that  processes  the  actual  values  determined 
by  the  learning  medium  (e.g.,  the  instructor,  learning 
program,  experimenter). 

A  distinction  between  external  and  internal  feed¬ 
back  loops  means  that  it  is  also  necessary  to  differen¬ 
tiate  between  the  following  elements: 

•  Sensors— Internal  and  external  feedback 
loops  require  a  diagnostic  component  that 
registers  the  actual  values  of  the  controlled 
variables. 

•  Reference  values— Control  of  internal  and 
external  feedback  loops  can  only  be  carried 
out  on  the  basis  of  relevant  reference  values. 

In  the  ITFL  model,  it  is  assumed  that  internal 
reference  values  are  generated  on  the  basis  of 
a  subjective  representation  of  the  demands  of 
learning  tasks,  whereas  external  reference 
values  are  based  on  an  external  representation 
of  these  demands.  Subjective  task  represen¬ 
tations  are  mainly  governed  by  individual 
prerequisites  such  as  existing  knowledge, 
metacognitive  and  motivational  strategies, 
and  individual  learning  goals.  External  repre¬ 
sentations  of  task  demands  are  closely  related 
to  the  features  of  an  instructional  context, 
particularly  to  the  specific  instructional  goals. 

•  Controller— For  the  actual  values  registered 
by  the  internal  and  external  sensors  to  be 
processed,  each  requires  a  component  in 
which  reference  and  actual  values  can  be 
compared;  thus,  both  external  and  internal 
controllers  in  which  this  process  can  be  car¬ 
ried  out  are  needed. 

In  an  instructional  context  that  provides  external  infor¬ 
mative  feedback,  the  differentiations  made  in  the  ITFL 
model  lead  to  the  following  assumptions  regarding  the 
interaction  between  internal  and  external  feedback 
loops: 

•  The  starting  points  for  internal  and  external 
regulatory  processes  are  the  relevant  con¬ 
trolled  variables  for  the  particular  controlled 
process  (i.e.,  mastery  of  learning  task 
requirements). 


•  The  actual  value  of  the  controlled  variable 
or  variables  is  registered  by  both  the 
learner  and  by  an  external  actor  such  as  an 
instructor  or  a  computer-based  instruc¬ 
tional  system. 

•  External  actual  values  are  initially  pro¬ 
cessed  externally  in  the  external  controller 
of  the  teaching  medium.  The  external  ref¬ 
erence  value,  the  comparison  between  the 
reference  value  and  the  actual  value,  and 
the  externally  specified  rules  for  calculat¬ 
ing  the  correction  value  determine  the  ini¬ 
tial  value  of  the  external  controller.  This 
initial  value,  which  in  systems  theory 
would  be  referred  to  as  an  external  correc¬ 
tion  variable ,  is  fed  to  the  internal  control¬ 
ler  as  external  feedback. 

•  This  external  feedback  is  processed  in  the 
internal  controller  along  with  the  internal 
actual  value  (i.e.,  internal  feedback).  This 
means  that  several  comparisons  must  be  car¬ 
ried  out  by  the  internal  controller.  These 
include  comparisons  between: 

•  Internally  measured  actual  value  (internal 
feedback)  and  internal  reference  value 

•  External  feedback  and  internal  feedback 

•  External  feedback  and  internal  reference 
value 

•  From  these  comparison  processes  a  correc¬ 
tion  variable  (i.e.,  an  internal  correction 
variable)  must  be  generated.  The  learner’s 
main  task  in  this  case  is  to  locate  the  source 
of  any  discrepancies  that  are  detected 
between  these  various  values.  Such  dis¬ 
crepancies  can  occur  when,  for  example, 
internal  or  external  sensors  register  feed¬ 
back  values  inaccurately,  the  quality  of 
internal  or  external  feedback  is  poor,  or  the 
subjective  task  representation  is  incorrect 
or  imprecise  and  thus  leads  to  incorrect 
reference  values.  The  results  of  this  causal 
analysis  are  important  for  calculating  the 
internal  correction  variable.  This  means 
that  the  internal  correction  variable  is  the 
result  of  a  number  of  internal  processing 
procedures . 

•  The  internal  correction  variable  is  channeled 
to  the  first  stage  of  the  controlled  process  — 
the  control  element— where  it  serves  as  the 
basis  for  selection  and  activation  of  correc¬ 
tive  measures.  These  corrective  measures 
can  in  turn  have  an  impact  on  the  controlled 
variables. 
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Factors  Affecting  the  Efficiency 
of  External  Feedback 

The  assumptions  of  the  ITFL  model  lead  to  the  con¬ 
clusion  that  the  efficient  regulation  of  task  processing 
with  external  feedback  may  be  affected  by  factors  of 
both  the  internal  and  the  external  feedback  loops.  Both 
feedback  loops  contribute  to  the  regulation  of  the  same 
controlled  process,  which  is  characterized  by  the 
requirements  of  the  learning  tasks. 

Requirements  of  Learning  Tasks 
and  Instructional  Objectives 

As  mentioned  earlier,  the  starting  point  for  both  feed¬ 
back  loops  is  the  controlled  process,  which  can  be 
more  or  less  complex  depending  on  the  requirements 
of  the  learning  tasks  and  the  instructional  objectives. 
For  a  system  to  be  regulated  successfully,  it  is  crucial 
that  its  controlled  process  be  described  carefully  and 
precisely.  At  the  same  time,  it  is  necessary  to  define 
which  variables  will  serve  as  controlled  variables  that 
will  be  measured  and  regulated,  how  these  are  to  be 
measured,  and  the  procedures  through  which  correc¬ 
tions  are  to  be  carried  out.  In  instructional  contexts, 
this  involves  initially  analyzing  exactly  what  require¬ 
ments  are  associated  with  the  instructional  content, 
goals,  and  tasks.  Moreover,  to  select  corrective  mea¬ 
sures  for  the  regulation  of  controlled  variables,  the 
errors  and  difficulties  that  could  arise  in  connection 
with  mastering  task  requirements  must  also  be  identi¬ 
fied,  as  well  as  the  information  and  strategies  that  are 
necessary  to  eliminate  these  errors  or  difficulties. 

Instructional  content,  goals,  and  tasks  may  be  more 
or  less  complex  concerning  their  requirements. 
Bloom’s  revised  version  of  the  taxonomy  of  learning 
objectives  may  serve  as  a  basis  for  categorizing  task 
requirements  (Anderson  et  al.,  2001).  Analyzing  learn¬ 
ing  task  requirements  on  the  basis  of  this  taxonomy 
makes  it  clear  that  it  is  more  difficult  to  identify  pre¬ 
cisely  the  content-related,  cognitive,  metacognitive, 
and  motivational  requirements  for  complex  tasks  (i.e., 
those  that  require  higher  order,  content-related  knowl¬ 
edge  or  operations)  than  for  simple  tasks.  As  a  conse¬ 
quence,  one  may  assume  that  the  internal  and  external 
feedback  loops  might  function  less  efficiently  for  com¬ 
plex  learning  tasks.  The  notion  that  task  complexity 
affects  the  internal  feedback  loop  was,  for  example, 
identified  in  Mory’s  studies,  which  aimed  at  general¬ 
izing  Kulhavy  and  Stock’s  model  of  response  certitude 
(Mory,  1994,  1996,  2004).  Mory  found  that  for  higher 
order  learning  tasks,  students’  response  certitude  could 
not  be  used  as  a  reliable  measure  for  adapting  feed¬ 
back,  because  students  were  not  able  to  assess  their 


answers  to  these  tasks  correctly  (in  terms  of  the  ITFL 
model,  they  were  not  able  to  generate  a  reliable  internal 
feedback) . 

This  assumption  is  also  reflected  in  many  studies 
on  elaborated  feedback  that  were  conducted  to  inves¬ 
tigate  the  hypothesis  that  elaborated  feedback  is  more 
effective  with  more  complex  tasks;  however,  results  of 
these  studies  are  rather  mixed  (see  reviews  by  Azevedo 
and  Bernard,  1995;  Bangert-Drowns  et  al.,  1991; 
Mory,  1996,  2004).  Yet,  feedback  studies  that  devel¬ 
oped  elaborated  feedback  on  the  basis  of  thorough 
analyses  of  task  requirements  generally  found  the 
developed  elaborated  feedback  types  to  be  superior  to 
simple  outcome  feedback  (Birenbaum  and  Tatsuoka, 
1987;  Nagata,  1993,  1997;  Nagata  and  Swisher,  1995; 
Narciss,  2004,  2006;  Narciss  and  Huth,  2004,  2006). 
In  some  studies,  however,  with  very  complex  and  dif¬ 
ficult  tasks  or  with  serious  errors,  elaborated  feedback 
was  not  efficient  even  if  it  was  developed  on  the  basis 
of  task  analyses  (Birenbaum  and  Tatsuoka,  1987;  Clar- 
iana  and  Lee,  2001;  Nagata,  1997). 

Internal  Loop  Factors:  Prior  Knowledge,  Cognitive, 
Metacognitive,  and  Motivational  Skills 

According  to  the  ITFL  model,  the  learner’s  represen¬ 
tation  of  task  requirements,  the  learner’s  ability  to 
assess  his  or  her  responses  (quality  of  the  internal 
sensor),  the  learner’s  abilities  and  strategies  with 
regard  to  analyzing  and  comparing  internal  and  exter¬ 
nal  information  and  identifying  corrective  actions 
(quality  of  the  internal  controller),  and,  finally,  the 
learner’s  abilities  and  motivation  in  applying  these  cor¬ 
rective  actions  (quality  of  the  control  actuator)  are  core 
factors  contributing  to  the  efficiency  of  the  internal 
feedback  loop.  All  internal  factors  influence  the  exter¬ 
nal  feedback  loop  because  the  two  loops  interact. 

Subjective  Task  Representation:  Prior  Knowledge 

The  starting  point  for  processes  in  the  internal  control¬ 
ler  is  a  precise  definition  of  reference  values  of  con¬ 
trolled  variables.  These  reference  values  are  generated 
on  the  basis  of  how  learners  understand  and  represent 
the  requirements  of  the  learning  tasks.  Meaningful  ref¬ 
erence  values  can  only  be  generated  if  the  subjective 
representation  of  task  requirements  is  adequate. 
Whether  learners  are  able  to  represent  task  require¬ 
ments  adequately  and  precisely  depends  on  the  com¬ 
plexity  of  these  requirements  but  also  on  individual 
factors  such  as  prior  knowledge,  metacognitive  knowl¬ 
edge,  and  strategies  and  motivation.  How  much  indi¬ 
vidual  difference  in  subjective  task  representations 
affects  the  impact  of  feedback  on  learning  is  an  inter¬ 
esting  question  for  future  research. 
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Learners’  Self-Assessment  Skills 

Comparing  the  reference  values  with  the  actual  values 
of  controlled  variables  yields  meaningful  information 
only  if  the  actual  values  of  the  controlled  variables  are 
determined  accurately.  In  the  internal  loop,  this  depends 
a  great  deal  on  learners’  abilities  or  skills  in  assessing 
their  responses  and  performance  (Mory,  1996,  2004). 
Learners  must  identify  indicators  for  each  task  require¬ 
ment  that  can  help  them  evaluate  the  extent  to  which 
the  task  requirements  are  fulfilled.  How  external  feed¬ 
back  may  support  the  acquisition  of  self-assessment 
skills  is  another  interesting  issue  for  future  research. 

Learners’  Skills  and  Strategies 
in  Information  Processing 

To  generate  an  appropriate  control  action,  learners  must 
compare  the  internal  and  external  feedback,  the  internal 
feedback  and  reference  values,  and  the  external  feedback 
and  the  internal  reference  values.  As  discussed  in  the 
five-stage  model  of  mindful  feedback  processing 
(Bangert-Drowns  et  al.,  1991)  and  in  Butler  and  Winne’s 
(1995)  synthesis  on  feedback  and  self-regulated  learning, 
many  individual  factors  may  affect  how  learners  process 
these  informational  components,  particularly  when  dis¬ 
crepancies  exist  between  the  different  components. 

Learners’  Will  and  Skills  in 
Overcoming  Errors  and  Obstacles 

As  shown  in  studies  on  feedback  seeking,  even  the 
most  sophisticated  feedback  is  useless  if  learners  do 
not  attend  to  it  (Aleven  et  al.,  2003;  Narciss  et  al., 
2004)  or  are  not  willing  to  invest  time  and  effort  in 
error  correction.  In  addition  to  having  the  will,  students 
also  need  the  skills  necessary  to  fulfill  the  requirements 
related  to  error  correction.  Butler  and  Winne  (1995) 
derived  six  maladaptive  ways  of  feedback  seeking  and 
processing  from  Chinn  and  Brewer’s  (1993)  work  on 
how  misconceptions  may  hinder  conceptual  change: 
Students  may  (1)  ignore  the  external  feedback,  (2) 
reject  the  external  feedback,  (3)  judge  the  external 
feedback  irrelevant,  (4)  consider  external  and  internal 
feedback  to  be  unrelated,  (5)  reinterpret  external  feed¬ 
back  to  make  it  conform  to  the  internal  feedback,  or 
(6)  make  superficial  rather  than  fundamental  changes 
to  their  knowledge  or  beliefs.  In  all  these  cases,  the 
effect  of  the  external  feedback  will  be  small. 

External  Loop  Factors:  Instructional  Goals, 
Diagnostic  Procedures,  Feedback  Quality 

In  addition  to  these  internal  loop  factors,  the  ITFL 
model  attracts  attention  to  external  loop  factors  that 
might  affect  the  efficiency  of  both  feedback  loops. 


These  include  the  external  representation  of  task 
requirements  related  to  the  instructional  goals;  the 
accuracy  of  the  diagnostic  procedures  assessing  learn¬ 
ers’  responses  (equal  to  the  quality  of  the  external 
sensor);  the  teaching  medium  abilities  and  strategies 
with  regard  to  analyzing  learners’  responses— namely, 
errors— and  identifying  corrective  actions  with  regard 
to  these  errors  (quality  of  the  external  controller);  and, 
finally,  the  teaching  medium’s  ability  in  communicat¬ 
ing  these  corrective  actions  (equal  to  the  quality  of  the 
external  feedback). 

External  Representation  of  Task 
Requirements  and  Instructional  Goals 

The  starting  point  for  processes  in  the  external  con¬ 
troller  is  a  precise  definition  of  reference  values  of 
controlled  variables.  In  the  external  loop,  these  refer¬ 
ence  values  are  generated  on  the  basis  of  how  the 
instructional  medium  (e.g.,  teacher,  computer-based 
learning  environment)  represents  the  requirements  of 
the  learning  tasks.  As  in  the  internal  loop,  meaningful 
reference  values  can  only  be  generated  if  the  represen¬ 
tation  of  task  requirements  is  adequate.  This  means 
that  learning  goals  must  be  operationalized  in  such  a 
way  that  valid  and  reliably  verifiable  learning  out¬ 
comes  can  be  defined  in  the  form  of  reference  values. 
As  mentioned  above,  this  might  be  more  difficult  for 
more  complex  task  requirements. 

Accuracy  of  Diagnostic  Procedures 

In  the  external  loop,  the  controlled  variables  must 
also  be  diagnosed  accurately  to  make  the  comparison 
between  the  reference  values  with  the  actual  values 
of  controlled  variables  meaningful.  This,  in  turn, 
means  that  the  indicators  appropriate  for  measuring 
different  levels  of  mastery  in  a  valid  and  reliable  way 
must  be  determined.  How  challenging  an  accurate 
diagnosis  might  be  was  elicited  in  a  recent  study  of 
Chi  et  al.  (2004)  on  the  accuracy  of  human  tutors. 
Chi  and  her  colleagues  found  that  tutors  were  only 
able  to  assess  students’  understanding  from  their  own 
perspective,  and  they  were  not  able  to  diagnose  stu¬ 
dents’  alternative  understanding  from  the  perspective 
of  the  students’  knowledge. 

Quality  of  External  Data  Processing 
and  Feedback  Design 

If  a  discrepancy  between  the  actual  and  reference  val¬ 
ues  of  controlled  variables  is  detected,  a  correction 
variable  must  be  defined.  A  key  issue  here  is  how  well 
the  external  controller  (i.e.,  the  learning  medium)  is 
able  to  transform  this  discrepancy  value  into  a  correc¬ 
tion  variable  that  has  a  high  level  of  information  relevant 
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to  mastering  the  task  requirements.  Especially  with 
difficult  and  complex  learning  tasks,  a  series  of  trans¬ 
formations  may  be  necessary  so  learners  can  obtain 
information  about  the  external  correction  variable 
(external  feedback)  that  they  can  use  to  correct  errors 
or  overcome  obstacles.  The  starting  point  for  the  nec¬ 
essary  transformational  steps  is  precise  knowledge  of 
the  controlled  process.  It  is  necessary  to  know  which 
factors,  in  the  sense  of  controlled  variables,  are  respon¬ 
sible  for  the  system’s  performance  and  thus  must  be 
addressed  by  the  correction  variable— that  is,  the  exter¬ 
nal  feedback. 


DESIGNING  AND  EVALUATING 
(TUTORING)  FEEDBACK 

Researchers  used  a  large  variety  of  feedback  types. 
Widely  used  feedback  types  include: 

•  Knowledge  of  performance  (KP)  provides 
learners  with  a  summative  feedback  after 
they  have  responded  to  a  set  of  tasks.  This 
feedback  contains  information  on  the 
achieved  performance  level  for  this  set  of 
tasks  (e.g.,  percentage  of  correctly  solved 
tasks). 

•  Knowledge  of  result! response  (KR)  provides 
learners  with  information  on  the  correctness 
of  their  actual  response  (e.g.,  correct/incor¬ 
rect). 

•  Knowledge  of  the  correct  response  (KCR) 
provides  the  correct  answer  to  the  given  task. 

•  Answer-until-correct  (AUC)  feedback  pro¬ 
vides  KR  and  offers  the  opportunity  of  fur¬ 
ther  tries  with  the  same  task  until  the  task  is 
answered  correctly. 

•  Multiple-try  feedback  (MTF)  provides  KR 
and  offers  the  opportunity  of  a  limited  num¬ 
ber  of  further  tries  with  the  same  task. 

•  Elaborated  feedback  (EF)  provides  addi¬ 
tional  information  besides  KR  or  KCR. 

Complex  elaborated  feedback  exists  in  multiple  forms 
and  is  thus  related  to  a  large  if  not  fuzzy  set  of  mean¬ 
ings.  Several  authors  have  attempted  to  classify  the 
numerous  feedback  types  (Dempsey  et  al.,  1993;  Kul- 
havy  and  Stock,  1989;  Mason  and  Bruning,  2001; 
Schimmel,  1988).  There  is  some  congruency  with 
regard  to  classifying  simple  feedback  types  such  as 
KR  or  KCR,  even  though  these  feedback  types  are 
sometimes  denoted  by  different  terms  (e.g.,  knowledge 
of  result,  confirmation  feedback,  simple  verification 
feedback,  knowledge  of  the  correct  answer/response) . 


The  various  classifications  differ,  however,  in  how  they 
organize  the  different  types  of  elaborated  feedback: 
Kulhavy  and  Stock  (1989)  differentiate  among  task- 
specific  elaborated  information,  which  in  cases  of  mul¬ 
tiple-choice  tasks  is  considered  to  be  knowledge  of  the 
correct  response;  instruction-based  elaborated  infor¬ 
mation  (e.g.,  hints  to  the  section  of  the  instructional 
text  that  is  relevant  for  answering  the  task);  and  extra- 
instructional  elaborated  information,  which  goes 
beyond  the  instructional  text  and  might,  for  example, 
address  metacognitive  strategies.  Mason  and  Bruning 
(2001)  differentiate  among  the  following  elaborated 
feedback  components:  topic-contingent  (provides  item 
verification  and  general  information  concerning  the 
topic),  response-contingent  (provides  KR,  KCR,  and 
explanations  as  to  why  answers  are  correct  or  incor¬ 
rect),  bug-related  (provides  KR  and  error- specific 
information)  (Schimmel,  1988),  and  attribute  isolation 
(provides  KR  and  highlights  the  relevant  attributes  of 
the  concept)  (Merrill,  1987). 

Comparing  these  classifications  reveals  that  feed¬ 
back  types  can  vary  in  functional,  content-related,  and 
formal  characteristics.  One  may  conclude  that  the 
nature  and  quality  of  an  external  feedback  message  is 
determined  by  at  least  three  facets  of  feedback:  (1) 
functional  aspects  related  to  instructional  objectives 
(e.g.,  cognitive  functions  such  as  promoting  informa¬ 
tion  processing,  motivational  functions  such  as  rein¬ 
forcing  correct  responses  or  sustaining  effort  and  per¬ 
sistence);  (2)  semantic  aspects  related  to  the  content 
of  the  feedback  message;  and  (3)  formal  and  technical 
aspects  related  to  the  presentation  of  the  feedback  mes¬ 
sage  (e.g.,  frequency,  timing,  mode,  amount,  form) 
(Narciss,  2006;  Narciss  and  Huth,  2004). 

The  purposes  of  the  following  sections  are  (1)  to 
present  principles  for  selecting  and  specifying  the 
functional,  content-related,  and  formal  dimensions  of 
elaborated  feedback  components  that  can  be  imple¬ 
mented  in  a  tutoring  feedback  algorithm,  and  (2)  to 
outline  implications  for  future  feedback  research. 

Selecting  and  Specifying  the 
Functions  of  External  Feedback 

Different  theoretical  frameworks  use  different  types  of 
feedback  and  attribute  different  functions  to  feedback 
in  learning  situations.  From  a  behavioral  viewpoint, 
feedback  is  considered  to  reinforce  correct  responses. 
In  behavioral  learning  contexts,  the  focus  of  interest 
is  therefore  more  on  formal  and  technical  feedback 
characteristics  such  as  frequency  and  delay  than  on  the 
complexity  of  the  feedback  contents;  hence,  behavioral 
studies  use  outcome -related  feedback  types  such  as 
knowledge  of  result  or  knowledge  of  the  correct 
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TABLE  11.1 

Feedback  Functions  in  Four  Sources 


Cusella 

Sales 

Wager  and 

Butler  and  Winne 

(1987) 

(1993) 

Mory  (1993) 

(1995) 

Reinforcing 

Stimulating 

Confirming 

Confirming 

Informing 

Informing 

Informing 

Informing 

Indicating 

Guiding 

Indicating 

Indicating 

Motivating 

Motivating 

Motivating 

Correcting 

Regulating 

Regulating 

Correcting 

Making  suggestions 

Instructing 

Instructing 

Instructing 

Completing  knowledge 

Assessing 

Assessing 

Differentiating 

Advising 

Restructuring 

response  (for  a  review,  see  Kulik  and  Kulik,  1988). 
From  a  cognitive  viewpoint,  feedback  is  considered  a 
source  of  information  necessary  for  the  correction  of 
incorrect  responses  (Anderson  et  al.,  1971;  Kulhavy 
and  Stock,  1989).  The  question  of  which  type  of  elab¬ 
orated  feedback  information  is  most  efficient  is  of 
major  interest  in  cognitive  feedback  studies;  however, 
in  most  of  these  studies  even  elaborated  informative 
feedback  has  only  been  conceptualized  as  seeking  to 
confirm  or  change  a  learner’s  domain  knowledge. 
Feedback  models  that  view  feedback  in  the  context  of 
self-regulated  learning  theorize  that  the  most  important 
function  of  feedback  is  tutoring  or  guiding  the  learner 
to  regulate  the  learning  process  successfully  (Butler 
and  Winne,  1995). 

This  brief  summary  of  prior  research  reveals  that 
feedback  can  affect  the  learning  process  at  various 
levels  and  can  therefore  have  numerous  different  func¬ 
tions.  For  this  reason,  a  number  of  authors  have  made 
more  subtle  distinctions  (Butler  and  Winne,  1995; 
Cusella,  1987;  Sales,  1993;  Wager  and  Mory,  1993) 
(see  Table  1 1 .1).  A  comparison  of  these  differentiated 
treatments  of  feedback  functions  reveals  that  all  of 
these  authors  advocate  feedback  as  an  acknowledging 
or  reinforcing  function,  an  informing  function,  and 
some  form  of  guiding  or  steering  function.  Moreover, 
all  of  them  have  postulated  a  regulatory  or  correcting 
function  for  feedback.  In  addition,  Cusella  (1987), 
Sales  (1993),  and  Wager  and  Mory  (1993)  drew  atten¬ 
tion  to  the  motivational  and  instructional  function  of 
feedback.  Butler  and  Winne  (1995)  described  at  least 
three  subfunctions  of  the  instructing  function  (tuning 
or  completing,  differentiating,  and  restructuring).  In 
addition,  these  authors  have  pointed  out  that  feedback 
can  activate  metacognitive  processes  such  as  monitor¬ 
ing  or  information  seeking. 

If  external  informative  feedback  is  viewed  from 
the  standpoint  of  the  current  ITFL  model,  it  becomes 
clear  that  as  a  general  rule  multiple  feedback  functions 


come  into  play  simultaneously,  according  to  how  the 
controlled  and  command  variables  are  defined.  On  the 
basis  of  the  models  of  good  information  processors 
(Pressley,  1986),  intelligent  novices  (Mathan  and 
Koedinger,  2005),  and  self-directed  learning  (Boe- 
kaerts,  1996),  possible  feedback  functions  can  be 
defined  from  the  cognitive,  metacognitive,  and  moti¬ 
vational  standpoints.  Because  finer  differentiations  of 
feedback  functions  make  it  possible  to  work  out  which 
information  will  be  useful  in  which  settings,  careful 
selection  and  specification  of  the  intended  feedback 
functions  provide  the  basis  for  designing  tutorial  feed¬ 
back. 

Cognitive  Functions 

In  the  case  of  complex  tasks,  incorrect  answers  and 
solutions  can  occur  for  widely  varying  reasons  (Van- 
Lehn,  1990).  The  content-related,  procedural,  or  stra¬ 
tegic  knowledge  elements  that  a  learner  needs  to  arrive 
at  a  correct  solution  may  be  lacking,  erroneous,  or 
imprecise.  The  necessary  knowledge  elements  may 
also  be  incorrectly  linked  or  the  conditions  for  their 
use  incorrect  or  ill-defined.  Feedback  can  offer  infor¬ 
mation  on  all  of  these  aspects.  A  distinction  can  be 
made  between  the  following  cognitive  feedback  func¬ 
tions  in  connection  with  incorrect  responses: 

•  An  informative  function  in  cases  where  the 
number,  location,  and  type  of  error  or  reason 
for  the  error  are  unknown 

•  A  completion  function  in  cases  where  the 
error  is  attributable  to  lack  of  content- 
related,  procedural,  or  strategic  knowledge 
and  the  feedback  provides  information  on 
the  missing  knowledge 

•  A  corrective  function  in  cases  where  the 
error  is  attributable  to  erroneous  content  or 
erroneous  procedural  or  strategic  elements 
and  the  feedback  provides  information  that 
can  be  used  to  correct  the  erroneous  ele¬ 
ments 

•  A  differentiation  function  in  cases  where  the 
error  is  attributable  to  imprecise  content- 
related,  procedural,  or  strategic  knowledge 
elements  and  the  feedback  provides  infor¬ 
mation  that  allows  for  clarification  of  the 
imprecise  elements 

•  A  restructuring  function  in  cases  where  the 
error  is  attributable  to  erroneous  connections 
between  content,  procedural,  or  strategic  ele¬ 
ments  and  the  feedback  provides  informa¬ 
tion  that  can  be  used  to  restructure  these 
incorrectly  connected  elements 
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Metacognitive  Functions 

According  to  Butler  and  Winne  (1995),  external  feed¬ 
back  can  have  numerous  metacognitive  functions  apart 
from  those  listed  in  Table  11.1;  for  example,  external 
feedback  can  address  metacognitive  strategies  and  their 
deployment  options ,  provide  criteria  for  monitoring  and 
evaluating  goals,  or  motivate  learners  to  generate  their 
own  monitoring  related  information.  In  addition,  it  can 
serve  as  a  basis  for  assessing  the  suitability  of  solution 
strategies  employed  or  of  error  search  and  correction 
strategies;  hence,  at  least  the  following  feedback  func¬ 
tions  can  be  differentiated  from  each  other  with  regard 
to  mastery  of  metacognitive  requirements: 

•  An  informative  function  in  cases  where 
metacognitive  strategies  or  the  conditions 
for  their  use  are  unknown  and  feedback  pro¬ 
vides  information  about  metacognitive  strat¬ 
egies 

•  A  specification  function  in  cases  where  feed¬ 
back  provides  criteria  for  monitoring  goals 
or  where  conditions  for  the  use  of  specific 
solution  strategies  or  metacognitive  strate¬ 
gies  are  specified 

•  A  corrective  function  in  cases  where  errors 
have  arisen  in  the  use  of  metacognitive  strat¬ 
egies  and  the  feedback  provides  information 
that  can  be  used  to  correct  erroneous  strate¬ 
gies 

•  A  guiding  function  in  cases  where  learners 
are  encouraged  (e.g.,  through  leading  ques¬ 
tions)  to  generate  their  own  criteria  for  mon¬ 
itoring  or  evaluation  or  to  assess  the 
suitability  of  their  own  solution  strategies  or 
other  actions 

Recent  studies  on  the  effects  of  feedback  addressing 
metacognitive  processes  and  strategies  have  provided 
mixed  results  (Roll  et  al.,  2006;  van  den  Boom  et  al., 
2004). 

Motivational  Functions 

Even  though  feedback  has  been  assigned  an  important 
role  for  both  achievement  and  motivation  (Hoska, 
1993;  Kluger  and  DeNisi,  1996;  Mory,  1996),  most 
studies  on  external  informative  feedback  have  focused 
on  learner  achievement  and  neglected  the  impact  of 
feedback  on  motivation.  At  the  motivational  level, 
however,  it  is  crucial,  despite  errors  and  the  resulting 
negative  effect,  to  maintain  the  level  of  effort,  persis¬ 
tence,  and  intensity  of  task  processing.  Many  theories 
of  motivation  suggest  that  perceived  values  of  task 


processing  and  self-perceptions  of  competence  are  cru¬ 
cial  factors  in  learners’  motivation  (Pintrich,  2003). 

Generally,  all  types  of  feedback  contain  an  evalu¬ 
ative  feedback  component  (i.e.,  information  regarding 
the  correctness  or  quality  of  the  solution)  that  reveals 
success  or  failure  in  task  processing.  Feedback  thus 
has  an  impact  on  the  attainment  value  of  the  task  that 
might  result  in  more  effort  or  strategy  investment  and 
might  affect  performance.  Symonds  and  Chase  (1929) 
and  Brown  (1932)  reported  supportive  results  for  this 
motivational  effect  of  feedback.  Recently,  a  study  of 
Vollmeyer  and  Rheinberg  (2005)  revealed  that  this 
impact  of  feedback  is  present  even  if  feedback  is 
merely  announced.  Moreover,  Ulicsak  (2004)  found 
that  students  spent  more  time  reflecting  group  activi¬ 
ties  if  they  believed  that  the  instructional  system  was 
observing  them  and  would  provide  feedback. 

If  feedback  provides  additional  elaborated  com¬ 
ponents  that  guide  learners  to  successful  task  com¬ 
pletion  without  immediately  providing  knowledge  of 
the  correct  response,  it  offers  mastery  experiences 
that  can  be  linked  to  personal  causation.  As  such, 
mastery  experiences  are  considered  the  most  impor¬ 
tant  source  for  developing  a  positive  self-efficacy— in 
other  words,  positive  perceptions  of  competence 
(Bandura,  1997;  Usher  and  Pajares,  2006).  Feedback 
may  also  affect  how  the  difficulty  of  such  tasks,  the 
prospects  of  success,  and  the  attributions  of  success 
or  failure  are  assessed  in  future  situations;  hence,  at 
least  the  following  basic  motivational  functions 
should  be  considered  when  evaluating  informative 
elaborated  feedback: 

•  An  incentive  function,  in  that  feedback  ren¬ 
ders  the  results  of  task  processing  visible 

•  A  task  facilitation  function  to  contribute 
information  for  overcoming  task  difficulties 

•  A  self-efficacy  enhancing  function,  if  it  pro¬ 
vides  information  that  makes  it  possible  to 
master  tasks  successfully,  even  if  errors  are 
committed  or  difficulties  arise 

•  A  reattribution  function,  if  it  provides  infor¬ 
mation  that  contributes  to  mastery  experi¬ 
ences  that  can  be  linked  to  personal 
causation 

In  addition  to  informative  elaborated  feedback 
types,  a  variety  of  motivational  elaborated  feedback 
types  has  been  investigated  by  motivational  research¬ 
ers.  Such  motivational  feedback  types  include  reattri¬ 
bution  feedback  (Dresel  and  Ziegler,  2006;  Schunk, 
1983);  mastery-oriented  feedback,  which  makes 
learner’s  progress  visible  (Schunk  and  Rice,  1993); 
and  task  vs.  competence  feedback  (Sansone,  1986, 
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TABLE  11.2 

Content-Related  Classification  of  Feedback  Components 


Category 

Knowledge  of  performance  (KP) 
Knowledge  of  result/response  (KR) 
Knowledge  of  the  correct  results  (KCR) 

Elaborated  concepts 

Knowledge  about  task  constraints  (KTC) 


Knowledge  about  concepts  (KC) 


Knowledge  about  mistakes  (KM) 


Knowledge  about  how  to  proceed  (KH) 


Knowledge  about  metacognition  (KMC) 


Examples 

15  of  20  correct;  85%  correct 
Correct/incorrect 

Description/indication  of  the  correct  response 

Hints/explanations  on  type  of  task 
Hints/explanations  on  task-processing  rules 
Hints/explanations  on  subtasks 
Hints/explanations  on  task  requirements 
Hints/explanations  on  technical  terms 
Examples  illustrating  the  concept 
Hints/explanations  on  the  conceptual  context 
Hints/explanations  on  concept  attributes 
Attribute-isolation  examples 
Number  of  mistakes 
Location  of  mistakes 
Hints/explanations  on  type  of  errors 
Hints/explanations  on  sources  of  errors 
Bug-related  hints  for  error  correction 
Hints/explanations  on  task-specific  strategies 
Hints/explanations  on  task-processing  steps 
Guiding  questions 
Worked-out  examples 

Hints/explanations  on  metacognitive  strategies 
Metacognitive  guiding  questions 


Source:  Narciss,  S.,  Informatives  tutorielles  Feedback.  Entwicklungs-  und  Evaluationsprin- 
zipien  auf  der  Basis  instruktionspsychologischer  Erkenntnisse  ( Informative  Tutoring  Feed¬ 
back),  Waxmann,  Munster,  2006.  With  permission. 


1989;  Senko  and  Harackiewicz,  2005).  In  summary, 
elaborated  motivational  feedback  components  that  had 
a  positive  impact  on  learners’  motivation  (namely,  on 
perceptions  of  competence):  (1)  stressed  the  relation 
between  effort,  ability,  and  success;  (2)  made  progress 
visible;  (3)  provided  task  information  rather  than  per¬ 
formance  information;  or  (4)  elicited  goal  discrepancy. 

Selecting  and  Specifying  the 
Content  of  Feedback  Elements 

In  general,  the  content  of  a  feedback  message  may 
consist  of  two  components  (Kulhavy  and  Stock,  1989). 
The  first  component,  the  evaluative  or,  in  Kulhavy’s 
terms,  the  verification  component,  relates  to  the  learn¬ 
ing  outcome  and  indicates  the  performance  level 
achieved  (e.g.,  correct/incorrect  response,  percentage 
of  correct  answers,  and  distance  to  the  learning  crite¬ 
rion).  This  component  is  attributed  a  controlling  func¬ 
tion  (Keller,  1983).  The  second  component,  the  infor¬ 
mational  component,  consists  of  additional  information 
relating  to  the  topic,  the  task,  errors,  or  solutions.  Com¬ 


bining  the  evaluation  and  information  component  of 
feedback  might  result  in  a  large  variety  of  feedback 
contents. 

Overview  on  Elaborated  Feedback  Components 

Table  1 1 .2  presents  a  content-related  classification  of 
feedback  components  that  provides  a  structured  over¬ 
view  of  simple  and  elaborated  feedback  components 
by  organizing  the  components  with  regard  to  which 
aspect  of  the  instructional  context  is  addressed.  This 
content-related  classification  assumes  that  elaborated 
information  might  address:  (1)  task  rules,  task  con¬ 
straints,  and  task  requirements;  (2)  conceptual  knowl¬ 
edge;  (3)  errors  or  mistakes;  (4)  procedural  knowledge; 
and  (5)  metacognitive  knowledge.  Five  categories  of 
elaborated  feedback  components  can  thus  be  defined: 

•  Elaborated  components  that  provide  infor¬ 
mation  on  task  rules,  task  constraints,  and 
task  requirements  are  linked  by  the  category 
of  knowledge  on  task  constraints  (KTC). 
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•  Elaborated  components  that  provide  infor¬ 
mation  on  conceptual  knowledge  relevant 
for  task  processing  are  linked  by  the  cate¬ 
gory  of  knowledge  about  concepts  (KC). 

•  Elaborated  components  that  provide  informa¬ 
tion  on  errors  or  mistakes  are  linked  with  the 
category  of  knowledge  about  mistakes  (KM). 

•  Elaborated  components  that  provide  infor¬ 
mation  on  procedural  knowledge  relevant  for 
task  processing  are  linked  by  the  category 
knowledge  on  how  to  proceed  or,  briefly, 
know-how  (KH). 

•  Elaborated  components  that  provide  infor¬ 
mation  on  metacognitive  knowledge  are 
linked  by  the  category  knowledge  on  meta¬ 
cognition  (KMC). 

To  design  feedback  algorithms  with  elaborated  com¬ 
ponents,  several  simple  and  elaborated  feedback  com¬ 
ponents  can  be  combined.  In  most  of  the  feedback 
studies,  elaborated  feedback  was  designed  by  combin¬ 
ing  knowledge  of  the  correct  result  or  knowledge  of 
the  result  with  elaborated  components  such  as  expla¬ 
nations  of  errors  or  to  correct  responses. 

Cognitive  Task  and  Error  Analyses 

Narciss  and  Huth  (2004)  derived  the  steps  necessary 
to  select  and  specify  the  feedback  content  from 
knowledge  about  cognitive  task  analysis  and  error 
analysis  (for  a  detailed  description,  see  Jonassen  et 
al.,  1999;  VanLehn,  1990).  Similar  steps  were  pro¬ 
posed  by  VanLehn  and  his  collaborators  (2005)  and 
by  Rittle-Johnson  and  Koedinger  (2005)  based  on 
insights  and  experiences  in  developing  intelligent 
tutoring  systems. 

The  first  step  consists  of  the  selection  and  specifi¬ 
cation  of  instructional  objectives  (e.g.,  acquisition  of 
a  knowledge  domain,  mastery  of  learning  tasks,  liter¬ 
acy  in  the  given  context).  The  starting  point  of  this 
step  is  the  curriculum  and  its  objectives,  which  in 
general  have  to  be  specified  to  obtain  explicit,  con¬ 
crete,  and  measurable  learning  outcomes.  The  revised 
version  of  Bloom’s  taxonomy  of  educational  objec¬ 
tives  offers  a  well-founded  framework  for  this  speci¬ 
fication  of  learning  objectives  (Anderson  et  al.,  2001). 
The  specified  concrete  learning  outcomes  provide  the 
basis  for  the  selection  of  the  feedback  functions,  con¬ 
tent,  and  forms. 

Feedback  is  presented  after  the  accomplishment  of 
learning  tasks;  consequently,  learning  tasks  are  espe¬ 
cially  relevant  to  the  design  of  feedback.  The  second 
step  is,  therefore,  to  select  typical  learning  tasks  and 
match  them  to  the  required  learning  outcomes. 


The  third  step  consists  of  analyzing  the  require¬ 
ments  for  each  type  of  task.  The  aim  of  these  task 
analyses  is  to  identify:  (1)  domain-specific  knowl¬ 
edge  items  (e.g.,  facts,  concepts,  events,  rules,  mod¬ 
els,  theories);  (2)  cognitive  operations  related  to  these 
items  (e.g.,  remember,  transform,  classify,  argue, 
infer);  and  (3)  cognitive  and  metacognitive  skills 
involved  in  the  mastery  of  the  selected  learning  tasks. 
The  informative  components  of  a  feedback  message 
can  refer  to  each  of  these  aspects  of  a  learning  task; 
hence,  the  results  of  these  task  analyses  provide  an 
overview  of  both  task  requirements  and  possible 
informative  components  that  can  be  implemented  in 
a  feedback  message. 

As  mentioned  above,  from  a  cognitive  and  from 
a  self-regulated  learning  viewpoint,  elaborated  or 
informative  feedback  is  considered  a  necessary  source 
of  information,  especially  if  the  learner  encounters 
obstacles  or  proceeds  incorrectly.  A  next  important 
step  for  the  design  of  informative  feedback  is  there¬ 
fore  to  describe  typical  errors  and  typical  incorrect 
steps.  Furthermore,  it  is  necessary  to  identify  miscon¬ 
ceptions  and  incorrect  or  inefficient  strategies  that  can 
be  attributed  to  the  described  errors  (Crippen  and 
Brooks,  2005;  Narciss  and  Huth,  2004,  2006;  Van¬ 
Lehn,  1990). 

The  steps  described  above  are  essential  prerequi¬ 
sites  for  the  selection  and  specification  of  helpful  infor¬ 
mation.  The  results  of  the  task  and  error  analyses  pro¬ 
vide  information  that  is  necessary  to  select  those 
informative  components  that  match  the  task  require¬ 
ments.  If  the  major  function  of  the  feedback  message 
is  tutoring  learners  to  master  the  given  learning  tasks 
and  the  related  requirements,  then  feedback  should  not 
immediately  provide  the  correct  response  or  explain 
the  correct  strategy.  This  information  should  only  be 
offered  if  the  learners  do  not  succeed  otherwise;  hence, 
offering  adequate  tutoring  when  learners  encounter 
obstacles  requires  providing  information  that  gives 
knowledge  on  how  to  proceed  without  presenting 
knowledge  of  the  correct  response.  Table  1 1 .2  presents 
examples  of  such  informative  tutoring  feedback  com¬ 
ponents. 

Smith  and  Ragan  (1993)  recommended  that  the 
content  should  be  tailored  to  the  type  of  learning 
tasks;  however,  it  should  be  kept  in  mind  that  studies 
comparing  the  efficiency  of  different  types  of  infor¬ 
mation  with  regard  to  various  learning  tasks 
reported  rather  mixed  results  (for  a  detailed  review, 
see  Mory,  1996, 2004).  Furthermore,  with  the  devel¬ 
opment  of  new  paradigms  for  learning  and  instruc¬ 
tion,  the  question  of  which  knowledge  should  be 
addressed  by  the  feedback  content  is  getting  more 
and  more  complex. 
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Selecting  and  Specifying  the  Form 
and  Mode  of  Feedback  Presentation 

Feedback  types  vary  not  only  in  their  content-related 
aspects  but  also  in  formal  and  technical  aspects  rele¬ 
vant  for  feedback  presentation.  Using  formal  criteria 
(e.g.,  timing,  frequency),  Holding  (1965)  differenti¬ 
ated,  for  example,  32  different  types  of  feedback.  The 
interactive  capabilities  of  modern  information  technol¬ 
ogy  increase  the  range  of  feedback  strategies  that  can 
be  implemented  efficiently  in  computer-based  instruc¬ 
tion  (Hannafin  et  al.,  1993).  Using  the  interactive  capa¬ 
bilities  of  modern  information  technology,  it  is,  for 
example,  possible  to  combine  elaborated  feedback, 
tutoring,  and  mastery  learning  strategies  to  design 
informative  tutoring  feedback.  The  term  informative 
tutoring  feedback  (ITF)  refers  to  feedback  strategies 
that  provide  elaborated  feedback  components  to  guide 
the  learner  toward  successful  task  completion.  The 
focus  of  this  elaborated  information  is  on  tutoring  stu¬ 
dents  to  detect  errors,  overcome  obstacles,  and  apply 
more  efficient  strategies  for  solving  the  learning  tasks. 
In  contrast  to  elaborated  feedback  types,  which  pro¬ 
vide  learners  with  immediate  knowledge  of  the  correct 
response  and  additional  information,  ITF  components 
are  presented  without  immediate  knowledge  of  the 
correct  response.  Additionally,  ITF  strategies  offer  the 
opportunity  to  apply  the  feedback  information  on 
another  try  (Narciss,  2006).  These  ITF  strategies  are 
rooted  in  studies  on  tutoring  activities  (McKendree, 
1990;  Merrill  et  al.,  1992,  1995).  The  following  sec¬ 
tions  present  an  overview  on  important  aspects  of  feed¬ 
back  that  must  be  taken  into  consideration  when 
choosing  the  form  and  mode  of  feedback  presentation. 

Immediate  vs.  Delayed  Feedback  Timing 

An  aspect  of  feedback  that  received  much  attention  in 
feedback  research  is  the  timing  of  the  feedback  (Demp¬ 
sey  and  Wager,  1988;  for  areview,  see  Kulik  and  Kulik, 
1988).  From  Skinner’s  operant  learning  theory,  one 
might  assume  that  the  feedback  should  be  provided 
soon  after  the  response;  however,  experimental  studies 
that  used  paradigms  similar  to  those  of  studies  testing 
the  effects  of  blocked  or  massed  vs.  distributed  practice 
found  that  delaying  feedback  can  be  beneficial,  espe¬ 
cially  for  retention  in  a  delayed  post-test.  This  effect 
is  referred  to  as  the  delay  retention  effect  (Brackbill  et 
al.,  1963).  Kulhavy  and  Anderson  (1972)  explained  the 
delay  retention  effect  by  an  interference  perseveration 
hypothesis,  which  suggests  that  immediate  feedback 
might  proactively  interfere  with  the  incorrect  response, 
and  this  interference  might  hinder  the  acquisition  of 
the  correct  response.  Delayed  feedback  is  not  related 


to  proactive  interference,  because  the  incorrect 
response  is  not  present  and  probably  forgotten. 
Research  based  on  the  interference  perseveration 
hypothesis  provided  mixed  results  (Kulhavy  and 
Anderson,  1972;  Kulhavy  and  Stock,  1989;  Marko- 
wotz  and  Renner,  1966;  Peek  and  Tillema,  1978; 
Rankin  and  Trepper,  1978;  Schroth  and  Lund,  1993; 
Sturges,  1969,  1972,  1978;  Surber  and  Anderson, 
1975).  Kulik  and  Kulik  (1988)  proposed  a  dual-trace 
information-processing  explanation  for  the  delay 
retention  effect.  They  pointed  out  that,  with  immediate 
feedback,  learners  only  have  one  trial,  whereas  with 
delayed  feedback  they  have  two  separate  trials  with  an 
item.  In  the  case  of  memorization,  two  separate  trials 
are  better  than  one,  and  delayed  feedback  might  be 
superior  to  immediate  feedback. 

Clariana  has  developed  a  connectionist  descrip¬ 
tion  of  feedback  timing  to  better  explain  the  existing 
results  and  to  provide  a  basis  for  new  insights  on 
immediate  and  delayed  feedback  (Clariana,  1999; 
Clariana  et  al.,  2000).  With  regard  to  the  potential 
effects  of  immediate  vs.  delayed  feedback,  Clariana’s 
model  proposes  a  strengthening  effect  for  incorrect 
responses  with  delayed  feedback,  whereas  immediate 
feedback  weakens  the  association  between  incorrect 
responses  and  items.  These  hypotheses  were  con¬ 
firmed  by  a  study  of  Clariana  and  Koul  (2005);  yet, 
the  superiority  of  delayed  feedback  (i.e.,  the  delay 
retention  effect)  was  only  found  in  experimental  sit¬ 
uations  with  test  items,  and  it  was  not  found  in 
applied  studies  (Kulik  and  Kulik,  1988).  Because 
researchers  used  a  variety  of  immediate  and  delayed 
feedback  types— item  per  item  vs.  end  of  session; 
directly  after  the  session  vs.  hours  or  days  after  ses¬ 
sion  (Dempsey  and  Wager,  1988)— Mory  (2004,  p. 
256)  stated  that  the  field  of  research  on  feedback 
timing  is  “muddied.” 

Recently,  Mathan  and  Koedinger  (2005)  reconsid¬ 
ered  the  debate  on  feedback  timing  from  a  metacog- 
nitive  perspective.  They  suggested  that  the  question  of 
when  to  provide  feedback  following  an  error  has  to  be 
answered  on  the  basis  of  a  model  of  desired  perfor¬ 
mance.  If  this  model  includes  metacognitive  skills  for 
error  detection  and  correction,  then  feedback  providing 
knowledge  of  the  correct  response  should  not  be 
offered  immediately,  because  it  does  not  foster  the 
acquisition  of  these  skills.  In  contrast,  feedback  offer¬ 
ing  knowledge  of  the  result  together  with  knowledge 
about  mistakes  implemented  in  a  multiple-try  algo¬ 
rithm  that  requires  students  to  analyze  their  erroneous 
responses  and  to  identify  error  correction  steps  can  be 
provided  immediately  (see,  for  example,  Mathan  and 
Koedinger,  2005;  Moreno  and  Valdez,  2005;  Narciss 
and  Huth,  2006). 
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Single-Try  vs.  Multiple-Try:  Simultaneous  vs. 
Sequential  Presentation  of  Elaborated  Feedback 

A  second  formal  aspect  is  related  to  the  question  of 
how  many  tries  are  offered  to  learners  after  they  have 
received  feedback.  Many  studies  offer  only  a  single 
try  per  item;  that  is,  learners  respond  to  an  item,  are 
provided  with  feedback,  and  do  not  have  the  oppor¬ 
tunity  to  respond  again  to  this  item.  Some  studies, 
however,  have  offered  multiple  tries  after  providing 
feedback.  Most  of  these  studies  use  answer-until- 
correct  (AUC)  feedback  (for  a  review,  see  Clariana, 
1993).  Clariana’s  review  of  30  studies  that  compared 
single-try  feedback  types  (immediate  knowledge  of 
result,  immediate  knowledge  of  the  correct  response, 
delayed  feedback,  no  feedback)  to  multiple-try  feed¬ 
back/ AUC  found  a  superiority  of  all  feedback  types 
over  no  feedback  but  no  differences  between  single- 
try  and  multiple-try  feedback.  In  a  more  recent 
review,  Clariana  and  Koul  (2004)  contrasted  multiple- 
try  feedback  effects  (AUC)  for  verbatim  outcomes 
with  higher  order  “more  than  verbatim”  outcomes 
(i.e.,  drawing  and  labeling  biological  diagrams).  This 
review  revealed  that  AUC  is  less  effective  for  verba¬ 
tim  outcomes  but  more  effective  for  higher  order 
outcomes  (Clariana  and  Koul,  2005). 

Multiple-try  feedback  types  other  than  AUC  can 
be  developed  if  one  considers  a  third  formal  aspect  of 
feedback  presentation:  Complex  elaborated  feedback 
can  be  presented  simultaneously  (i.e.,  all  information 
in  one  step)  or  sequentially  (cumulatively  or  step  by 
step).  Most  studies  on  complex  elaborated  feedback 
provide  the  elaborated  information  simultaneously 
with  knowledge  of  the  result  or  knowledge  of  the  cor¬ 
rect  response  (Kulhavy  et  al.,  1985;  Phye,  1979;  Phye 
and  Bender,  1989).  However,  only  half  of  the  studies 
utilizing  this  simultaneous  presentation  of  elaborated 
feedback  produced  significant  positive  effects  (Kul¬ 
havy  and  Stock,  1989;  Mory,  1996,  2004). 

In  addition  to  these  empirical  findings  on  pre¬ 
senting  complex  elaborated  feedback  simulta¬ 
neously,  research  on  cognitive  load  in  instructional 
contexts  would  suggest  that  a  sequential  presentation 
of  complex  elaborated  feedback  should  be  superior 
to  a  simultaneous  presentation  (Chandler  and 
Sweller,  1992).  Indeed,  the  few  controlled  experi¬ 
mental  studies  that  have  investigated  the  tutorial 
feedback  types  that  present  elaborated  feedback 
components  sequentially  have  reported  positive 
effects  (Albacete  and  VanLehn,  2000;  Heift,  2004; 
Nagata,  1993;  Nagata  and  Swisher,  1995;  Narciss 
and  Huth,  2006;  VanLehn  et  al.,  2005). 

Because  a  sequential  presentation  of  feedback 
components  requires  offering  multiple  tries  with  the 


same  item,  a  direct  comparison  of  the  effects  of  simul¬ 
taneous  vs.  sequential  feedback  presentations  is  very 
difficult  if  not  impossible.  An  important  issue  for 
future  research,  however,  would  be  addressing  the 
question  of  how  many  feedback  steps  or  cycles  are 
effective  under  which  individual  and  situational  con¬ 
ditions. 

Adaptive  vs.  Nonadaptive  Feedback  Presentation 

A  fourth  formal  aspect  of  feedback  presentation  is 
whether  the  feedback  is  presented  in  an  adaptive  or  a 
nonadaptive  way.  The  adaptation  issue  is  related  at 
such  questions  as  these: 

•  Which  learner  characteristics  are  critical 
for  adaptation?  Crucial  characteristics  that 
have  been  extensively  addressed  by  feed¬ 
back  research  and  by  most  research  on 
tutoring  systems  include  the  learner’s  prior 
knowledge  or  knowledge  state  (Albert  and 
Lukas,  1999;  Hancock  et  al.,  1995a)  and 
the  learner’s  metacognitive  state  in  general 
measured  by  the  learner’s  response  certi¬ 
tude  (Hancock  et  al.,  1992,  1995b;  Mory, 
1991,  1994).  Other  important  characteris¬ 
tics  that  have  received  attention  only  in 
recent  studies  include  the  learner’s  motiva¬ 
tion  (e.g.,  self-efficacy)  (Narciss,  2004), 
goal  orientation  (Senko  and  Harackiewicz, 
2005),  and  metacognitive  skills  other  than 
response  certitude  (Aleven  et  al.,  2006). 

•  Which  task  characteristics  are  critical  for 
adaptation?  According  to  Sanz  (2004),  this 
question  is  sometimes  neglected  by 
instructional  designers;  however,  adapta¬ 
tion  may  be  more  or  less  necessary  for 
different  tasks,  and  there  might  be  critical 
task  characteristics  (i.e.,  specific  task 
requirements)  that  can  be  used  as  indica¬ 
tors  for  deciding  when  and  how  much 
adaptation  would  be  reasonable.  In  the 
algebra  tutoring  system  Ms.  Lindquist,  the 
three  feedback  strategies  are,  for  example, 
determined  by  the  exercise  and  its  structure 
(Heffernan,  2001). 

•  How  do  we  diagnose  the  individual  charac¬ 
teristics  in  a  reliable  and  valid  way?  Several 
approaches  to  diagnosing  learner  character¬ 
istics  have  been  investigated  by  researchers 
who  are  developing  intelligent  tutoring  sys¬ 
tems:  manually  authored  finite-state 
machines  (Koedinger  et  al.,  2004);  genera¬ 
tive  approaches,  such  as  model  tracing 
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(Anderson  et  al.,  1995);  evaluative 
approaches  (Mitrovich  et  al.,  2002);  and 
decision  theoretic  approaches  (Murray  et 
al.,  2004).  Recently,  several  authors  have 
suggested  using  observable  data  on  stu¬ 
dents’  activities  to  infer  nonobservable 
learner  characteristics  (Kutay  and  Ho, 
2005;  Melis  and  Anders,  2005;  Romero  et 
al.,  2005). 

•  How  do  we  adapt  feedback  to  the  critical 
situational  and  individual  factors?  Adap¬ 
tive  feedback  can  be  implemented  in  sev¬ 
eral  ways.  An  approach  used  frequently  in 
intelligent  tutoring  systems  involves  con¬ 
trolling  the  sequence,  content,  and  instruc¬ 
tional  activities  (program-controlled 
adaptation).  A  second  type  of  adaptation  is 
based  on  the  idea  that  the  learner  has  to 
take  an  active  part  in  instruction  and  thus 
is  presented  with  a  choice  of  instructional 
activities  (learner-controlled  adaptation). 
Unfortunately,  learners  sometimes  lack  the 
metacognitive  skills  and  motivation 
required  to  decide  which  instructional 
activities  would  be  best  for  them  (for 
reviews  on  the  effects  of  learner-  vs.  pro¬ 
gram-controlled  instruction,  see  Steinberg, 
1977, 1989;  see  also  Corbett  and  Anderson, 
1990;  Narciss  et  al.,  2004).  Recent  studies 
and  frameworks  on  adaptive  feedback 
include  metacognitive  feedback  compo¬ 
nents  that  should  foster  the  acquisition  of 
metacognitive  skills  (Aleven  et  al.,  2006; 
Gouli  et  al.,  2005).  A  third  type  of  adapta¬ 
tion  consists  of  combining  program  and 
learner  control,  which  offers  a  variety  of 
other  possibilities  for  adapting  feedback 
and  raises  new  issues  for  future  research 
(e.g.,  when  and  how  to  shift  from  program 
to  learner  control  and  vice  versa). 


Unimodal  vs.  Multimodal  Feedback  Presentation 

The  capabilities  of  modern  information  technologies 
allow  the  presentation  of  feedback  not  only  as  written 
text  but  also  as  narrated  text  (Narciss  and  Huth,  2004, 
2006)  or  as  a  static  or  dynamic  graphic.  Furthermore, 
feedback  can  be  provided  by  animated  agents 
(Moreno,  2004).  When  and  how  to  apply  the  princi¬ 
ples  of  multimedia  learning  derived  from  Mayer’s 
theory  of  multimedia  learning  (Mayer,  2001)  for  the 
multimodal  presentation  of  feedback  have  yet  to  be 
investigated. 


Implications  for  Evaluating 
(Tutoring)  Feedback 

The  design  principles  outlined  above  show  that  exter¬ 
nal  feedback,  particularly  informative  tutorial  feed¬ 
back,  is  a  multidimensional  instructional  measure. 
Moreover,  the  interactive,  two-feedback-loop  model 
described  earlier  suggests  that  the  effects  of  external 
feedback  occur  through  an  interaction  with  the 
learner  (i.e.,  with  a  complex  information  processing 
system).  This  in  turn  means  that  the  effects  of  exter¬ 
nal  feedback  are  not  general  but  only  emerge  in 
specific  situational  and  individual  settings;  for  exam¬ 
ple,  the  amount  of  time  it  takes  for  errors  to  be 
eliminated  with  the  help  of  external  feedback 
depends  on:  (1)  the  individual  characteristics  of  the 
learner;  (2)  the  quality  of  the  external  feedback  com¬ 
ponents;  (3)  the  type,  complexity,  and  difficulty  of 
the  tasks;  and  (4)  the  type  of  error.  In  highly  skilled 
learners  or  for  easy  tasks  or  simple  slips,  for  exam¬ 
ple,  knowledge-of-result  feedback  alone  is  sufficient 
to  yield  a  correct  response  the  next  time.  In  learners 
with  a  low  level  of  skill,  for  very  complex  and  dif¬ 
ficult  tasks,  or  in  the  case  of  serious  errors,  it  is 
possible  that  even  informative  tutorial  feedback  may 
not  be  sufficient  for  mastering  the  high  demands. 

The  effects  of  various  feedback  strategies  also 
largely  depend  on  how  learners  process  and  interpret 
the  information  provided.  In  addition  to  cognitive 
requirements  (e.g.,  prior  knowledge,  strategic  knowl¬ 
edge),  individual  motivational  factors,  such  as  self- 
efficacy  and  perceived  task  values,  and  individual 
metacognitive  factors ,  such  as  monitoring  competen¬ 
cies  and  strategies,  play  a  role.  To  draw  differentiated 
conclusions  about  the  effects  of  various  types  of 
feedback,  not  only  cognitive  but  also  individual 
motivational  and  metacognitive  factors  and  the 
nature  of  how  individuals  process  the  feedback 
should  be  controlled. 

External  feedback  can  contribute  to  changes  that 
occur  (1)  during  the  treatment,  (2)  shortly  after  the 
treatment,  or  (3)  long  after  the  treatment;  thus,  evalu¬ 
ating  the  effects  of  various  feedback  strategies  requires 
collecting  data  both  during  and  after  the  treatment 
(Phye,  1991,  2001;  Phye  and  Sanders,  1994).  When 
investigating  the  effects  of  various  types  or  strategies 
of  external  feedback,  it  should  no  longer  be  a  question 
of  which  feedback  type  is  the  best  but  rather  one  of 
the  following  questions: 

•  Under  which  individual  and  situational  con¬ 
ditions  do  which  feedback  components  or 
strategies  have  high  information  value  for 
learners? 
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Performance  Performance 


Figure  11.2  Summary  of  factors  and  effects  of  external  feedback.  (From  Narciss,  S .,  Informatives  tutorielles  Feedback.  Entwicklungs- 
und  Evaluationsprinzipien  auf  der  Basis  instruktionspsychologischer  Erkenntnisse  (Informative  Tutoring  Feedback ),  Waxmann, 
Munster,  2006.  With  permission.) 


•  Under  these  individual  and  situational  con¬ 
ditions,  what  cognitive,  metacognitive,  and 
motivational  effects  do  the  various  feedback 
components  or  strategies  have? 

•  When  are  these  effects  expected  to  occur, 
and  what  is  their  expected  duration? 

Figure  11.2  summarizes  these  considerations  regard¬ 
ing  requirements  for  and  the  effects  of  various  kinds 
of  external  feedback. 
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ABSTRACT 

Design  studies  provide  evidence  for  the  effectiveness 
of  specific  supports  for  learning  in  technology- 
enhanced  environments  and  suggest  guidelines  for  the 


design  and  use  of  such  features.  The  Design  Principles 
Database  is  a  public  collaborative  knowledge -building 
tool  that  helps  capture  and  synthesize  this  knowledge 
using  “design  principles”  as  a  basic  construct.  In  this 
chapter,  we  highlight  eight  pragmatic  design  principles 
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from  the  Design  Principles  Database  that  are  most 
likely  to  support  learning,  and  we  provide  evidence 
that  shows  how  learning  is  supported  by  features  in 
technologies  that  apply  these  principles.  We  discuss 
the  advantages  and  limitations  of  design  principles  to 
guide  a  design  process  and  suggest  that,  for  design 
principles  to  be  more  effective  for  guiding  new  inno¬ 
vations,  they  should  be  complemented  with  a  design 
patterns  approach. 


KEYWORDS 

Design  principles:  Research-based  guidelines  for 
instructional  design;  design  principles  can  be  artic¬ 
ulated  at  different  grain-sizes:  specific  principles 
characterize  rationales  for  designing  specific  fea¬ 
tures  in  a  learning  environment,  pragmatic  princi¬ 
ples  connect  rationales  behind  several  features,  and 
meta-principles  synthesize  a  cluster  of  pragmatic 
principles. 

Knowledge  integration:  The  process  of  adding,  distin¬ 
guishing,  organizing,  and  evaluating  accounts  of 
phenomena,  situations,  and  abstractions. 

Learning  environment:  A  system  that  incorporates  a 
set  of  features  including  a  navigation  system;  learn¬ 
ing  environments  can  deliver  curricula  in  any  topic 
area. 

Software  features :  Specific  applications  of  technology 
intended  to  advance  learning;  features  include 
designed  artifacts  such  as  modeling  tools,  simula¬ 
tions,  micro- worlds,  visualizations,  collaboration 
tools,  reflection  prompts,  games,  and  embedded 
assessments. 


INTRODUCTION 

This  chapter  synthesizes  the  benefits  of  technology- 
enhanced  supports  for  inquiry  learning  and  is  intended 
to  help  designers  build  on  past  work  and  help  research¬ 
ers  report  new  findings  in  the  context  of  current  work. 
To  achieve  these  goals,  we  take  advantage  of  efforts 
to  collate  design  principles,  such  as  those  devised  by 
Brown  (1992),  Kali  (2006),  Kali  (in  press),  Merrill 
(2002),  Quintana  et  al.  (2004),  Reigeluth  (1999),  and 
van  den  Akker  (1999).  We  draw  on  current  views  of 
the  learner  informed  by  research  in  cognition  and 
instruction  (Bransford  et  al.,  1999;  Linn  et  al.,  2004b). 

Designers  have  created  and  refined  numerous  sup¬ 
ports  for  inquiry  learning.  We  define  support  strategies 
as  features  of  technology-enhanced  instruction  that  the 
designers  report  contribute  to  learning  such  as  guid¬ 
ance  ,  prompts  for  reflection ,  and  varied  representations 


of  content.  We  define  inquiry  broadly  to  refer  to  any 
activity  that  engages  the  learner  in  exploring  a  scien¬ 
tific  phenomenon.  These  supports  are  embedded  in 
many  technology-enhanced  inquiry  programs  that 
have  been  tested  in  classrooms  (Barab  et  al.,  2000; 
Bruer,  1993;  diSessa,  2000;  Edelson  et  al.,  1999;  Kra- 
jcik  et  al.,  1998;  Linn  et  al.,  2003;  Means,  1994;  Reiser 
et  al.,  2001;  Roschelle  et  al.,  2000;  Scardamalia  and 
Bereiter,  1996;  Schwartz  et  al.,  1999;  Songer  et  al., 
2003;  Tinker,  2005;  White  and  Frederiksen,  1998). 

Studies  of  inquiry  learning  show  that  learners  grap¬ 
ple  with  multiple,  conflicting,  and  often  confusing 
ideas  about  scientific  phenomena.  Successful  supports 
for  inquiry  help  learners  identify  their  repertoire  of 
ideas  by  engaging  them  in  multiple,  often  authentic 
problem  situations.  Successful  technology-enhanced 
instruction  often  takes  advantage  of  models,  simula¬ 
tions,  or  visualizations  to  introduce  new  ideas.  Effec¬ 
tive  instruction  does  not  stop  with  eliciting  and  adding 
ideas,  however.  To  gain  durable,  generative  under¬ 
standing,  students  need  to  devise  criteria  for  sorting 
out  these  ideas  in  varied  contexts  and  at  multiple  levels 
of  analysis.  They  need  supports  that  allow  them  to 
formulate,  often  in  collaboration  with  others,  more  and 
more  nuanced  criteria  for  evaluating  ideas.  They  need 
time  to  sort  out  their  ideas  and  develop  an  increasingly 
linked  set  of  views  about  any  phenomenon.  In  this 
chapter,  we  synthesize  recent  research  using  technol¬ 
ogy-enhanced  curriculum  materials  to  support  inquiry. 
We  identify  promising  supports  for  inquiry  that 
emphasize  all  four  processes:  eliciting  ideas,  adding 
ideas,  developing  criteria,  and  sorting  out  ideas.  These 
processes  are  the  main  elements  of  the  knowledge 
integration  framework  (Linn,  1995;  Linn  and  Hsi, 
2000;  Linn  et  al.,  2004b). 

Designers  often  find  building  on  past  successes 
challenging  because  studies  do  not  tease  apart  the  con¬ 
sequential  elements  of  the  innovation  (Cognition  and 
Technology  Group  at  Vanderbilt,  1997).  We  seek  ways 
to  facilitate  effective  use  of  technology  in  science 
learning.  In  spite  of  extensive  research  on  supports  for 
inquiry  learning,  most  uses  of  technology  for  precol- 
lege  instruction  use  drill  and  practice,  word  processing, 
and  Web  surfing  (Fishman  et  al.,  2004).  University 
instruction  primarily  relies  on  Web  delivery  of  infor¬ 
mation  (Herrington  et  al.,  2005).  Mioduser  and  col¬ 
leagues  (1999,  p.  757)  summarized  the  current  uses  of 
technology  in  education  as:  “One  step  ahead  for  the 
technology,  two  steps  back  for  the  pedagogy.” 

To  identify  promising  elements  of  supports  for 
inquiry,  we  synthesize  findings  from  design  research. 
Design  researchers  conduct  iterative  refinements  to 
develop  successful  innovations  (Bell  et  al.,  2004; 
Design-Based  Research  Collective,  2003;  Simon, 
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1969).  Studies  comparing  alternative  designs  or 
sequences  of  refinements  provide  evidence  for  the 
effectiveness  of  specific  supports,  shed  light  on  the 
mechanism  behind  supports,  and  suggest  guidelines 
for  implementation.  These  studies  often  summarize 
findings  in  design  principles,  learning  principles,  pat¬ 
terns,  and  related  synthesis  methods  to  capture  both 
the  innovations  and  the  mechanisms  that  govern  their 
success.  Brown  (1992)  offered  learning  principles  to 
synthesize  her  research  findings.  Collins  (1992)  called 
for  guidelines  to  capture  research-based  practical 
design  knowledge.  These  efforts  echo  practices  in 
other  design-based  fields  that  have  found  principles 
helpful,  including  architecture  (Alexander  et  al., 
1977),  graphical  communication  (Tufte,  1983),  and 
computer  science  (Gamma  et  al.,  1995). 

The  current  synthesis  starts  with  features  of  inquiry 
innovations  captured  by  Kali  (2006)  and  Kali  (in  press) 
in  the  Design  Principles  Database  (http://www.design- 
principles.org).  The  current  entries  in  the  Design  Prin¬ 
ciples  Database  represent  the  contributions  of  over  50 
individual  researchers.  The  database  includes  about 
100  features  (mainly  from  physical,  life,  and  earth 
sciences).  The  database  connects:  (1)  descriptions  of 
promising  features,  (2)  the  rationale  for  the  feature, 
and  (3)  evidence  for  the  impact  of  the  feature  to  prag¬ 
matic  design  principles.  Pragmatic  principles  are 
abstracted  guidelines  that  connect  similar  rationales 
behind  features  in  different  learning  environments. 
Although  features  are  entered  in  the  database  as  inde¬ 
pendent  entities,  they  are  often  parts  of  sequences  of 
features  that  comprise  a  learning  environment. 

The  database  is  organized  around  meta- principles. 
Meta-principles  are  overarching  ideas  that  synthesize 
a  cluster  of  pragmatic  principles.  The  meta- principles 
in  the  database  include  make  thinking  visible,  make 
science  accessible,  help  learners  learn  from  each  other, 
and  promote  autonomous  lifelong  learning. 

The  structure  of  the  Design  Principles  Database 
emerged  from  longitudinal  research  on  technology- 
enhanced  science  learning  (Linn  and  Hsi,  2000;  Linn 
et  al.,  2004b).  The  Computer  as  Learning  Partner 
research  program  identified  the  4  meta-principles  and 
the  first  14  pragmatic  principles  in  their  20-year- long 
effort  to  iteratively  refine  effective  interactive  science 
experiences  (Linn  and  Hsi,  2000).  The  Design  Princi¬ 
ples  Database  has  grown  with  contributions  from  par¬ 
ticipants  in  workshops,  from  course  activities,  and 
from  the  public  (Kali  et  al.,  2002).  It  serves  as  a  col¬ 
laborative  knowledge-building  tool  for  communities 
who  design  and  explore  educational  technologies 
(Kali,  2006).  The  Design  Principles  Database  enables 
designers  to  explain  the  pedagogical  rationales  behind 
each  feature  in  a  learning  environment  and  for  com¬ 


munity  members  to  respond  and  add  their  experiences. 
It  is  based  on  the  idea  that  explaining  the  rationale  of 
a  feature  can  be  useful  for  other  designers.  Researchers 
have  added  additional  principles  (Linn  et  al.,  2004b) 
and  applied  the  ideas  to  design  of  assessments  (Clark 
and  Linn,  2003),  professional  development  (Williams 
and  Linn,  2003),  and  learning  environments  (Linn  et 
al.,  2003).  Researchers  can  explore  the  application  of 
principles  in  new  contexts  and  add  their  findings  back 
to  the  Design  Principles  Database.  The  design  knowl¬ 
edge  grows  as  principles  are  debated,  refined,  or  war¬ 
ranted  with  additional  field-based  evidence. 


SUPPORTS  FOR  INQUIRY  LEARNING 

To  synthesize  supports  for  inquiry  learning,  we  start 
with  the  four  meta-principles  in  the  Design  Principles 
Database  and  select  pragmatic  principles  that  connect 
with  the  largest  number  of  software  features.  We  have 
highlighted  two  promising  features  for  each  pragmatic 
principle.  The  features  vary  in  their  grain  size;  some 
represent  whole  learning  environments  (such  as 
Model-It) ,  some  represent  tools  in  a  learning  environ¬ 
ment  (such  as  the  inquiry  map  in  WISE),  and  others 
represent  elements  in  software  (such  as  the  manipula¬ 
tive  animated  three-dimensional  illustrations  in 
Geo3D).  Table  12.1  shows  the  features  and  their  con¬ 
nections  with  pragmatic  and  meta-principles.  We 
describe  the  meta-principles,  associated  pragmatic 
principles,  and  illustrative  features  to  characterize  sup¬ 
ports  for  inquiry  learning. 

Meta-Principle:  Make  Science  Accessible 

Designers  seek  to  make  science  accessible  for  learners 
to  elicit  the  full  repertoire  of  ideas.  They  create  sup¬ 
ports  that  increase  the  relevance  of  science  for  all 
learners  — those  aspiring  to  careers  in  science  and  those 
taking  their  last  science  course.  These  supports 
respond  to  the  common  complaint  that  science  is  not 
relevant  or  useful.  They  also  remedy  the  often  inade¬ 
quate  images  of  science  held  by  students  (Hofer  and 
Pintrich,  2002).  Two  pragmatic  principles  that  follow 
this  meta-principle  call  for  communicating  the  diver¬ 
sity  of  science  inquiry  and  for  connecting  to  personally 
relevant  examples. 

Pragmatic  Principle:  Communicate 
the  Diversity  of  Science  Inquiry 

This  principle  calls  on  designers  to  expose  learners  to 
the  rich  diversity  of  the  inquiry  process.  Ear  too  often 
students  leave  science  class  with  an  image  of  inquiry 
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TABLE  12.1 

Features  Described  in  This  Chapter  and  Their  Connections  to  Pragmatic  and  Meta-Principles 


Pragmatic  Principles 

Meta-Principle:  Make  Science  Accessible 
Communicate  the  diversity  of  science  inquiry 

Connect  to  personally  relevant  examples 

Meta-Principle:  Make  Thinking  Visible 
Provide  students  with  templates  to  organize  ideas 

Provide  knowledge  representation  tools 

Enable  three-dimensional  manipulation 

Meta-Principle:  Help  Learners  Learn  from  Each  Other 
Encourage  learners  to  learn  from  others 

Meta-Principle:  Promote  Autonomous  Lifelong  Learning 
Enable  manipulation  of  factors  in  models  and  simulation 

Encourage  reflection 


Features 

Inquiry  map  in  WISE 
SenseMaker  in  WISE 
Authentic  contexts  in  the  Jasper  project 
Contextualized  definition  in  TELS 

Principle  Maker  in  WISE 

Design  rule-of-thumb  template  in  SMILE 

Model-It 

Causal  Mapper 

Three-dimensional  illustrations  in  Geo3D 

Scaffolds  to  support  student  use  of  molecular  modeling  software 

Automated  gathering  of  peer-evaluation  outcomes  in  CeLS 
Supports  for  collaboration  in  eStep 

Manipulable  models  of  molecules  in  Molecular  Workbench 
Modeling  derivatives  in  Visual  Mathematics 
Prompts  for  reflection  on  action  in  CASES 
Note  taking  in  WISE 


as  dogmatic  and  inflexible  or  abstract  and  incompre¬ 
hensible  (Linn  et  al.,  2004a).  Technology-enhanced 
learning  environments  can  help  students  become  aware 
of  the  diversity  of  science  inquiry  by  engaging  students 
in  a  variety  of  inquiry  processes.  We  exemplify  how 
this  principle  is  applied  in  two  different  features  that 
are  part  of  the  Web-Based  Inquiry  Science  Environ¬ 
ment  (WISE)  (Slotta,  2004). 


Inquiry  Map  in  WISE 

To  help  students  explore  inquiry  processes,  WISE  uses 
an  inquiry  map,  which  is  a  dynamic-graphic  guide 
shown  in  each  of  the  WISE  projects  (Figure  12.1).  It 
graphically  represents  the  steps  of  the  inquiry  in  the 
project.  This  enables  students  to  get  an  overview  of  the 
project  and  the  inquiry  strategies  it  includes.  The  inquiry 
map  also  expands  and  collapses  each  project  into  its 


Figure  12.1  WISE  inquiry  map  (in  middle)  with  index  of  activities  (left)  and  reelection  note  (right). 
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Figure  12.2  Sorting  of  evidence  Web  pages  in  SenseMaker. 

main  inquiry  components.  Teachers  using  WISE  indi¬ 
cate  that  the  map  makes  students  more  independent  in 
their  inquiry  activities,  thus  strengthening  their  under¬ 
standing  of  the  diversity  of  inquiry  processes.  In  many 
cases,  the  inquiry  map  provides  answers  to  “What  do  I 
do  now?”  questions,  making  the  inquiry  process  more 
self-directed.  The  map  also  helps  students  get  a  better 
understanding  of  how  each  step  in  the  project  relates  to 
the  whole  inquiry  process  (Linn  and  Hsi,  2000). 

SenseMaker  in  WISE 

Another  feature  that  communicates  the  idea  of  diver¬ 
sity  in  science  inquiry  engages  students  in  controver¬ 
sial  scientific  debates  (Bell,  2004).  The  WISE  Sense- 
Maker  tool  (Figure  12.2)  helps  students  figure  out  the 
relationships  that  exist  between  different  Web 
resources.  As  they  investigate  pieces  of  Web-based 
evidence,  students  organize  the  items  into  categories 
in  SenseMaker.  This  sorting  of  pieces  of  evidence 
helps  students  consolidate  their  own  stance  about  the 
controversy  and  prepare  for  a  class  debate.  During  the 
debate,  the  graphical  representations  of  student  argu¬ 
ments  are  displayed.  Students  can  see  the  diversity  of 
inquiry  strategies  by  comparing  their  arguments  to 
those  of  other  students  and  of  experts  (Bell,  2004). 
SenseMaker  can  support  debates  about  such  issues  as 
the  threat  of  malaria.  Students  explore  the  ethical 


trade-offs  between  protecting  human  life  (by  spraying 
DDT)  and  protecting  wildlife  and  the  environment  (by 
banning  the  use  of  DDT)  (Seethaler  and  Linn,  2004). 
Research  shows  that  engaging  students  in  such  debates 
and  supporting  their  inquiry  process  with  SenseMaker 
can  help  students  develop  a  more  integrated  under¬ 
standing  of  complex  science  topics  (Bell,  2004). 

Pragmatic  Principle:  Connect  to 
Personally  Relevant  Examples 

Personally  relevant  problems,  such  as  determining  how 
to  keep  a  drink  cold,  make  science  accessible  because 
they  elicit  intuitive  ideas  to  fuel  inquiry  (Linn  and  Hsi, 
2000;  Songer  and  Linn,  1991).  Linn  et  al.  (2004b) 
showed  that  eliciting  the  broad  range  of  student  ideas 
about  science  and  engaging  these  ideas  enables  stu¬ 
dents  to  build  more  coherent,  durable  scientific  views. 
This  principle  advocates  using  personally  relevant 
examples  as  contexts  for  scientific  inquiry.  The  exam¬ 
ples  below  show  how  this  principle  was  applied  by 
two  research  groups. 

Authentic  Contexts  in  the  Jasper  Project 

The  Jasper  project,  using  the  anchored  instruction 
approach  (Cognition  and  Technology  Group  at  Vander¬ 
bilt,  1990, 1997),  was  one  of  the  earliest  large  endeavors 
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All  objects  v  become  the  same  temperature  as  each  other  v 
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Figure  12.3  Scaffolding  students  in  the  creation  of  principles  with  Principle  Maker. 


to  use  technology  to  anchor  instruction  in  authentic 
contexts.  Jasper  includes  a  set  of  12  video-based  adven¬ 
tures  that  focus  on  mathematical  problem  solving.  Each 
video  ends  in  a  complex  challenge.  The  adventures  are 
designed  like  good  detective  novels,  and  all  the  data 
necessary  to  solve  the  adventure  are  embedded  in  the 
story.  The  Jasper  adventures  present  a  believable  story 
that  has  interesting  characters,  a  complex  and  important 
challenge,  and  extensions  to  a  variety  of  curricular 
areas.  To  solve  the  challenge,  the  students  combine 
problem-solving  skills,  mathematics  concepts,  and  the 
information  in  the  video.  The  adventures  were  designed 
to  bridge  the  gap  between  everyday  and  school  prob¬ 
lems.  They  provide  a  common  context  for  instruction, 
an  authentic  task,  and  a  chance  to  see  that  school 
knowledge  can  be  used  to  solve  real  problems. 

Contextualized  Definitions  in  TELS  Modules 

The  Technology  Enhanced  Learning  in  Science  (TELS) 
Hanging  with  Friends,  Velocity  Style!  module  (Tate, 
2005)  embeds  scientific  terms  in  the  context  of  an  inter¬ 
view  with  a  teenager.  The  purpose  of  the  interview  is  to 
find  the  teenager’s  velocity,  but  the  context  of  the  inter¬ 
view  is  her  trip  from  Lake  Park  to  the  movie  theater  to 
meet  her  friends.  The  interviewee  speaks  in  everyday 
language  while  communicating  the  information  needed 
to  determine  her  velocity,  saying,  for  example,  “I  was 
running  a  bit  late  and  almost  didn’t  get  a  seat.  I  arrived 
at  the  movie  theatre  at  5:05  p.m.  This  is  referred  to  as 
my  final  time.”  The  discourse  blends  everyday  language 
and  events  (being  late  to  a  movie)  with  information 
needed  to  determine  velocity.  By  bringing  everyday 
events  (a  conversation  or  interview)  into  play,  the  feature 
draws  students  into  the  activity  and  helps  them  place  it 
in  a  familiar  context.  They  compute  the  velocity  of  each 
friend  to  see  if  all  will  arrive  in  time.  The  feature  moti¬ 
vates  students  to  understand  the  specific  terms  and  data 
needed  to  compute  velocity  for  an  everyday  event. 


Meta-Principle:  Make  Thinking  Visible 

To  promote  inquiry,  designers  often  encourage  students 
and  teachers  to  make  their  thinking  visible.  When  stu¬ 
dents  make  thinking  visible  ,  they  can  inspect  their  own 
knowledge  integration  processes  and  deliberately  guide 
their  learning  (Bransford  et  al.,  1999;  Collins  et  al., 
1991;  Linn,  1995).  To  support  these  processes,  design¬ 
ers  create  tools  that  students  use  to  map  their  ideas  and 
externalize  their  thoughts  at  different  stages  of  the 
learning  process.  Designers  also  use  models  or  visual¬ 
izations  embedded  in  inquiry  projects  to  make  complex 
concepts  and  scientific  phenomena  visible.  We  high¬ 
light  three  pragmatic  principles  that  follow  this  meta¬ 
principle.  The  first  two  are  intended  to  help  students 
make  their  own  thinking  visible,  and  the  third  is 
intended  to  make  complex  scientific  phenomena  visi¬ 
ble.  We  exemplify  each  pragmatic  principle  with  two 
features  from  different  contexts. 

Pragmatic  Principle:  Provide  Students 
with  Templates  to  Organize  Ideas 

To  support  students  in  articulating  complex  scientific 
ideas,  designers  have  created  what  might  be  called 
templates.  Templates  scaffold  students  in  representing 
their  ideas  and  revising  them  as  they  complete  complex 
activities  (Kolodner  et  al.,  2004).  Below  are  two  exam¬ 
ples  that  show  software  features  from  two  contexts  that 
apply  this  principle. 

Principle  Maker  in  WISE 

One  feature  that  exemplifies  how  templates  can  help 
organize  ideas,  is  the  Principle  Maker  (Clark  and 
Sampson,  2007).  The  Principle  Maker  (Ligure  12.3)  is 
a  tool  in  the  WISE  environment  that  helps  students 
synthesize  data  that  they  have  collected  or  experienced 
into  a  principle.  By  providing  building-block  phrases, 
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Figure  12.4  Design  rule-of-thumb  template  in  SMILE. 


the  tool  scaffolds  the  task  and  gives  students  clear 
alternatives  without  dictating  ideas.  The  Principle 
Maker  is  part  of  a  TELS  project  called  Thermodynam¬ 
ics:  Probing  Your  Surroundings.  Research  conducted 
by  Clark  and  Sampson  (2007)  suggests  that  scaffolding 
students  in  the  creation  of  principles  helps  make  stu¬ 
dent  ideas  explicit.  Clark  and  Sampson  take  advantage 
of  the  principles  students  build  to  set  up  discussions 
that  include  groups  with  opposing  ideas.  They  argue 
that  this  process  promotes  dialogical  argumentation. 
By  making  thinking  visible,  the  Principle  Maker 
enables  a  more  sophisticated  form  of  argumentation 
than  is  found  in  typical  science  classrooms.  As  a  result, 
students  have  a  good  sense  of  the  views  of  their  peers 
and  can  spend  their  time  supporting,  evaluating,  and 
critiquing  ideas. 

Design  Rule  of  Thumb  Template  in  SMILE 

Another  feature  designed  according  to  this  principle 
is  part  of  the  Learning  by  Design  classrooms,  and  the 
Supporting  Multiuser  Interactive  Learning  Environ¬ 
ment  (SMILE)  (Kolodner  et  al.,  2004).  This  feature 
assists  students  in  generating  and  revising  design 
rules  of  thumb  throughout  a  project  experience. 
Design  rules  of  thumb  are  lessons  that  are  learned 
from  experience.  The  template  includes  constructs 
that  help  students  construct  a  design  rule  of  thumb  in 


the  following  format:  When/If  (describe  the  action, 
design,  or  choice  you  are  working  within)  use/con¬ 
nect/build/employ/measure  (list  your  suggestion  or 
method)  because  (list  or  supply  the  evidence  or  sci¬ 
ence  principle  or  concept  that  backs  up  your  sugges¬ 
tion)  (Ligure  12.4).  Students  initially  attempt  to  gen¬ 
erate  these  rules  of  thumb  in  small  groups  based  on 
their  experimental  results  or  on  cases  they  are  read¬ 
ing.  They  discuss  the  rules  of  thumb  as  a  class  and 
revise  them.  Ideally,  students  notice  ideas  they  cannot 
explain  and  identify  the  science  they  need  to  learn. 
Research  shows  that,  before  use  of  the  template,  stu¬ 
dents  were  often  unable  to  make  the  appropriate  con¬ 
nections  to  science.  When  templates  were  used  in  the 
context  of  a  class,  teachers  were  better  able  to  intro¬ 
duce  the  appropriate  scientific  concepts.  When  the 
teacher  helped  students  create  rules  of  thumb  as  a 
class  before  using  the  software,  students  using  the 
software  created  better  rules  of  thumb  (with  a  richer 
situation  description  and  justification)  than  students 
who  did  not  have  the  template  available  in  the  soft¬ 
ware  (Kolodner  et  al.,  2004).  This  principle  calls  for 
providing  learners  with  tools  with  which  they  can 
visually  represent,  at  different  learning  stages,  their 
understanding  of  scientific  ideas.  Linn  et  al.  (2004b) 
claim  that  knowledge  representation  tools  can  pro¬ 
mote  interpretation  and  theorizing  about  evidence. 
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Figure  12.5  Running  a  simulation  built  by  students  in  Model-It. 


Pragmatic  Principle:  Provide 
Knowledge  Representation  Tools 

Model-It 

An  example  of  a  tool  that  enables  students  to  represent 
and  test  their  knowledge  is  Model-It,  developed  at  the 
University  of  Michigan.  Model-It  is  a  learner-centered 
tool  for  building  dynamic,  qualitative-based  models. 
Model-It  was  designed  to  support  students,  even  those 
with  only  very  basic  mathematical  skills,  in  building 
dynamic  models  of  scientific  phenomena  and  running 
simulations  with  their  models  to  verify  and  analyze 
the  results  (Jackson  et  al.,  2000).  For  example,  students 
can  build  models  of  water  quality  and  then  test  how 
various  pollutants  would  affect  water  quality.  Model- 
It  provides  an  easy-to-use  visual  structure  with  which 
students  can  plan,  build,  and  test  their  models  (Figure 
12.5).  Model-It  has  been  used  with  thousands  of  stu¬ 
dents  and  their  teachers  in  both  urban  and  suburban 
areas.  Research  shows  that  when  properly  integrated 
into  the  curriculum,  Model-It  allows  students  to  take 
part  in  a  variety  of  scientific  practices  such  as  testing, 
debugging,  building  relationships,  specifying  vari¬ 
ables,  and  synthesis  (Jackson  et  al.,  2000). 

Causal  Mapper 

Another  example  of  a  feature  that  enables  students  to 
represent  their  understanding  is  Causal  Mapper.  This 
feature,  developed  by  Baumgartner  (2004),  is  a  stand¬ 
alone  application  that  allows  learners  to  make  sense 
of  a  set  of  causal  relationships.  Causal  mapping  refers 
to  the  use  of  directed  node  and  link  graphs  — similar 


to  concept  maps,  in  some  ways  — to  represent  a  set  of 
causal  relationships  within  a  system.  The  causal  map 
shown  in  Figure  12.6,  for  example,  reflects  the  repre¬ 
sentation  by  two  sixth-grade  girls  of  the  factors  that 
contribute  to  the  health  of  a  stream.  Causal  mapping 
is  more  structured  than  concept  mapping,  in  that  links 
capture  causal  relations.  Students  can  develop  a  shared 
representation  for  causality,  and  groups  can  quickly 
examine  and  critique  each  other’s  causal  maps  and 
discuss  complex  causal  chains.  Baumgartner  (2004) 
showed  that,  when  students  map  their  own  data  using 
Causal  Mapper,  they  develop  their  ability  to  interpret 
their  data  and  use  it  as  evidence  for  their  investigations. 

Pragmatic  Principle:  Enable 
Three-Dimensional  Manipulation 

In  the  earlier  description  of  the  “make  thinking  visible” 
meta-principle,  we  mentioned  that  one  aspect  is  ani¬ 
mating  complex  scientific  phenomena.  “Enable  three- 
dimensional  manipulation”  is  a  pragmatic  principle  that 
emphasizes  making  scientific  phenomena  visible. 
Many  students  have  difficulty  perceiving  three-dimen¬ 
sional  (3D)  structures,  that  are  presented  in  textbooks 
as  two-dimensional  (2D)  representations.  Technology 
can  provide  tools  that  enable  students  to  manipulate 
representations  of  these  structures.  Visualizations  can 
enable  students  to  rotate  objects  being  studied  and  thus 
view  them  from  various  directions  (Dori  et  al.,  2003; 
Hsi  et  al.,  1997;  Kali  et  al.,  1997).  Other  types  of  3D 
visualizations  can  also  improve  understanding,  as 
described  in  the  following  two  features: 
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Figure  12.6  Map  created  by  students  with  Causal  Mapper  to  represent  water-quality  factors. 


Three-Dimensional  Illustrations  in  Geo3D 

Geo3D  was  designed  to  respond  to  the  spatial  abilities 
required  in  structural  geology  and  the  difficulties  that 
high-school  students  have  in  the  perception  of  geolog¬ 
ical  structures  (Kali  and  Orion,  1996).  In  Geo3D,  stu¬ 
dents  can  visually  bisect  illustrations  of  geological 
structures  (see  Figure  12.7).  They  explore  relation¬ 
ships  between  observable  and  unseen  properties  of  the 
geological  structures.  These  relationships  strengthen 
the  perceptions  of  geological  structures  created  by 
folding,  uplifting,  and  erosion  (Kali  et  al.,  1997).  Even 
short  interactions  with  these  animations  ( 1  to  2  hours) 
improves  students’  skills  in  the  visualization  of  geo¬ 
logical  structures. 


Scaffolds  to  Support  Student  Use 
of  Molecular  Modeling  Software 

Many  students  have  difficulties  relating  symbolic  rep¬ 
resentation  of  molecules  to  2D  and  3D  models,  espe¬ 
cially  when  organic  compounds  are  involved.  Dori  et 
al.  (2003)  and  Barak  and  Dori  (2005)  designed  a  suite 
of  activities  that  takes  advantage  of  molecular  model¬ 
ing  software  originally  designed  for  experts,  such  as 
WebLab  Viewer  and  ISIS/Draw.  To  use  software 
designed  for  experts,  learning  materials  must  highlight 
the  salient  information  for  students  (Edelson  et  al., 
1999).  Guided  by  this  suit  of  activities,  students  con¬ 
struct  2D  representations  of  chemical  substances  using 
ISIS/Draw  and  then  use  WebLab  to  transform  the  2D 


Figure  12.7  Animation  showing  bisection  of  a  geological  structure  in  Geo3D. 
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Figure  12.8  Ball-and-stick  model  created  by  a  student  using  the  scaffolding  activities  for  the  WebLab  tool. 


representations  into  a  3D  image  (framework,  ball-and- 
stick,  or  space-filling)  (Figure  12.8).  Students  compare 
their  representations  to  those  of  their  peers.  Dori  et  al. 
(2006)  showed  that  these  activities  increase  students’ 
understanding  of  the  physical  and  chemical  properties 
of  simple  and  complex  compounds. 

Meta-Principle:  Help  Learners 
Learn  from  Each  Other 

To  help  students  develop  criteria  and  distinguish 
among  ideas,  designers  embed  social  supports  in 
inquiry  activities.  These  opportunities  encourage  stu¬ 
dents  to  listen  and  learn  from  others  and  take  advan¬ 
tage  of  the  collective  knowledge  in  the  classroom  com¬ 
munity.  Encouraging  students  to  analyze  and  build  on 
ideas  from  peers  can  introduce  new  perspectives  and 
motivate  students  to  form  criteria  (Scradamalia  and 
Bereiter,  1994).  Additionally,  when  students  interact, 
they  bring  to  light  the  alternative  views  held  by  learn¬ 
ers  and  the  criteria  used  to  interpret  ideas  (Bransford 
et  al.,  1999).  We  highlight  one  pragmatic  principle  that 
follows  this  meta-principle  and  describe  two  features 
that  show  its  use  in  different  contexts. 

Pragmatic  Principle:  Encourage 
Learners  to  Learn  from  Others 

This  principle  emphasizes  helping  learners  to  listen 
and  learn  from  others.  When  students  explain  their 
thoughts  to  other  students,  they  sort  out  their  own  ideas 


and  learn  new  ideas  from  others.  Students  can  help 
their  peers  understand  an  idea  by  articulating  concepts 
using  familiar  vocabulary  and  relevant  examples. 

Automated  Gathering  of  Peer-Evaluation 
Outcomes  in  CeLS 

One  example  of  this  principle  involves  automating 
peer  evaluation.  CeLS  (Collaborative  e-Learning 
Structures;  http://www.mycels.net)  allows  instructors 
to  construct  online  structured  collaborative  activities, 
including  peer  evaluation.  CeLS  automatically  gathers 
and  analyzes  information  submitted  by  students  and 
shows  it  in  various  customizable  forms.  Figure  12.9 
shows  an  example  of  the  type  of  information  that  can 
be  presented  in  a  peer  evaluation  activity  designed  in 
CeLS,  including  statistical  analysis,  a  histogram,  and 
a  collection  of  student  justifications  for  their  grading 
(presented  anonymously).  Kali  and  Ronen  (2005)  used 
a  peer-evaluation  activity  designed  with  CeLS  in  a 
philosophy  of  education  course.  Undergraduate  stu¬ 
dents  constructed  a  conceptual  model  of  their  “ideal 
school”  and  developed  more  sophisticated  epistemol¬ 
ogies  as  a  result  of  peer  evaluation. 

Supports  for  Collaboration  in  eStep 

Another  example  of  technology  supports  that  encour¬ 
age  learners  to  learn  from  others  is  the  eStep  system 
(Derry  et  al.,  2005;  Hmelo-Silver  et  al.,  2005).  In  eStep, 
learners  read  and  view  a  case  study  that  presents  a 
classroom  dilemma.  They  individually  reflect  on  the 
dilemma  and  propose  an  initial  solution,  then  they 
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I  I  my  Activities  |  my  Groups  |  Activity  Structures  |  Sample  Activities  |  New  Activity  |  J 

Peer  Evaluation  for  Group  f(S:  “Ideal  School”  Presentation 


Sample  size 

40 

Average 

5.03 

Standard  Dev.  1 .04 

|  Median 

5  | 

Poor  (1)  (2)  (3)  (4)  (5)  (6)  (7)  Excellent 

2  0  0  2  9  17  8 

5.2%  0%  0%  5.2%  23%  44  6%  21% 


Student 

Score 

Justification 

1 

6 

The  rationale  behind  most  of  the  activities  in  this  school  was 
presented  in  a  very  clear  manner.  1  didt  understand  the  rationale 
for  having  election  days  every  other  week 

The  activities  were  illustrated  beautifully.  One  could  get  a  feeling  of 
how  this  works.  The  dialogue  between  students  was  very  helpful  in 

Figure  12.9  Peer  evaluation  activity  designed  with  CeLS. 


collaborate  with  other  learners  to  collectively  arrive  at 
a  revised  solution.  The  lesson  ends  with  individual  cri¬ 
tiques  of  the  group  solution  and  reflection  on  the  learn¬ 
ing,  collaboration,  lesson  design,  and  usefulness  of  the 
solution  to  their  own  professional  practice.  Derry  et  al. 
(2005)  reported  that  eStep  produced  significant 
increases  in  teacher  and  learner  abilities  to  think  deeply 
about  student  understanding  and  that  the  course  was 
more  effective  at  producing  transfer  than  a  traditional 
lecture-based  approach  covering  the  same  material. 

Meta-Principle:  Promote 
Autonomous  Lifelong  Learning 

To  become  lifelong  learners,  students  need  supports 
that  help  them  guide  their  own  learning,  recognize  new 
ideas,  and  develop  a  view  of  effective  inquiry.  They 
need  to  engage  in  sustained  project  work  so  they  can 
connect  personally  relevant  problems  to  class  topics 
and  reflect  on  experience  using  a  robust  inquiry  pro¬ 
cess  in  diverse  contexts  (Linn  et  al.,  2004b).  Students 
benefit  from  learning  to  monitor  their  progress.  To 
encourage  autonomy,  designers  scaffold  comprehen¬ 
sive  inquiry  processes  that  students  can  apply  to  varied 
problems,  both  in  class  and  throughout  their  fives,  to 
explore  ways  to  ensure  that  these  practices  are  inter¬ 
nalized.  We  present  two  pragmatic  principles  from  the 
Design  Principles  Database  that  apply  this  meta-prin¬ 
ciple  and  exemplify  each,  with  two  features. 


Pragmatic  Principle:  Enable  Manipulation 
of  Factors  in  Models  and  Simulations 

Interactive  models,  simulations,  and  visualizations  sup¬ 
port  autonomy  but  often  frustrate  learners  because  they 
are  too  complex  or  too  sophisticated  (Hegarty  et  al., 
1999).  To  enable  students  to  benefit  from  models,  sim¬ 
ulations,  and  visualizations,  designers  guide  interac¬ 
tions  and  seek  ways  to  promote  autonomy.  Models, 
simulations,  and  visualizations  enable  learners  to  con¬ 
nect  everyday,  microscopic,  and  symbolic  representa¬ 
tions  of  phenomena.  They  can  be  used  in  virtual  labs 
when  it  is  impossible,  dangerous,  difficult,  expensive, 
or  unethical  (in  the  case  of  animal  studies)  to  conduct 
a  hands-on  experiment.  They  can  illustrate  many  fields, 
such  as  finance,  mathematics,  physics,  meteorology, 
biology,  or  social  sciences.  Shternberg  and  Yerushalmy 
(2003)  distinguished  between  models  that  illustrate 
concepts  (such  as  the  relationship  between  a  function 
and  its  derivative)  and  models  of  physical  phenomena 
such  as  chemical  reactions.  In  both  types,  students  need 
strategies  for  exploring  how  the  model  behaves  under 
varied  conditions.  Many  computer-based  models  allow 
learners  to  explore  the  effect  of  each  variable  in  a  sys¬ 
tem  by  holding  others  constant.  Oftentimes,  instruc¬ 
tional  materials  help  learners  internalize  the  strategies 
appropriate  for  exploring  complex  models  and  simula¬ 
tions.  Lor  example,  students  need  skill  in  identifying 
extreme  situations  and  exploring  limitations  of  models. 
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Figure  12.10  Model  for  exploring  reversible  chemical  reaction  designed  with  Molecular  Workbench. 


Students  also  need  to  connect  computer  simulations  to 
hands-on  experiments.  Ways  to  guide  students  to 
explore  inquiry  strategies  for  models  and  simulations 
are  exemplified  in  the  following  features. 

Models  of  Molecules  in  Molecular  Workbench 

The  Molecular  Workbench  software  allows  designers 
to  create  dynamic  visualizations  to  illustrate  scientific 
ideas  (Pallant  and  Tinker,  2004).  Students  can  manip¬ 
ulate  visualizations  that  link  atomic-level  models  with 
observable  phenomena  to  conceptualize  events  such  as 
the  production  of  greenhouse  gases.  An  understanding 
of  atomic-level  interactions  is  essential  to  most  of 
modem  science.  The  idea  that  many  macroscopic  phe¬ 
nomena  emerge  from  large  numbers  of  simple  inter¬ 
actions  is  both  simple  and  profound.  For  example,  the 
example  model  shown  in  Figure  12.10  enables  students 
to  view  how  their  manipulation  of  the  temperature  and 
the  mole  fraction  of  various  substances  changes  the 
speed  of  movement  of  the  particles  and  the  reactions 
between  them.  To  promote  autonomy,  Molecular 
Workbench  visualizations  have  an  intuitive  interface. 
Typical  students  using  Molecular  Workbench  demon¬ 
strate  large  gains  in  understanding  of  atomic-  and 
molecular- level  interactions,  reasoning  about  atoms 
and  molecules,  and  transfer  to  understanding  of  new 
problems  (http://www.concord.org). 

Modeling  Derivatives 

To  help  students  develop  a  qualitative  understanding 
of  the  relationship  between  a  function  and  its  deriva¬ 
tive,  even  before  they  are  taught  the  formality  of  math¬ 
ematics,  Shternberg  and  Yerushalmy  (2002)  developed 
the  function  and  derivative  model  (Figure  12.11)  as 
part  of  the  Visual  Math  project  (http://www.cet.ac.il/ 
math/function/english/).  The  model  allows  comparison 
of  two  views:  a  function  view  at  the  top  and  a  derivative 


view  at  the  bottom.  Using  a  set  of  seven  graphical 
icons,  learners  can  build  their  own  function.  As  they 
build  and  manipulate  the  function,  they  view  how  the 
derivative  changes.  This  model  can  be  used  for  student- 
initiated  problems  or  as  part  of  the  activities  in  the 
Visual  Math  curriculum. 

Pragmatic  Principle:  Encourage  Reflection 

A  well-established  method  for  promoting  lifelong 
learning  is  to  encourage  learners  to  reflect  on  their  own 
learning  and  generate  explanations.  Linn  and  Hsi 
(2000)  found  that,  when  learners  reflect,  they  monitor 
their  progress  and  reach  new  insights.  The  pattern  of 
conducting  an  exploration  and  then  reflecting  improves 
understanding  (Davis,  2006).  Combining  an  experi¬ 
ment,  investigation,  or  research  endeavor  with  reflec¬ 
tion  can  improve  both  activities.  In  contrast  to  text 
materials,  technology-enhanced  materials  can  prompt 
students  to  reflect  and  capture  student  ideas  while  they 
are  learning.  Finding  the  appropriate  amount  and  type 
of  reflection  requires  iterative  design  and  depends 
heavily  on  the  context.  Designing  prompts  that  elicit 
reflection  is  challenging.  Some  prompts  just  lead  stu¬ 
dents  to  conclude  that  they  were  successful  (Davis  and 
Linn,  2000).  The  examples  below  show  how  this  prin¬ 
ciple  helps  teachers  and  learners. 

Prompts  for  Reflection  on  Action  in  CASES 

Prompts  for  reflection  are  a  part  of  a  teachers’  online 
journal  tool  in  CASES  (Davis,  2006).  They  appear  as 
sentence  starters  or  questions  listed  on  the  left  side  of 
the  journal.  Two  to  six  prompts  are  listed  under  each  of 
three  different  categories:  thinking  about  today,  planning 
ahead,  and  general  thoughts.  A  teacher  selects  a  prompt 
from  the  left  column  and  the  prompts  appear  in  the 
journal.  The  prompts  are  designed  to  elicit  information 
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Figure  12.11  Function  and  derivative  model  in  Visual  Math. 


that  may  otherwise  remain  tacit,  such  as  justifications 
for  curricular  decisions.  This  reflection  on  action  influ¬ 
ences  student  learning  outcomes  by  helping  teachers 
to  think  critically  about  their  lessons  and  teaching 
methods  and  to  make  more  effective  decisions  in  real 
time  in  the  classroom  (Davis  and  Krajcik,  2005). 

Note  Taking  in  WISE 

The  WISE  environment  allows  designers  to  embed 
notes  and  students  to  view  them  at  any  time.  Prompts 
direct  students  to  explain  their  ideas,  make  connec¬ 
tions,  or  make  predictions  (see  Figure  12.1).  Slotta 
(2004)  showed  that  reflection  notes  can  help  students 
monitor  their  own  learning. 


DISCUSSION  AND  CONCLUSIONS 

In  summary,  to  identify  promising  supports  for  inquiry 
learning  we  took  advantage  of  a  community 
resource— the  Design  Principles  Database.  We  orga¬ 
nized  the  discussion  around  the  meta-principles  to  cap¬ 
ture  essential  elements  of  effective  instruction.  To 
illustrate  the  meta-principles,  we  selected  the  prag¬ 
matic  principles  that  connected  to  the  largest  number 


of  features.  We  provided  evidence  for  the  pragmatic 
principles  by  describing  research  on  illustrative  fea¬ 
tures.  These  features  communicate  the  complexity  of 
inquiry  instruction  as  well  as  the  insights  emerging  in 
recent  research  programs. 

These  results,  other  findings  in  the  Design  Princi¬ 
ples  Database,  and  related  research  all  provide  strong 
support  for  the  four  meta- principles.  First,  effective 
supports  for  inquiry  make  science  accessible  by  con¬ 
necting  to  the  interests  or  ideas  held  by  the  learner 
(Krajcik  et  al.,  1998;  Linn  and  Hsi,  2000).  When  stu¬ 
dents  grapple  with  everyday  examples,  they  can  eval¬ 
uate  their  intuitive  ideas  and  distinguish  them  from 
normative  views.  Second,  effective  supports  make 
thinking  about  scientific  phenomena  visible  to  teachers 
and  learners  by  animating,  visualizing,  articulating,  or 
representing  complex  phenomena  in  multiple  ways 
(Linn  et  al.,  2006).  Modem  technologies  offer  a  win¬ 
dow  on  unseen  scientific  phenomena,  combined  with 
supports  that  enable  interpretation  of  these  events. 
Third,  effective  supports  help  students  learn  from  each 
other  by  asking  students  to  explain  their  ideas  and  to 
critique  the  ideas  of  others  (Davis,  2006).  When  stu¬ 
dents  discuss  their  ideas,  they  can  develop  criteria  to 
distinguish  them.  Fourth,  effective  supports  promote 
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autonomy  by  stimulating  learners  to  monitor  their 
progress  and  reflect  on  their  learning  (White  and  Fre- 
deriksen,  1998).  When  students  evaluate  their  own 
ideas,  they  can  learn  to  think  critically  about  their 
progress. 

As  these  examples  reveal,  designers  of  inquiry 
environments  have  created  powerful  features  that  can 
give  new  designers  a  head  start  on  building  effective 
inquiry  instruction.  The  value  of  the  full  set  of  meta- 
principles  is  also  reflected  in  the  connections  among 
the  features  in  the  Design  Principles  Database  and  in 
the  characteristics  of  learning  environments  that 
include  multiple  features.  Features  are  linked  to  the 
pragmatic  principle  that  they  illustrate,  but  some  fea¬ 
tures  also  have  elements  that  connect  to  other  prag¬ 
matic  principles.  For  example,  the  features  that  enable 
three-dimensional  manipulation  of  scientific  phenom¬ 
ena  also  have  elements  that  promote  autonomy  by 
asking  students  to  reflect  on  their  observations.  Learn¬ 
ing  environments  such  as  WISE  and  Model-It  typically 
include  features  that  connect  to  all  four  meta-princi- 
ples;  for  example,  WISE  includes  SenseMaker,  Prin¬ 
ciple  Maker,  online  discussions,  and  reflection  notes. 
Evidence  in  the  Design  Principles  Database  suggests 
that  it  is  reasonable  to  conclude  that  effective  supports 
for  inquiry  should  take  advantage  of  all  four  meta- 
principles. 

Another  way  to  evaluate  the  features  is  to  examine 
how  they  support  the  process  of  knowledge  integration 
described  in  the  introduction.  The  features  in  the 
Design  Principles  Database  support  one  or  more  of  the 
four  processes  of  knowledge  integration:  eliciting 
ideas,  adding  ideas,  developing  criteria,  and  sorting 
out  ideas.  Features  such  as  collaborative  brainstorming 
or  reflections  on  everyday  phenomena  elicit  student 
ideas.  Features  such  as  representations  or  animations 
of  scientific  phenomena  add  new  ideas  to  those  held 
by  the  learner.  Features  such  as  peer  evaluation  or 
SenseMaker  encourage  students  to  develop  criteria  for 
distinguishing  among  ideas.  Features  such  as  Causal 
Mapper  or  prompts  for  reflection  encourage  learners 
to  sort  out  their  ideas.  These  connections  between  fea¬ 
tures  and  the  processes  of  knowledge  integration 
prompted  the  development  of  design  patterns  that 
describe  promising  combinations  of  features  (Finn, 
2006;  Linn  and  Eylon,  2006). 

A  design  pattern  is  a  sequence  of  activities  fol¬ 
lowed  by  teachers  and  students  in  a  classroom.  Linn 
and  Eylon  (2006)  synthesized  a  broad  range  of 
research  on  inquiry  science  to  identify  patterns  that 
employ  the  four  knowledge  integration  processes  in 
productive  ways.  These  four  processes  play  out  in  ten 
design  patterns  that  research  has  shown  to  promote 
knowledge  integration;  for  example,  the  pattern  using 


modeling  or  simulation  to  enable  knowledge  integra¬ 
tion  starts  by  eliciting  predictions  about  an  observable 
phenomenon  such  as  heating  water.  The  pattern  then 
adds  ideas  using  a  feature  such  as  Molecular  Work¬ 
bench.  Next,  the  pattern  might  guide  learners  to  form 
conjectures  and  compare  them  using  a  feature  such  as 
Principle  Maker.  Finally,  the  pattern  might  help  learn¬ 
ers  consolidate  ideas  with  a  feature  such  as  note  taking. 
We  are  currently  linking  design  principles  and  design 
patterns. 

The  Design  Principles  Database  is  effective  when 
it  is  built  into  a  structured  design  process  and  used  in 
a  social  context,  such  as  a  graduate  course  or  a  work¬ 
shop  (Kali  et  al.,  2007).  Participants  in  such  courses 
found  that  the  design  principles  approach  assisted  them 
in  brainstorming  ideas  for  activities,  in  generating 
alternatives,  and  in  designing  specific  activities;  how¬ 
ever,  they  found  that  the  Design  Principles  Database 
did  not  provide  sufficient  guidance  for  putting  these 
activities  together  to  create  learning  environments. 

Design  patterns  can  help  with  this  dilemma  by 
suggesting  sequences  of  activities.  The  Design  Princi¬ 
ples  Database  helps  when  designers  seek  ways  to 
implement  the  activities  in  the  sequence.  By  testing 
and  refining  these  resources  for  designers  and  by  add¬ 
ing  additional  resources  that  capture  the  experiences 
of  designers,  the  field  can  become  more  cumulative. 

The  Design  Principles  Database  is  a  work  in 
progress.  As  more  designers  add  their  features,  new 
pragmatic  principles  may  emerge,  and  existing  princi¬ 
ples  may  need  revisions.  The  Design  Principles  Data¬ 
base  can  help  the  field  become  more  cumulative  by 
capturing  the  interplay  between  the  learning  context 
and  the  design  guidelines.  Many  instructional  design¬ 
ers  use  the  analyze,  design,  develop,  implement,  and 
evaluate  (ADDIE)  framework  to  create  and  test  inno¬ 
vations  (Dick  et  al.,  2001).  Those  committed  to  design 
research  acknowledge  the  importance  of  the  iterative 
refinement  process,  in  which  designs  are  tested  multi¬ 
ple  times  (Barab  and  Squire,  2004;  Bell  et  al.,  2004; 
Collins  et  al.,  2004;  Design-Based  Research  Collec¬ 
tive,  2003).  Both  approaches  yield  results  that  can  be 
added  to  the  database.  In  addition,  we  continue  to 
sponsor  opportunities  to  add  and  refine  features  and 
principles.  These  activities  will  elaborate  and  improve 
our  understanding  of  supports  for  inquiry  learning. 
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ABSTRACT 

Collaborative  learning  is  often  cited  as  a  means  to  a 
productive  educational  experience  for  students.  With 
this  in  mind,  designers  and  researchers  have  taken  to 
building  technologies  that  support  various  forms  of  col¬ 
laborative  activity  in  schools  and  other  learning  envi¬ 
ronments.  In  this  chapter,  we  take  a  look  at  how  studies 


of  collaborative  learning  have  developed  over  time  in 
relation  to  a  shift  away  from  exclusively  individualistic 
conceptions  of  human  activity  in  educational  research 
and  design.  We  use  prescriptions  for  and  descriptions 
of  collaborative  learning  as  two  general  categories  for 
understanding  research  and  design.  By  prescriptions, 
we  mean  ideas,  principles,  and  representations  of 
designs  for  collaboration.  By  descriptions,  we  mean 
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empirical  accounts  of  collaboration.  In  our  view,  a 
broad  survey  of  writing  on  collaborative  activity  dis¬ 
plays  an  imbalance;  we  are  light  on  descriptions  and 
heavy  on  prescriptions.  Toward  tipping  this  imbalance 
back,  we  argue  that  a  productive  perspective  for  casting 
descriptions  of  collaborative  learning  is  a  distributed 
one.  A  distributed  perspective  is  one  that  considers 
human  activity  in  relation  to  both  other  people  and 
things .  We  use  some  of  our  own  work  to  illustrate  this 
perspective.  We  conclude  by  using  the  distributed  per¬ 
spective  to  challenge  researchers  to  consider  techno¬ 
logical  artifacts  as  inherently  collaborative  and  to 
move  their  work  forward  under  this  assumption. 


KEYWORDS 

Collaboration:  Activity  involving  multiple  people 
developing  shared  meaning  while  working  together 
on  a  common  problem;  often  involves  harmonious 
cooperation  but  is  not  contingent  upon  it. 

Descriptive  accounts:  Analysis  of  naturally  occurring 
instances  of  human  activity. 

Distributed  perspective:  An  analytic  approach  to 
understanding  human  activity  that  distributes 
agency  across  people  and  material  artifacts. 

Prescriptive  accounts:  Designed  tools,  strategies,  and 
interventions. 


INTRODUCTION 

Over  the  last  15  to  20  years,  educational  researchers 
have  taken  a  special  interest  in  sustaining  and  under¬ 
standing  collaborative  learning.  In  our  chapter,  we 
offer  a  necessarily  incomplete  survey  of  the  concepts, 
methods,  and  technologies  used  in  research  on  collab¬ 
oration.  We  use  two  general  categories  to  characterize 
work  on  collaborative  learning.  The  first  category, pre¬ 
scriptions  for  collaboration,  includes  designed  tools, 
strategies,  and  interventions  to  support  collaborative 
learning.  The  purpose  of  this  research  is  to  develop 
new  technologies  and  methods  based  on  theories  of 
learning  to  support  collaboration  in  classrooms  and 
other  settings.  The  second  category,  which  represents 
much  less  research  literature,  we  call  descriptions  of 
collaboration.  Descriptive  work  focuses  analysis  on 
examples  of  collaboration  that  naturally  occur  in  social 
situations  or  have  been  designed  but  are  studied  as 
naturally  occurring  events. 

The  purpose  of  descriptive  accounts  is  to  under¬ 
stand  collaboration  on  multiple  levels,  at  the  moment- 
to-moment  unfolding  of  collaborative  activity  and  at  a 
larger  social  organizational  level  (Becker,  1996;  Hall 


and  Stevens,  1995).  Our  view  is  that  descriptions  beget 
prescriptions  and  prescriptions  need  description,  but 
the  field  is  too  light  on  careful  descriptions  (Stevens, 
2001).  Our  goal  in  this  chapter,  beyond  highlighting 
this  imbalance,  is  to  show  how  basic  ethnographic 
descriptions  of  collaboration  are  helpful  to  understand¬ 
ing  and  designing  for  collaboration  and  for  working 
with  the  essential  tension  of  the  individual  and  the 
group  in  learning  situations.  From  there,  we  suggest 
that  a  distributed  perspective  on  human  activity  is  gen¬ 
erally  useful  for  research  on  collaboration,  and  we 
sketch  some  of  the  core  concepts  and  studies  that  take 
this  perspective.  The  distributed  perspective  reconsid¬ 
ers  how  we  account  for  collaboration  through  an  anal¬ 
ysis  of  interactions  between  humans  and  nonhumans, 
to  use  terms  from  Bruno  Latour  (1987,  1996). 

In  this  chapter,  we  use  Hutchins’  definition  of 
learning  as  “adaptive  reorganization  in  a  complex  sys¬ 
tem”  (Hutchins,  1995a,  p.  289)  to  guide  our  look  at 
examples  of  collaboration  and  their  impact  on  learn¬ 
ing.  While  using  this  notion  of  learning  we  keep  in 
mind  that  the  “relevant  complex  system  includes  a  Web 
of  coordination  [italics  added]  among  media  and  pro¬ 
cesses  inside  and  outside  the  individual  task  perform¬ 
ers”  (Hutchins,  1995a,  p.  289).  We  take  collaboration 
to  be  activity  that  involves  multiple  people  developing 
shared  meaning  while  working  together  on  a  common 
problem  (Stahl  et  al.,  2006);  this  often  involves  har¬ 
monious  cooperation  but  is  not  contingent  upon  it  (Pea, 
1994;  Stevens,  2000).  Our  take  is  that  a  key  aspect  of 
working  together  involves  coordinating  people  and 
representational  devices,  including  computers  and 
other  forms  of  media  (Hutchins,  1995a).  We  see  the 
study  of  collaborative  learning  as  being  about  the 
emergent  coordination  of  people  and  representational 
devices  to  accomplish  a  set  of  tasks  (Stevens,  2000; 
Stevens  and  Hall,  1998).  Coordination  is  a  part  of  what 
sets  adaptive  reorganization  apart  from  just  any  reor¬ 
ganization.  In  our  view,  adaptive  reorganization 
implies  a  reaction  to  a  new  task  or  chronic  snag  (de  la 
Rocha,  1986);  to  successfully  resolve  the  snag  or 
accomplish  the  task  coordination  of  people  and  objects 
is  necessary.  As  a  concept,  then,  the  complex  system 
is  useful  for  considering  how  interactions  between 
people  and  the  objects  they  use  change  during  collab¬ 
orative  activity. 

We  consider,  in  light  of  Hutchins’  definition,  indi¬ 
vidual  change  to  be  an  important  aspect  of  the  learning 
process,  but  we  want  to  highlight  that  our  view  of 
learning  extends  beyond  individuals.  Much  of  the  work 
we  draw  on  in  this  chapter  comes  from  perspectives 
that  consider  learning  primarily  as  an  individual  pro¬ 
cess  and  measures  the  results  of  collaboration  based 
on  individual  notions  of  learning.  We  use  this  work  in 
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an  attempt  to  build  a  bridge  between  the  two  perspec¬ 
tives  so  individualist  terms  are  not  the  only  way  to 
conceive  of  learning.  Along  these  lines,  we  do  not  have 
any  easy  solutions  for  measuring  individual  change 
with  the  distributed  perspective;  however,  in  particular 
domains  (e.g.,  video  gaming,  math,  science,  cooking) 
we  suggest  that  accessing  one’s  ability  to  develop  a 
distributed  system  where  people  and  tools  are  effec¬ 
tively  coordinated  is  a  potential  starting  point. 

By  casting  learning  as  a  process  that  extends 
beyond  individual  change  we  are  attempting  to  con¬ 
front  a  mismatch  present  in  much  of  the  research  on 
collaboration.  As  Barron  (2003,  p.  352)  pointed  out, 
“much  of  the  empirical  research  on  collaboration  has 
utilized  individually  defined  traits  and  outcomes.”  In 
other  words,  collaboration  as  a  process  with  multiple 
actors— both  human  and  nonhuman— is  often  held  up 
to  measures  that  only  account  for  abilities  in  individ¬ 
uals  (Stevens,  2000).  With  Hutchins’  definition  as  a 
starting  point,  we  are  attempting  to  move  beyond  this 
mismatch. 


CONCEPTS,  METHODOLOGIES, 

AND  TECHNOLOGIES 

The  relatively  recent  focus  on  collaboration  occurred 
as  research  on  learning  and  cognition  expanded  its  per¬ 
spective  to  include  socially  situated  human  activity.  For 
most  of  the  20th  century,  educational  research  on  learn¬ 
ing  was  closely  tied  to  mainstream  psychology.  Before 
the  1970s,  mainstream  psychology  and  a  great  deal  of 
educational  research  took  the  form  of  behaviorism, 
which  concerned  itself  with  focusing  individuals  on 
narrowly  defined  tasks.  Eventually,  the  paradigm  of 
behaviorism  gave  way  to  the  so-called  “cognitive  rev¬ 
olution”  (Bruner,  1997),  which  offered  a  more  complex 
view  of  human  learning  but  one  that  still  focused  on 
individuals.  It  is  beyond  the  scope  of  this  chapter  to 
dwell  on  the  details  of  how  research  on  learning  is 
shifting  with  more  global  paradigm  shifts  from  behav¬ 
iorism  to  cognitivism  and  beyond  (Stevens  et  al.,  2005); 
however,  we  would  like  to  point  out  that,  for  much  of 
the  history  of  learning  research,  the  concern  has  been 
with  individuals  and  what  goes  on  inside  their  heads 
(Bransford  et  al . ,  2000) .  Our  look  at  collaborative  learn¬ 
ing  begins  at  a  time  when  sociocultural  and  situated 
perspectives  on  learning  compete  with  mainstream  psy¬ 
chology’s  mentalistic  approach.  The  shift  was  based  in 
part  on  a  move  from  mentalist  conceptions  of  human 
learning  to  a  perspective  that  treats  concepts  such  as 
practice  and  activity  as  primary  (Cole,  1996;  Lave, 
1988;  Vygotsky,  1962;  Wertsch,  1998). 


Sociocultural  Research  on 
Learning  and  Situated  Action 

The  expanded  influence  of  interests  beyond  main¬ 
stream  psychology  on  educational  research  is  due,  in 
part,  to  sociocultural  perspectives  on  learning  having 
taken  hold  among  designers  and  researchers.  The 
sociocultural  view  of  human  activity  is  based  in  part 
on  Vygotskian  notions  of  human  action  which  stresses 
the  role  that  culture,  tools,  and  artifacts  play  in  human 
thinking  (Cole,  1996;  Wertsch,  1998).  The  expansion 
beyond  mainstream  psychology  to  include  a  situated 
action  perspective  that  focuses  less  on  the  properties 
of  the  human  mind  and  more  on  social  and  material 
aspects  of  a  learning  context  has  not  been  without 
controversy  (Anderson  et  al.,  1996;  Greeno,  1997).  A 
strong  element  of  the  situated  action  movement  has 
been  to  understand  collaboration  as  an  important  ele¬ 
ment  of  learning. 

One  of  the  more  common  ways  in  which  the  socio¬ 
cultural  perspective  has  found  its  way  into  designed 
technologies  for  learning  is  through  scaffolded  learn¬ 
ing  environments  that  attempt  to  engage  students  in 
collaborative  scientific  inquiry.  These  prescriptions  are 
based  on  the  idea  that  students  are  able  to  achieve  more 
with  guided  support  from  a  knowledgeable  helper  than 
they  would  on  their  own  (Wood  et  al.,  1976).  Scaffold¬ 
ing  technologies  are  designed  to  aid  students  in  col¬ 
laborative  activities  such  as  scientific  inquiry  (Kolod- 
ner  et  al.,  2003)  and  argumentation  (Bell,  2001). 
Scaffolded  environments  build  off  the  idea  that  learn¬ 
ers  are  able  to  achieve  more  in  a  context  where  external 
supports,  either  social  or  material,  provide  for  more 
complex  and  difficult  thinking.  It  is  postulated  that 
over  time  the  students  will  no  longer  need  the  techno¬ 
logical  or  social  supports  due  to  conceptual  change 
and  the  development  of  new  inquiry  practices.  In  other 
words,  learners  internalize  relevant  aspects  of  the  scaf¬ 
fold.  Research  on  scaffolded  learning  environments 
often  takes  a  prescriptive  form,  arguing  for  new  ways 
to  engage  students  in  collaborative  scientific  inquiry 
(Kolodner  et  al.,  2003).  The  primary  focus  of  this  work 
is  to  better  understand  how  to  engage  students  in  sci¬ 
entific  inquiry  to  promote  conceptual  change. 

Roschelle  (1992)  attempted  the  integration  of  two 
theoretical  ideas— conceptual  change  and  a  collabora¬ 
tive-interactional  perspective— to  understand  how  two 
or  more  people  develop  shared  meanings.  His  work  is 
one  example  of  how  descriptions  have  been  used  to 
identify  aspects  of  student  interactions  with  technol¬ 
ogy  that  are  relevant  to  collaboration.  He  analyzed  the 
interactions  of  two  young  girls  using  software  that 
simulated  velocity  and  acceleration  from  the  perspec¬ 
tive  of  Newtonian  physics.  Over  several  iterations  of 
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displaying,  confirming,  and  repairing  actions,  the  two 
girls  developed  a  set  of  shared  meanings  for  the  con¬ 
cepts  at  hand.  Roschelle  showed  how  their  interactions 
led  to  new  ideas  about  the  way  the  physical  world 
works.  His  analysis,  which  focused  on  their  conceptual 
change,  revealed  that  “conversational  interaction  can 
enable  students  to  construct  relational  meanings  incre¬ 
mentally”  (Roschelle,  1992,  p.  237).  The  importance 
of  this  work  was  to  integrate  collaboration  and  con¬ 
ceptual  change  through  careful  description  of  moment- 
to-moment  action.  Its  contribution  was  in  identifying 
conceptual  change  as  a  process  that  involves  social  and 
material  aspects  of  an  environment  and  the  develop¬ 
ment  of  shared  meaning  across  students. 

Stevens  and  Hall’s  (1998)  descriptive  comparison 
of  a  tutor  assisting  a  student  through  mathematical 
problem  solving  and  two  civil  engineers  working  col- 
laboratively  provides  a  nice  complement  to  the  study 
by  Roschelle.  They,  too,  found  that  over  time  a  con¬ 
vergence  of  shared  meaning  occurred  incrementally, 
resulting  in  the  emergent  coordination  of  people  and 
objects;  however,  rather  than  cast  two  people  in  the 
roles  of  learners  as  Roschelle  did,  Stevens  and  Hall 
looked  at  naturally  occurring  asymmetrical  pairs  who 
were  working  together.  They  showed  how  a  student’s 
and  a  junior  engineer’s  perception  of  mathematical 
representations,  what  they  characterized  as  disciplined 
perception ,  changed  through  interactions  with  a  more 
knowledgeable  collaborator. 

Brigid  Barron  (2003)  expanded  Roschelle’s  (1992) 
study  of  collaboration  and  conceptual  change  to  better 
understand  how  problem-solving  space  is  used  in  suc¬ 
cessful  collaborative  groups.  Barron’s  (2003)  work, 
another  descriptive  account,  compared  successful  col¬ 
laborative  groups  with  unsuccessful  groups.  As  a  con¬ 
dition,  she  found  that  successful  groups  have  a  rela¬ 
tional  space  and  a  content  space.  The  relational  space 
exists  for  interactional  challenges  and  opportunities, 
while  the  content  space  is  for  the  problem  to  be  solved. 
The  existence  of  these  different  spaces  meant  success¬ 
ful  groups  had  fewer  challenges  achieving  joint  atten¬ 
tion  and  were  able  to  better  capitalize  on  their  mem¬ 
bers’  insights.  Her  work  reinforced  the  idea  that 
cognitive  limitations  are  not  sufficient  in  explaining 
why  groups  fail. 

Computer-Supported  Collaborative  Learning 

The  expansion  beyond  individualist  notions  of  human 
activity  also  led  to  the  development  of  a  field  known 
as  computer- supported  collaborative  learning  (CSCL) 
(Koschmann,  1994).  CSCL  work  involves  designing, 
developing,  and  describing  technologies  to  support 
collaboration  in  learning  environments.  In  CSCL,  the 


“focus  [is]  no  longer  on  what  might  be  taking  place  in 
the  heads  of  individual  learners,  but  what  [is]  taking 
place  between  them  in  their  interactions”  (Stahl  et  al„ 
2006,  p.  415).  Two  of  the  earliest  and  most  established 
CSCL  environments,  the  Computer-Supported  Inten¬ 
tional  Learning  Environment  (CSILE)  and  the  Fifth 
Dimension,  are  representative  of  the  difference 
between  a  focus  on  changes  in  social  interactions  and 
mental  states  as  outcomes  of  collaboration. 

The  Computer-Supported  Intentional 
Learning  Environment  (CSILE) 

The  Computer-Supported  Intentional  Learning  Envi¬ 
ronment  (now  referred  to  as  Knowledge  Forum)  was 
one  of  the  earliest  examples  of  a  technological  pre¬ 
scription  for  enabling  collaborative  learning  (Scarda- 
malia  and  Bereiter,  1994).  The  environment  was 
designed  to  promote  learning  by  encouraging  the 
development  of  collaborative  knowledge-building 
communities  rather  than  focusing  students  on  specific 
tasks.  A  collaborative  knowledge-building  community 
acts  somewhat  like  a  small  version  of  a  scientific  com¬ 
munity  where  big  ideas  are  collectively  constructed 
over  time  by  a  group  of  people  (Latour,  1987).  With 
CSILE  and  Knowledge  Forum,  students  post  graphical 
or  text -based  notes  to  relay  questions,  research  plans, 
and  newly  found  information.  These  notes  can  then  be 
linked  to  form  a  kind  of  collaborative  inquiry  over  a 
local  networked  environment. 

Almost  two  decades  of  work  on  CSILE  and 
Knowledge  Forum  have  demonstrated  how  networked 
environments  can  be  used  to  promote  collaborative 
knowledge  building.  In  a  knowledge -building  commu¬ 
nity  in  school,  “learning  is  not  asymptotic  because 
what  one  person  does  in  adapting  changes  the  envi¬ 
ronment  so  that  others  must  readapt”  (Scardamalia  and 
Bereiter,  1994,  p.  267).  In  other  words ,  individual  con¬ 
tributions  raise  the  standard  that  others  must  work  for 
and  the  nature  of  learning  changes  when  compared 
with  a  traditional  classroom.  This  notion,  where  learn¬ 
ing  involves  adapting  to  changes  in  the  context  created 
through  social  interactions,  echoes  the  sentiments  of 
Hutchins’  definition  of  learning  as  “adaptive  reorgani¬ 
zation  in  a  complex  system”  (Hutchins,  1995a,  p.  289) 
and  may  be  an  example  of  how  the  coordination  of 
people  and  objects  can  impact  an  individual’s  thinking. 
Gilbert  and  Driscoll  (2002)  built  off  the  work  on 
CSILE  in  an  attempt  to  better  understand  what  condi¬ 
tions  are  necessary  for  knowledge-building  communi¬ 
ties.  They  employed  four  learning  principles  to  support 
collaborative  knowledge  building  in  a  graduate  level 
course.  Their  principles  involve  setting  a  collective 
goal,  using  cooperative  groups,  allowing  for  selection 
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of  materials  to  support  ownership,  and  using  techno¬ 
logical  tools  to  archive  work  and  facilitate  communi¬ 
cation. 

The  Fifth  Dimension 

Another  early  CSCL  environment  was  the  Fifth 
Dimension  (5th  D)  project  initiated  by  Mike  Cole  and 
others  at  the  Laboratory  for  Comparative  Human  Cog¬ 
nition  (LCHC).  The  primary  focus  here  was  to  develop 
an  activity  system  in  after-school  clubs  that  provided 
“optimal  conditions  for  children’s  learning  and  devel¬ 
opment”  (Kaptelinin  and  Cole,  2001,  p.  307).  The 
design  of  this  social  system  to  support  collaborative 
computer  game  playing  is  also  based  on  a  sociocultural 
perspective  of  human  activity  (Cole,  1996).  The  pri¬ 
mary  contribution  of  this  work  has  been  a  sustainable 
model  of  social  interaction  that  minimizes  the  power 
differential  between  children  and  adults  through  a  vari¬ 
ety  of  artifacts  in  the  environment.  Ito  has  provided 
important  descriptive  accounts  of  5th  D  and  shown,  in 
part,  that  children  appropriate  the  technologies  in  the 
environment  to  accomplish  their  own  goals,  which  are 
often  at  odds  with  those  of  the  adults  (Ito,  1997, 2002, 
2005,2006).  The  work  surrounding  the  5th  D  is  a  good 
example  of  how  a  descriptive  analysis  can  be  applied 
to  the  design  of  a  collaborative  system. 

CSCL:  A  Field  for  Prescription  and  Description 

Supporting  collaborative  learning  with  technological 
tools  is  an  important  element  of  CSCL  projects  along 
with  a  host  of  other  design  efforts.  A  challenge  for 
CSCL,  then,  is  to  attend  to  both  the  technical  and  the 
social  aspects  of  collaboration  (Kirschner  et  al.,  2004). 
Our  perspective  is  that  rich  descriptions  of  collabora¬ 
tion  that  analyze  interactions  between  people  and  tech¬ 
nologies  will  be  productive  in  achieving  that  challenge. 
Along  these  lines,  Stahl  et  al.  (2006)  view  the  field  of 
CSCL  as  having  both  descriptive  and  prescriptive  ele¬ 
ments  that  play  off  each  other  (Stevens,  2001). 

Distance  Education 

The  Internet  has  opened  many  opportunities  for  col¬ 
laborative  learning,  particularly  in  distance  education 
(Reeves  et  al.,  2004).  This  prescription  often  has  an 
explicit  focus  on  fostering  online  collaborative  learn¬ 
ing.  Additionally,  researchers  have  taken  care  to  craft 
descriptions  for  a  better  understanding  of  how  to  get 
the  most  out  of  collaboration  in  online  environments. 
For  example,  O’Neill  (2001)  described  how  collabo¬ 
rations  with  professional  scientists  over  networked 
environments  influenced  activity  in  science  class¬ 


rooms.  He  found  that  students  were  better  able  to 
work  within  scientific  genres  when  they  discussed 
and  shared  ideas  with  professionals.  Others  looked  at 
how  activities  in  online  courses  support  collaborative 
learning  and  confirmed  that  good  pedagogy  is  as 
important  if  not  more  important  than  the  technology 
in  use  (Moallem,  2003).  Kato  and  colleagues  (2001) 
investigated  uses  of  video  for  distance  learning,  based 
on  findings  from  observational  studies  of  how  people 
interact  in  real  settings.  Their  approach  was  to  design 
features  into  a  system  to  allow  students  and  instruc¬ 
tors  the  ability  to  use  gestures,  an  important  aspect 
of  naturally  occurring  human  activity.  In  another 
example,  Fischer  and  Mandl  (2005)  looked  at  how 
different  representational  tools  and  conditions  influ¬ 
enced  collaboration  during  video  conferencing  and 
found  that  in  all  of  their  prescribed  conditions  groups 
often  did  not  share  knowledge.  Many  of  the  recom¬ 
mendations  from  studies  of  distance  learning  involve 
distributing  properties  of  a  system  across  people  and 
artifacts  by  creating  shared  objects  and  assigning 
individual  tasks.  As  Moallem  (2003)  contends,  suc¬ 
cessful  online  collaborative  learning  is  about  more 
than  engaging  students  in  the  same  project,  but  also 
about  developing  shared  meaning  and  knowledge 
socially. 

Design-Based  Research 

The  shift  in  our  understanding  of  learning  from  an 
individual  mental  process  to  socially  situated  activity 
has  created  a  methodological  challenge  for  developing 
collaborative  learning  environments.  If  aspects  of  a 
context  such  as  materials  and  relationships  are  impor¬ 
tant  to  supporting  learning,  then  environments  for  col¬ 
laboration  can  be  quite  complex.  Much  of  the  work  on 
scaffolding  scientific  inquiry,  convergent  conceptual 
change,  CSCL,  and  distance  education  was  developed 
using  design-based  research  (Baumgartner  et  al.,  2003; 
Brown,  1992;  Collins,  1992;  Reeves,  2006)  to  meet 
the  methodological  challenges.  Design-based  research 
uses  iterative  cycles  of  design  and  observation  to  build 
prescriptions  for  new  learning  techniques.  The  imple¬ 
mentation  of  observational  methods  in  the  design  pro¬ 
cess  helps  researchers  identify  new  means  for  support¬ 
ing  collaboration. 


A  DISTRIBUTED  PERSPECTIVE 

From  our  position,  descriptive  accounts  influenced  by 
a  distributed  perspective  can  be  used  to  better  under¬ 
stand  the  relationship  between  material  objects  and  the 
social  world  in  collaborative  learning;  for  example. 
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consider  the  CSILE  environment  described  above. 
Where  students  engage  in  a  knowledge-building  com¬ 
munity,  a  distributed  look  will  take  the  coordinated 
reorganization  of  the  notes  and  messages  as  a  learning 
outcome.  The  distributed  perspective  we  use  here  is 
principally  informed  by  four  related  approaches  to 
understanding  culture,  society,  and  human  activity 
(Bateson,  1972;  Becker,  1982;  Hutchins,  1995a,b; 
Latour,  1996).  Taken  together  these  approaches 
emphasize  looking  beyond  individual  action  to  under¬ 
stand  human  behavior  as  much  of  the  work  on  collab¬ 
orative  learning  described  above  already  does.  What  a 
distributed  perspective  adds  is  an  increased  focus  on 
the  material  environment  as  an  important  aspect  of 
collaboration.  In  other  words,  extending  the  boundary 
of  analysis  beyond  an  individual  is  applied  to  nonhu¬ 
man  artifacts  as  well,  a  point  that  lends  itself  well  to 
a  community  concerned  with  the  design  of  learning 
technologies. 

The  anthropologist  Gregory  Bateson  (1972)  influ¬ 
enced  the  distributed  perspective  by  expanding  con¬ 
ceptualizations  of  the  mind  beyond  the  skin  as  a  means 
to  better  explain  human  behavior.  “The  mental  world— 
the  mind— the  world  of  information  processing  is  not 
limited  by  the  skin”  (Bateson,  1972,  p.  454).  Bateson 
explained  this  expansion  of  the  mind  by  asking  where 
the  mental  system  of  a  blind  person,  using  a  white 
cane,  begins:  Does  it  begin  at  the  handle,  at  the  end 
of  the  stick,  or  at  the  sidewalk?  He  settled  the  dilemma 
by  including  all  parts  of  the  system  and  suggested  that 
the  approach  was  applicable  to  other  behaviors:  “If 
what  you  are  trying  to  explain  is  a  given  piece  of 
behavior,  such  as  the  locomotion  of  the  blind  man, 
then,  for  this  purpose,  you  will  need  the  street,  the 
stick,  the  man;  the  street,  the  stick,  and  so  on,  round 
and  round”  (Bateson,  1972,  p.  459). 

The  sociologist  Howard  Becker  studied  social 
worlds  — a  concept  similar  to  communities  of  practice 
(Lave  and  Wenger,  1991;  Wenger,  1999)— in  art,  edu¬ 
cation,  and  medicine.  In  his  study  of  art  worlds 
(Becker,  1982),  he  analyzed  how  sets  of  conventions 
from  art  worlds  guide  the  production  of  new  work. 
Conventions  are  built  into  a  range  of  artifacts,  from 
the  tools  that  artists  use  to  the  funding  mechanisms 
that  support  their  craft,  and  make  the  production  of 
new  work  possible.  “Members  of  art  worlds  coordinate 
the  activities  by  which  work  is  produced  by  referring 
to  a  body  of  conventional  understandings  embodied  in 
common  practice  and  in  frequently  used  artifacts” 
(Becker,  1982,  p.  34).  His  point  was  that  each  work  of 
art  represents  more  than  the  efforts  of  an  individual 
artist  and  is  a  kind  of  collaboration  made  possible 
because  of  conventions  distributed  across  the  objects 
artists  use  and  the  practices  they  engage  in. 


Similarly,  Latour  (1996)  has  argued  that  much  of 
what  humans  can  do  is  built  into  the  material  world. 
In  reference  to  common  interactions  in  a  post  office, 
he  asked:  “How  could  you  compute  the  daily  balance 
of  an  office  without  formulae,  receipts,  accounts,  led¬ 
gers— and  how  can  one  miss  the  solidity  of  the  paper, 
the  durability  of  the  ink,  the  etching  of  the  chips,  the 
shrewdness  of  the  staples  and  the  shock  of  a  rubber 
stamp?”  (Latour,  1996,  p.  235).  Latour  was  trying  to 
highlight  that  much  of  what  humans  can  do  goes 
beyond  the  abilities  built  into  the  people  and  is  built 
into  the  properties  of  the  nonhuman  objects  them¬ 
selves.  It  is  the  physical  nature  that  gives  objects  dura¬ 
bility  in  time  and  space  and  forms  a  frame  for  society. 
In  other  words,  human  behavior  is  to  be  best  under¬ 
stood  as  a  distribution  across  people  and  artifacts. 

Finally,  Hutchins  (1995a,b)  took  the  expansion  of 
the  mind  and  the  agency  of  artifacts  and  applied  it  to 
his  ethnographic  observations.  The  result  is  a  human 
less  dependant  on  complex  mental  processes  and  a 
view  of  successful  activity  in  the  world  that  is  distrib¬ 
uted  across  people  and  artifacts:  "This  system-level 
cognitive  view  directs  our  attention  beyond  the  cogni¬ 
tive  properties  of  individuals  to  the  properties  of  exter¬ 
nal  representations  and  to  the  interactions  between 
internal  and  external  representations”  (Hutchins, 
1995b,  p.  287). 


PRESCRIPTIONS  AND  DESCRIPTIONS 

Common  approaches  to  promoting  collaboration  in 
educational  settings  could  be  recast  using  the  distrib¬ 
uted  perspective.  By  recasting  them  we  can  see  rele¬ 
vant  aspects  of  particular  systems  and  how  people  and 
objects  come  into  coordination  in  successful  collabo¬ 
rations.  Prescriptive  research  includes  work  on  collab¬ 
orative  problem  solving  where  students  are  organized 
into  groups  and  given  a  variety  of  tools  to  scaffold 
scientific  inquiry  (Kolodner  et  al.,  2003).  Research  has 
also  been  done  to  test  the  effectiveness  of  conditions 
for  collaboration  (Fischer  and  Mandl,  2005)  such  as 
setting  collective  goals,  providing  opportunities  for 
ownership,  and  developing  appropriate  representa¬ 
tional  tools.  The  distributed  perspective  would  con¬ 
sider  the  emergent  coordination  of  tools  and  people 
while  students  accomplish  particular  tasks  as  an  impor¬ 
tant  element  of  learning;  therefore,  in  recasting  with 
this  perspective  the  embodied  practices  that  develop 
with  the  use  of  these  tools  is  foregrounded,  and  inter¬ 
nationalization  of  the  properties  of  a  tool  or  scaffold 
is  less  of  a  concern.  The  distributed  perspective  can  be 
extended  to  a  range  of  prescriptions,  including  online 
environments  (Reeves  et  al.,  2004) ,  efforts  for  teaching 
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collaboration  skills  (Nath  and  Ross,  2001),  and  teacher 
development  programs  where  collaboration  is  a  means 
to  improving  instructional  practices.  We  contend  that 
this  is  an  analytic  approach  that  lends  itself  well  to 
pushing  new  designs  forward  and  advancing  reitera¬ 
tion  in  old  designs,  because  understanding  the  emer¬ 
gent  coordination  of  people,  objects,  and  practices  can¬ 
not  be  taken  lightly  when  learning  is  of  concern. 

We  note  that  there  are  many  important  descriptive 
accounts  of  collaboration  out  there  that  benefit  from  a 
distributed  perspective.  This  work  highlights  the 
importance  of  ensuring  that  there  are  spaces  where 
ideas  can  be  heard  from  all  group  members  (Barron, 
2003).  It  also  taught  us  that  collaborative  technologies 
are  often  not  the  latest  innovation  (Stevens,  2000). 
Additionally  descriptive  accounts  remind  us  that  learn¬ 
ers  come  with  their  own  agendas,  which  play  a  role  in 
how  the  technology  gets  used  (Ito,  2006). 

A  notable  description  outside  the  field  of  educa¬ 
tion,  but  relevant  to  technology  use,  took  place  at  the 
Xerox  Palo  Alto  Research  Center  (Xerox  PARC), 
where  Lucy  Suchman  (1987)  carefully  described  the 
interactions  between  users  and  copy  machine  help 
guides.  What  is  important  to  take  from  this  work  and 
is  relevant  to  the  distributed  perspective  on  collabora¬ 
tion  is  that  Suchman  showed  that  users  of  intelligent 
help  systems  employed  the  artificial  intelligence  as  a 
resource  rather  than  a  scripted  plan.  The  help  systems, 
from  a  prescription  perspective,  are  not  intended  to  be 
a  collaborative  technology  but  ended  up  serving  as  a 
resource  for  collaboration  when  they  came  into  coor¬ 
dination  with  the  users’  activity. 

Our  ethnographic  work  on  young  people’s  video 
gaming  practices  is  an  example  of  how  we  are  attempt¬ 
ing  to  roll  findings  from  descriptive  research  into 
designed  artifacts  using  a  distributed  perspective 
(Satwicz,  2006;  Stevens  et  al.,  2007).  We  began  this 
work,  in  part,  to  empirically  investigate  many  of  the 
claims  that  have  been  made  regarding  learning  and 
video  games  (Gee, 2003;  Johnson, 2005).  Additionally, 
because  game  systems  are  often  designed  for  use  by 
multiple  people  and  are  commonly  used  in  social  sit¬ 
uations,  we  saw  game  play  as  a  place  likely  for  natu¬ 
rally  occurring  instances  of  collaboration.  Currently, 
we  are  beginning  to  use  our  descriptions  as  a  means 
to  think  about  prescriptions  for  collaborative  learning. 

In  our  observations,  we  have  seen  kids  come  up 
with  a  host  of  collaborative  learning  arrangements.  In 
some  situations,  more  experienced  players  apprenticed 
younger  players  by  creating  spaces  for  observation  and 
easy  enrollment  into  the  game  (Lave  and  Wenger, 
1991).  At  other  times,  our  participants  traded  off  roles 
of  expert  and  novice  as  the  tasks  they  were  working 
on  changed.  Additionally,  we  have  seen  the  same  game 


played  in  quite  different  ways  by  different  players, 
suggesting  that  learning  was  not  programmed  into  the 
media  but  coordinated  by  the  players  themselves.  We 
have  also  seen  players  use  some  surprising  tools  in 
their  collaborations,  such  as  characters  and  actions  in 
the  game.  On  the  whole,  we  find  these  instances  pro¬ 
ductive  for  understanding  the  many  ways  in  which 
collaborative  learning  can  happen. 

One  lesson  we  are  beginning  to  learn  is  that  the 
conventions  embodied  in  gaming  materials  and  prac¬ 
tices  make  it  easy  for  kids  to  organize  their  own  col¬ 
laborative  learning  arrangements;  for  example,  we  have 
had  our  participants  describe  to  us  how  the  consistency 
of  button  combinations  across  games  and  the  design  of 
the  controllers  make  it  easy  for  new  games  to  be  taken 
up  and  learned.  Additionally,  because  video  games  are 
frequently  played  in  family  living  rooms  using  televi¬ 
sion  screens  viewable  by  many  people  at  one  time,  it 
is  easy  for  players  to  communicate  and  share  ideas. 
One  design  we  are  in  the  process  of  developing  is  a 
piece  of  software  that  utilizes  the  conventions  built  into 
gaming  materials  but  provides  a  resource  for  families 
to  discuss  the  complexities  of  institutionalized  educa¬ 
tion  and  how  to  be  a  successful  student.  The  artifact, 
at  this  point,  is  immature,  so  the  ongoing  design  process 
is  one  that  will  require  further  descriptive  work  for  a 
successful,  well-grounded  prescription. 

Video  Traces  is  a  medium,  designed  by  the  second 
author,  that  we  describe  here  as  a  concrete  example  of 
how  descriptions  of  collaborative  learning  situations 
can  lead  to  prescriptions  in  the  form  of  designed  arti¬ 
facts.  Across  a  range  of  settings,  Stevens  observed  a 
common  pattern  of  interaction;  when  working 
together,  people  made  joint  progress  through  the  coor¬ 
dinated  use  of  talk,  gesture  ,  and  drawing  over  the  sur¬ 
face  of  drawings  or  objects  that  all  participants  in  an 
interaction  could  see  (Hall  and  Stevens,  1995;  Stevens, 
2000,  2007;  Stevens  and  Hall,  1997,  1998).  As  an 
example,  architects  often  shared  ideas  by  using  their 
hands  or  pencils  in  coordination  with  their  speech  to 
point  at  and  envision  possible  designs  over  the  surface 
of  drawings.  These  moments  were  completely  ordi¬ 
nary— they  happened  all  the  time— but  Stevens  found 
this  coordinated  use  of  these  ordinary  resources  to  be 
essential  to  how  people  learned  and  taught  together. 

Video  Traces  capitalizes  on  this  coordinated  suite 
of  resources  by  allowing  users  to  record  annotations 
over  still  or  moving  images  through  watching,  talking, 
and  pointing  (or  drawing).  This  allows  people  to  com¬ 
municate  and  share  their  ideas  around  a  common  object 
in  ways  that  resemble  face-to-face  collaborative  learn¬ 
ing  situations  but  without  having  to  be  face  to  face;  in 
essence.  Video  Traces  borrows  the  technical  form  of 
interactional  turns  in  face-to-face  interaction  and 
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records  them.  Video  Traces  has  proved  useful  in  a  wide 
range  of  learning  and  teaching  contexts,  including 
teacher  education,  museums,  dance  education,  conduct¬ 
ing,  architecture,  and  collegiate  sports  (Stevens,  2007). 
The  important  point  from  the  perspective  of  this  chapter 
is  that  research  involving  close  description  of  a  range 
of  collaborative  settings  led  to  a  generalization  about 
collaborative  learning  that  in  turn  led  to  a  prescription 
that  was  embodied  in  the  design  digital  medium  that 
supports  collaboration  in  a  new  way. 


DISCUSSION 

Our  purpose  here  has  been  to  show  some  of  the  con¬ 
ceptual,  methodological,  and  technological  milestones 
in  research  on  collaborative  learning.  We  have  indi¬ 
cated  that  the  increase  in  attention  to  supporting  col¬ 
laborative  learning  over  the  last  decade  and  a  half 
mirrors  an  expansion  of  the  influences  on  educational 
research  to  include  sociocultural  and  situated  perspec¬ 
tives  on  human  activity.  Our  suggestion  is  that  the  next 
direction  in  this  work  should  continue  this  expansion 
to  include  a  distributed  perspective.  We  pointed  to 
descriptive  accounts  of  collaboration  that  show  how 
activity  is  distributed  across  people  and  artifacts. 

We  want  to  keep  pushing  for  good  descriptive 
accounts  that  challenge  the  notion  that  learning  is  a 
process  of  change  inside  a  person  (Stevens,  2001);  by 
doing  this,  we  begin  to  see  emergent  and  unanticipated 
uses  of  technology  not  as  a  sign  of  a  failed  design  but 
as  constitutive  features  of  a  learning  experience.  We 
also  remind  the  reader  that  when  learners  do  something 
unexpected,  as  descriptions  show,  we  have  an  impor¬ 
tant  foundation  to  work  from.  If  collaborative  learning 
happens  with  things  that  are  not  designed  to  be  col¬ 
laborative  learning  technologies  (e.g.,  paper,  single¬ 
player  video  games) ,  then  as  designers  we  should  take 
note  if  we  are  going  to  move  the  field  forward. 

These  ideas  echo  those  of  Howard  Becker,  who 
has  pointed  out  that  the  purpose  of  descriptive  work 
is  to  find  elements  of  a  context  that  can  help  us  answer 
our  questions:  “The  object  of  any  description  is  not  to 
reproduce  the  object  completely— why  bother  when 
we  have  the  object  already?— but  rather  to  pick  out  its 
relevant  aspects,  details  which  can  be  abstracted  from 
the  totality  of  details  that  make  it  up  so  that  we  can 
answer  some  questions  we  have”  (Becker,  1996,  p.  64). 

Our  questions  about  how  collaborative  learning  is 
distributed  across  people  and  artifacts  are  well  suited 
for  good  descriptive  accounts.  We  will  end  with  a  final 
caution  on  the  distributed  perspective  we  advocate  for 
here.  Following  Suchman  (2001),  we  do  not  intend  to 


attribute  full  human  agency  to  machines;  however,  we 
do  want  to  push  the  line.  Using  Latour’s  terms,  we 
want  to  put  humans  and  designed  things  on  the  same 
level.  Unlike  Latour,  however,  we  do  not  want  to  give 
objects  the  same  kind  of  agency  that  we  give  to  people 
in  our  analyses.  Humans  have  unique  qualities  that  our 
descriptive  accounts  should  recognize.  The  differ¬ 
ences,  however,  are  an  open  question  and  for  us  a 
reason  to  continue  this  line  of  work. 
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ABSTRACT 

This  chapter  reviews  some  of  the  prescriptive  princi¬ 
ples  that,  based  on  research  or  experience,  have  been 
identified  for  facilitating  effective,  efficient,  and 
engaging  instruction.  For  the  purposes  of  this  chapter, 
instruction  is  defined  as  a  deliberate  attempt  to  design 
a  product  or  environment  that  facilitates  the  acquisi¬ 
tion  of  specified  learning  goals.  This  chapter  first 
reviews  the  first  principles  of  instruction  identified 
by  Merrill  (2002a).  In  the  second  section,  recent 
specifications  of  instructional  design  principles  are 
compared  to  these  first  principles.  The  final  section 
reviews  approaches  for  designing  instruction  cen¬ 
tered  in  whole  tasks. 

KEYWORDS 

4C/ID:  Four-component  instructional  design  model 
(van  Merrienboer,  1997). 

Activation  principle:  Learning  is  promoted  when 
learners  activate  relevant  cognitive  structures. 
Application  principle:  Learning  is  promoted  when 
learners  engage  in  application  of  their  newly 
acquired  knowledge  or  skill. 

Cognitive  training  model:  A  five-task  instructional 
design  model  (Foshay  et  al.,  2003). 

Cycle  of  instruction:  The  activation-demonstration- 
application-integration  cycle  of  the  first  principles. 
Demonstration  principle:  Learning  is  promoted  when 
learners  observe  a  demonstration  of  the  skills  to  be 
learned. 

e-Learning  principles:  Prescriptive  principles  for 
designing  e-learning;  see  multimedia  learning  prin¬ 
ciples. 

First  principles  of  instruction:  Five  principles  funda¬ 
mental  to  effective,  efficient,  and  engaging  learn¬ 
ing. 

Instructional  design:  Creating  blueprints  for  effective, 
efficient,  and  engaging  instruction. 

Integration  principle:  Learning  is  promoted  when 
learners  integrate  their  new  knowledge  into  their 
everyday  life. 

Minimalist  principles:  Instructional  design  principles 
for  sparse  instruction. 

Multimedia  learning  principles:  Likely  effects  of  text, 
animation,  audio,  and  graphics  on  learning. 
Pebble-in-the-Pond  instructional  design:  A  content- 
first  approach  to  designing  instruction. 
Problem-centered  instruction:  See  task-centered 
instruction  principle  and  strategy. 

Scaled  instructional  strategies:  Hypothesis  that  appli¬ 
cation  of  the  first  principles  has  an  accumulating 
performance  effect  for  complex  skills. 


Structure-guidance-coaching-reflection  cycle:  Instruc¬ 
tional  assistance  within  the  activation-demonstra¬ 
tion-application-reflection  cycle  of  instruction. 

Task-centered  instruction  principle:  The  central  prin¬ 
ciple  that  learning  is  promoted  when  learners  are 
engaged  in  a  task-centered  approach. 

Task-centered  instructional  strategy:  Teaching  compo¬ 
nent  skills  in  the  context  of  a  progression  of  real- 
world  whole  tasks. 

Topic-centered  instructional  strategy:  Teaching  com¬ 
ponent  skills  in  sequence  prior  to  their  application 
to  a  whole  task. 


FIRST  PRINCIPLES  OF  INSTRUCTION 

Merrill  reviewed  a  number  of  instructional  design  the¬ 
ories  and  models  (Dijkstra  et  al.,  1997;  Gagne,  1985; 
Glaser,  1992;  Marzano  et  al.,  2001;  McCarthy,  1996; 
Reigeluth,  1983,  1987,  1999;  Reigeluth  and  Carr- 
Chellman,  in  press;  Tennyson  et  al.,  1997;  van  Mer¬ 
rienboer,  1997)  in  an  attempt  to  identify  underlying 
prescriptive  principles  common  to  all  or  most  of  these 
approaches.  He  concluded  that  they  do  share  common 
principles  and  that  they  do  not  incorporate  fundamen¬ 
tally  different  principles  Merrill  (2002a,b;  2006a,b; 
2007;  in  press  a,b).  These  first  principles  are: 

•  Task-centered  approach  —  Learning  is  pro¬ 
moted  when  learners  are  engaged  in  a  task- 
centered  approach,  which  includes  demon¬ 
stration  and  application  of  component  skills. 

A  task-centered  approach  is  enhanced  when 
learners  undertake  a  progression  of  whole 
tasks. 

•  Activation  principle  —  Learning  is  pro¬ 
moted  when  learners  activate  relevant  cog¬ 
nitive  structures  by  being  directed  to  recall, 
describe,  or  demonstrate  relevant  prior 
knowledge  or  experience.  Activation  is 
enhanced  when  learners  recall  or  acquire  a 
structure  for  organizing  the  new  knowl¬ 
edge. 

•  Demonstration  principle — Learning  is  pro¬ 
moted  when  learners  observe  a  demonstra¬ 
tion  of  the  skills  to  be  learned  that  is 
consistent  with  the  type  of  content  being 
taught.  Demonstrations  are  enhanced  when 
learners  receive  guidance  that  relates 
instances  to  generalities.  Demonstrations  are 
enhanced  when  learners  observe  media  rel¬ 
evant  to  the  content. 

•  Application  principle— Learning  is  promoted 
when  learners  engage  in  the  application  of 
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their  newly  acquired  knowledge  or  skill  that 
is  consistent  with  the  type  of  content  being 
taught.  Application  is  effective  only  when 
learners  receive  intrinsic  or  corrective  feed¬ 
back.  Application  is  enhanced  when  learn¬ 
ers  are  coached  and  when  this  coaching  is 
gradually  withdrawn  for  each  subsequent 
task. 

•  Integration  principle — Learning  is  pro¬ 
moted  when  learners  integrate  their  new 
knowledge  into  their  everyday  life  by  being 
directed  to  reflect  on,  discuss,  or  defend  their 
new  knowledge  or  skill.  Integration  is 
enhanced  when  learners  create,  invent,  or 
extrapolate  personal  ways  to  use  their  new 
knowledge  or  skill  to  situations  in  their 
world.  Integration  is  enhanced  when  learners 
publicly  demonstrate  their  new  knowledge 
or  skill. 

In  this  chapter,  we  summarize  some  recent  pre¬ 
sentations  of  prescriptive  principles  and  compare 
them  to  the  first  principles  identified  by  Merrill. 
Here,  a  prescriptive  principle  is  a  relationship  that 
is  always  true  under  appropriate  conditions  regard¬ 
less  of  program  or  practice.  A  practice  is  a  specific 
instructional  activity.  A  program  is  a  set  of  pre¬ 
scribed  practices.  The  principles  identified  in  this 
chapter  are  underlying  relationships  rather  than  alter¬ 
native  instructional  models.  A  given  practice,  pro¬ 
gram,  or  model  always  implements  or  fails  to  imple¬ 
ment  underlying  principles,  whether  or  not  these 
principles  are  specified.  These  same  principles  can 
be  implemented  by  a  wide  variety  of  programs  and 
practices.  The  principles  identified  in  this  chapter 
share  the  following  properties:  First,  learning  from 
a  given  program  will  be  facilitated  in  direct  propor¬ 
tion  to  its  implementation  of  these  principles.  Sec¬ 
ond,  these  principles  of  instruction  can  be  imple¬ 
mented  in  any  delivery  system  or  using  any 
instructional  architecture  (Clark,  2003).  Third,  these 
principles  of  instruction  are  design  oriented  rather 
than  learning  oriented.  They  relate  to  creating  learn¬ 
ing  environments  and  products  rather  than  describing 
how  learners  acquire  knowledge  and  skill  from  these 
environments  or  products. 

Our  premise  is  that  these  design  principles  apply 
regardless  of  the  instructional  program  or  practices 
prescribed  by  a  given  theory  or  model.  If  this  premise 
is  true,  research  will  demonstrate  that,  when  a  given 
instructional  program  or  practice  violates  or  fails  to 
implement  one  or  more  of  these  underlying  princi¬ 
ples,  there  will  be  a  decrement  in  learning  and  per¬ 
formance. 


Elaboration  of  First  Principles 

These  principles  are  familiar  to  many  instructional 
designers  and  educators.  They  have  been  stated  in  one 
form  or  another  for  at  least  the  past  200  years.  Clark 
(1999)  indicated  that  J.  F.  Herbart’s  (1776-1841)  follow¬ 
ers  designed  a  five-step  teaching  method  that  is  remark¬ 
ably  similar  to  the  first  principles  described  earlier: 

•  Prepare  the  pupils  to  be  ready  for  the  new 
lesson  (activation). 

•  Present  the  new  lesson  (presentation). 

•  Associate  the  new  lesson  with  ideas  studied 
earlier  (activation,  guidance,  and  coaching). 

•  Use  examples  to  illustrate  the  lesson’s  major 
points  (demonstration) . 

•  Test  pupils  to  ensure  they  have  learned  the 
new  lesson  (application). 

Even  though  these  principles  seem  to  have  been  avail¬ 
able  for  some  time,  they  are  not  often  used  in  instruc¬ 
tional  materials.  An  extensive  survey  of  1400  online 
courses  in  marriage  relationships  from  five  countries 
demonstrated  that  most  of  the  instructional  programs 
reviewed  failed  to  implement  even  one  of  these  prin¬ 
ciples  (Barclay  et  al.,  2004). 

Four-Phase  Cycle  of  Instruction 

The  identification  of  the  first  principles  does  more  than 
merely  collect  a  set  of  prescriptive  principles  that 
might  be  used  to  select  or  design  effective  instruction. 
These  principles  are  interrelated  to  one  another.  The 
four-phase  cycle  of  instruction  consists  of  activation, 
demonstration,  application,  and  integration.  Effective 
instruction  involves  all  four  of  these  activities  repeated 
as  required  for  different  problems  or  whole  tasks.  A 
similar  four-phase  cycle  (4-MAT)  of  instruction  con¬ 
sisting  of  meaning  (activation),  conceptualizing  (dem¬ 
onstration),  operationalizing  (application),  and  renew¬ 
ing  (integration)  was  described  by  McCarthy  (1996). 
The  Vanderbilt  group  described  a  learning  cycle  con¬ 
sisting  of  a  set  of  challenges  (task  or  problem),  the 
generation  of  ideas  (activation),  multiple  perspectives 
(demonstration),  research  and  revision  (demonstra¬ 
tion/application),  testing  your  mettle  (application), 
going  public  (integration),  and  looking  ahead  and 
reflecting  back  (integration)  (Schwartz  et  al.,  1999). 

Problem-Centered  Instruction 

Perhaps  the  most  important  notion  of  the  first  princi¬ 
ples  is  that  engaging  instruction  is  problem  centered; 
that  is,  individual  instructional  components  are  most 
effectively  taught  in  the  context  of  a  progression  of 
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real-world  problems  where  the  student  is  shown  a 
problem,  then  taught  the  components,  and  then  shown 
how  the  components  are  used  to  solve  the  problem  or 
do  the  whole  task.  Van  Merrienboer’s  4C/ID  model 
for  training  complex  learning  tasks  makes  a  very 
strong  research-based  argument  for  centering  instruc¬ 
tion  in  whole  real-world  tasks  and  then  teaching  com¬ 
ponent  knowledge  and  skill  in  the  context  of  these 
tasks  (van  Merrienboer,  1997;  van  Merrienboer  and 
Kirschner,  2007). 

The  first  principles  and  4C/ID  identify  a  task-cen¬ 
tered  approach  that  combines  the  solving  of  problems 
with  more  direct  instruction  of  problem  components 
as  contrasted  with  problem-based  approaches  in  which 
students  are  placed  in  collaborative  groups,  given 
resources  and  a  problem,  and  left  to  construct  their 
own  solution  for  the  problem.  Research  supports  this 
guided  instruction  approach  over  more  pure  learner- 
centered  approaches  with  less  guidance.  Klahr  and 
Nigam  (2004)  compared  guided  direct  instruction  with 
a  discovery  learning  approach  for  children  learning 
about  confounding  variables  in  scientific  experiments. 
The  children  were  actively  involved  in  performing 
experiments.  The  direct  instruction  group  observed 
demonstration  experiments  (demonstration-guid¬ 
ance),  whereas  the  discovery  group  did  their  own 
experiments.  Klahr  and  Nigam  (2004,  p.  661)  demon¬ 
strated  that  “many  more  children  learned  from  direct 
instruction  than  from  discovery  learning”  and  that  chil¬ 
dren  in  the  direct  instruction  group  made  broader  and 
richer  scientific  judgments  about  science-fair  posters 
than  those  in  the  discovery  group.  Two  important 
research  reviews  have  argued  that  instruction  involving 
minimal  guidance  including  problem-based  teaching 
does  not  work,  whereas  task-centered  approaches 
involving  guidance  and  coaching  are  more  effective 
(Kirschner  et  al.,  2006;  Mayer,  2004). 

Levels  of  Instructional  Strategies 

Previous  papers  (Merrill,  2006a,b)  emphasized  that  the 
first  principles  promote  enhanced  performance  on 
complex  tasks.  To  assess  the  affects  of  the  first  prin¬ 
ciples  it  is  necessary  to  assess  learners’  scaled  perfor¬ 
mance  on  these  complex  tasks.  Some  methods  for 
determining  level  of  performance  include:  (1)  the  num¬ 
ber  of  tasks  completed  in  a  progression  of  subse¬ 
quently  more  difficult  tasks,  (2)  the  amount  of  coach¬ 
ing  required  for  satisfactory  performance  on  difficult 
tasks,  and  (3)  the  number  of  stages  performed  satis¬ 
factorily  in  a  nested  complex  task. 

Merrill  (2006a)  further  suggests  scaled  instruc¬ 
tional  strategies  based  on  the  first  principles.  He 
labeled  information-only  as  a  level  0  instructional 


strategy  and  suggested  a  series  of  yet-to-be-tested 
hypotheses  for  scaled  strategies:  (1)  A  level  1  instruc¬ 
tional  strategy  that  adds  consistent  demonstration  to  a 
level  0  information-only  strategy  promotes  a  higher 
performance  level  on  scaled  complex  tasks.  (2)  A  level 

2  instructional  strategy  that  adds  consistent  application 
with  corrective  feedback  to  a  level  1  instructional  strat¬ 
egy  consisting  of  information  plus  demonstration  pro¬ 
motes  an  additional  level  of  performance  on  complex 
real-world  tasks.  (3)  A  level  3  instructional  strategy 
that  consists  of  a  task-centered  instructional  strategy 
that  includes  consistent  demonstration  and  consistent 
application  with  corrective  feedback  promotes  an  addi¬ 
tional  increment  in  the  level  of  performance  on  com¬ 
plex  tasks.  (4)  Providing  or  recalling  relevant  experi¬ 
ence  promotes  an  additional  increment  in  learning 
efficiency,  effectiveness,  and  engagement  when  added 
to  a  level  1,  level  2,  or  level  3  instructional  strategies. 
(5)  Providing  activation-structure  promotes  an  addi¬ 
tional  increment  in  learning  efficiency,  effectiveness, 
and  engagement  when  added  to  level  1 ,  level  2,  or  level 

3  instructional  strategies.  (6)  Adding  reflection-inte¬ 
gration  to  any  of  the  above  instructional  strategies 
promotes  an  additional  increment  in  learning  effi¬ 
ciency,  effectiveness,  and  engagement.  (7)  Adding  cre- 
ate-integrate  to  any  of  the  above  instructional  strate¬ 
gies  promotes  transfer  of  the  newly  acquired 
knowledge  and  skill  to  performance  on  similar  tasks 
in  the  real  world  beyond  the  instructional  situation. 

Structure-Guidance-Coaching-Reflection  Cycle 

Based  on  a  meta-analysis  of  research  on  instruction, 
Marzano  and  colleagues  (2001,  p.  32)  stated  that: 
“In  general,  research  has  demonstrated  that  making 
students  aware  of  specific  structures  in  information 
helps  them  summarize  that  information  [and  subse¬ 
quently  be  able  to  use  this  information  more  effec¬ 
tively]”  (see  also  Marzano,  1998).  Rosenshine 
(1997)  indicated  that  when  students  organize  infor¬ 
mation,  summarize  information,  and  compare  old 
material  to  new  material  these  activities  require  pro¬ 
cessing  that  strengthens  cognitive  structures  and 
helps  students  develop  more  appropriate  mental 
models.  These  findings  suggest  that  during  activation 
students  should  be  provided  or  helped  to  develop  a 
structure  for  organizing  the  to-be-learned  informa¬ 
tion.  During  the  demonstration  phase,  guidance 
should  help  the  students  relate  the  new  information 
to  this  structure.  During  the  application  phase  coach¬ 
ing  should  help  students  use  this  structure  to  com¬ 
plete  the  task.  During  the  integration  phase,  reflec¬ 
tion  helps  students  to  incorporate  this  structure  into 
their  mental  model  for  subsequent  application. 
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Applications  of  the  First  Principles 

The  authors  are  aware  of  several  attempts  to  use  ver¬ 
sions  of  this  approach  for  the  development  of  instruc¬ 
tional  materials.  In  consultation  with  M.D.  Merrill, 
Thompson/NETg  applied  a  version  of  this  approach  to 
the  revision  of  their  course  for  Excel  (Thomson,  2002). 
The  resulting  scenario-based  course  taught  Excel  com¬ 
mands  in  the  context  of  five  spreadsheet  problems.  The 
test  was  to  prepare  three  additional  spreadsheet  tasks 
without  using  the  help  system  in  Excel.  Thompson/ 
NETg  undertook  a  study  to  validate  the  first  princi¬ 
ples  of  instruction  and  the  Pebble-in-the-Pond  model 
for  instructional  development.  Their  development 
group,  with  consultation  from  Merrill,  developed  sce¬ 
narios  for  a  course  in  Excel.  They  then  developed  a 
strategy  of  problem-progression-component- 
instruction  with  guidance  for  teaching  this  course  as 
prescribed  by  the  first  principles  and  Pebble-in-the- 
Pond.  The  investigators  selected  study  participants 
from  among  NETg  customers  who  volunteered  to 
participate  in  the  study. 

The  customers  were  divided  into  three  groups: 
Group  1,  the  scenario  group  (n  =  49),  received  instruc¬ 
tion  as  prescribed  by  the  first  principles.  Group  2,  the 
straight  e-learning  group  (n  =  49),  received  the  existing 
commercial  version  of  the  NETg  Excel  course.  This 
commercial  version  of  the  course  systematically 
teaches  all  of  the  commands  and  operations  of  Excel 
using  a  guided  demonstration  that  instructs  learners  to 
execute  a  command  or  series  of  commands  and  then  to 
observe  the  consequence  of  their  action  on  the  screen. 
This  same  instruction  was  used  for  the  component 
instruction  in  the  scenario  group.  Both  groups  had 
access  to  the  same  guided  demonstration  instruction  of 
individual  Excel  commands.  Group  3,  a  control  group 
(n  =  30),  received  the  final  three  authentic  scenarios 
without  any  prior  instruction  in  Excel.  The  instruction 
was  delivered  online  from  a  company  website  that  also 
provided  frequently  asked  questions  and  access  to  an 
online  mentor  for  both  experimental  groups. 

On  the  three  authentic  tasks,  the  scenario  group 
scored  an  average  of  89%,  the  guided  demonstration 
group  scored  68%,  and  the  control  group  scored  34%. 
All  differences  are  statistically  significant  beyond  the 
.001  level.  Further,  the  mean  times  required  to  complete 
the  three  authentic  tasks  were  29  minutes  for  the  sce¬ 
nario  group  and  49  minutes  for  the  guided  demonstra¬ 
tion  group.  Most  of  the  control  group  failed  to  finish  the 
tasks  so  no  time  data  were  recorded.  These  differences 
are  also  statistically  significant  beyond  the  .001  level. 
Finally,  on  a  qualitative  questionnaire,  the  scenario 
group  expressed  considerably  more  satisfaction  with  the 
course  than  did  the  guided  demonstration  group. 


The  first  principles  formed  the  basis  of  a  tool  used 
by  Shell  EP  to  evaluate  and  redesign  their  courses 
(Collis  and  Margaryan,  2005,  2007;  Margaryan, 
2006).  Their  instrument  was  applied  to  over  65 
courses  that  were  redesigned  to  be  in  greater  compli¬ 
ance  with  the  first  principles.  Twelve  courses  were 
studied  in  detail.  The  course-scan  values  (score  on 
the  first  principles  plus)  were  compared  with  partic¬ 
ipant  evaluation  data  and  instructors’  reflections.  The 
findings  indicate  that  both  the  students  and  the 
instructors  felt  that  the  work-based  model  increased 
the  business  relevance  of  the  courses,  led  to  deeper 
learning  processes  and  more  effective  performance, 
and  enabled  more  immediate  application  of  the  con¬ 
tent  to  their  jobs  (Margaryan,  2006). 

An  entrepreneur  course  for  distance  delivery  to 
students  in  developing  countries  was  developed  at 
Brigham  Young  University-Hawaii  using  the  first  prin¬ 
ciples  and  Pebble-in-the-Pond  (Mendenhall  et  al., 
2006a,b).  Six  principles  for  starting  a  business  were 
taught  in  the  context  of  five  small  businesses.  A  final 
exam  required  evaluating  a  business  plan  for  a  sixth 
small  business.  A  pilot  study  compared  performance 
on  this  exam  by  8  business  majors  who  had  completed 
several  previous  business  courses  to  12  non-business 
majors  who  had  completed  only  this  one  new  course. 
Of  the  12  non-business  majors,  7  scored  as  well  as  the 
business  majors.  At  this  writing,  further  development 
and  evaluation  of  the  course  are  in  progress. 

Frick  et  al.  (2007)  administered  an  instructor  eval¬ 
uation  form  that  included  items  that  allowed  students  to 
report  the  extent  to  which  the  first  principles  were  imple¬ 
mented  in  numerous  courses  at  multiple  institutions. 
These  items  were  then  correlated  with  student  self- 
report  of  academic  learning  time,  learning  achievement, 
and  learner  satisfaction.  Preliminary  results  showed  that 
the  use  of  the  first  principles  in  a  course  is  highly  cor¬ 
related  with  academic  learning  time  (Spearman’s  p  = 
0.682,  p<  .0005,  n—  111),  self-reported  student  learning 
achievement  (p  =  0. 823,  p<  .0005,;;=  1 10) ,  and  student 
satisfaction  (p  =  0.830,/?  <  .0005,  n  =  1 12).  The  internal 
consistency  reliability  of  the  first  principles  scale  was 
0.941  (Cronbach's  alpha).  Frick  suggests  that  the  inclu¬ 
sion  of  the  first  principles  in  instructor  evaluations 
would  increase  the  ability  of  such  instruments  to  assess 
instructional  quality. 

OTHER  INSTRUCTIONAL 
DESIGN  PRINCIPLES 

Some  recent  prescriptive  books  on  designing  effec¬ 
tive  instruction  have  stated  prescriptive  principles 
for  instructional  design.  Merrill  (2007)  presented  a 
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TABLE  14.1 

Clark  and  Mayer’s  Principles  for  e-Learning  Aligned  with  Merrill’s  First  Principles 


E-Learning  Principles  (Clark  and  Mayer,  2003) 

Interactions  should  mirror  the  job  (p.  153). 

Critical  tasks  require  more  [distributed]  practice  (p.  159). 

Use  job  contexts  to  teach  problem-solving  processes  (p.  251). 
Incorporate  job-specific  problem-solving  processes  (p.  264). 

Use  job-realistic  or  varied  worked  examples  (p.  186). 

Replace  some  practice  problems  with  worked  examples  (p.  177). 
Apply  the  media  elements  principles  to  examples  (p.  179). 

Apply  the  media  elements  principles  to  practice  exercises  (p.  164), 
Train  learners  to  self-question  during  receptive  e-lessons  (p.  166). 
Teach  learners  to  self-explain  examples  (p.  190). 

Make  learners  aware  of  their  problem-solving  processes  (p.  260). 
Focus  training  on  thinking  processes  vs.  job  knowledge  (p.  256). 


First  Principles  (Merrill,  2002) 

Task-centered 

Task-centered-progression 

Task-centered 

Task-centered 

Task-centered 

Demonstration 

Demonstration-guidance 

Demonstration-media 

Application 

Integration-reflect 

Integration-reflect 

Integration-reflect 

Integration-extrapolate 


synthesis  of  some  of  these  sources  as  they  relate  to  the 
first  principles  of  instruction.  The  following  summa¬ 
rizes  some  of  the  principles  identified  by  other  authors 
and  attempts  to  relate  them  to  the  first  principles  of 
instructional  design. 

Principles  for  Multimedia  Learning 

Based  on  extensive  research,  Clark  and  Mayer  (Clark 
and  Mayer,  2003;  Mayer,  2001)  have  identified  prin¬ 
ciples  for  multimedia  learning.  These  principles  elab¬ 
orate  the  demonstration  principle  for  relevant  media 
(Mayer,  2001): 

•  Students  learn  better  from  words  and  pic¬ 
tures  than  from  words  alone  (p.  63). 

•  Students  learn  better  when  corresponding 
words  and  pictures  are  presented  near  rather 
than  far  from  each  other  on  the  page  or 
screen  (p.  81). 

•  Students  learn  better  when  corresponding 
words  and  pictures  are  presented  simulta¬ 
neously  rather  than  successively  (p.  96). 

•  Students  learn  better  when  extraneous  mate¬ 
rial  is  excluded  rather  than  included  (p.  113). 

•  Students  learn  better  from  animation  and 
narration  than  from  animation  and  on-screen 
text  (p.  134). 

•  Students  learn  better  from  animation  and 
narration  than  from  animation  and  narration 
and  text  (p.  147). 

•  Design  effects  are  stronger  for  low-knowl¬ 
edge  learners  than  for  high-knowledge  learn¬ 
ers  and  for  high- spatial  learners  rather  than 
low-spatial  learners  (p.  161). 


Principles  for  e-Learning 

In  addition  to  the  multimedia  principles  Clark  and 
Mayer  (2003)  recommend  additional  instructional  prin¬ 
ciples  (Table  14.1).  Correspondence  to  the  first  princi¬ 
ples  is  indicated  in  the  right  column  for  each  of  these 
principles.  It  should  be  evident  to  the  reader  that  some 
of  these  principles  have  a  close  correspondence,  some 
of  these  principles  provide  more  elaboration  than  the 
first  principles,  and  some  are  not  included  in  the  first 
principles.  They  include  personalization,  collaboration, 
and  learner-control  principles;  the  first  principles  do  not 
include  principles  related  to  implementation  of  instruc¬ 
tion,  such  as  personalization,  collaboration,  learner 
control,  and  navigation.  Perhaps  they  should.  Allen 
(2003)  identified  some  effective  principles  for  e-leam- 
ing  in  three  major  categories:  learner  motivation,  nav¬ 
igation,  and  instructional  interactivity.  Table  14.2  sum¬ 
marizes  his  keys  for  motivating  e-learning  and  his 
checklist  for  good  instructional  interactions. 

Minimalist  Principles 

Van  der  Meij  (1998)  identified  heuristics  for  designing 
minimalist  instruction  (Table  14.3).  The  task-centered 
orientation  of  these  heuristics  is  apparent.  Some  of 
these  demonstration,  application,  guidance,  and  coach¬ 
ing  heuristics  provide  more  specific  prescriptions  than 
the  first  principles. 

Cognitive  Training  Model 

Foshay  et  al.  (2003)  presented  a  cognitive  training 
model  that  identifies  five  tasks  learners  have  to  complete 
when  learning:  (1)  select  the  information  to  attend  to. 
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TABLE  14.2 

Allen’s  e-Learning  Principles  Aligned  with  Merrill’s  First  Principles 


e-Learning  Principles  (Allen,  2003) 

Have  the  learner  perform  multistep  tasks;  having  people  attempt  real  (or  “authentic”)  tasks  is 
much  more  interesting  than  having  them  repeat  or  mimic  one  step  at  a  time  (p.  209). 

The  learner  must  apply  genuine,  authentic  skills  and  knowledge;  activities  should  be  as  similar 
to  the  necessary  on-the-job  performance  as  possible  (p.  276). 

Good  interactions  are  purposeful  in  the  mind  of  the  learner;  learners  understand  what  they  can 
accomplish  through  participation  in  the  interaction  (p.  276). 

Build  on  anticipated  outcomes;  help  learners  see  how  their  involvement  in  the  e-learning  will 
produce  outcomes  they  care  about  (p.  158). 

Select  the  right  content  for  each  learner;  if  it  is  meaningless  or  learners  already  know  it,  it  is  not 
going  to  be  an  enjoyable  learning  experience  (p.  179). 

Use  an  appealing  context;  novelty,  suspense,  fascinating  graphics,  humor,  sound,  music,  and 
animation  all  draw  learners  in  when  done  well  (p.  193).a 

Provide  intrinsic  feedback:  Seeing  the  positive  consequences  of  good  performance  is  better 
feedback  than  being  told,  “Yes,  that  was  good”  (p.  214);  the  feedback  demonstrates  to  learners 
the  ineffectiveness  (even  risks)  of  poor  responses  and  the  value  of  good  responses  (p.  276). 

Delay  judgment;  if  learners  have  to  wait  for  confirmation,  they  will  typically  reevaluate  for 
themselves  while  the  tension  mounts— essentially  reviewing  and  rehearsing  (p.  220). 

Put  learners  at  risk;  if  learners  have  something  to  lose  they  pay  attention  (p.  169). 


First  Principles  (Merrill,  2002) 

Task-centered 

Task-centered 

Task-centered 

Activation 

Activation  (show  the  whole  problem) 
Activation-prior  knowledge 
Demonstration-media 
Application-intrinsic  feedback 


Application-feedback 
Integration-publicly  demonstrate 


a  When  implementing  this  guideline  one  would  do  well  to  remember  Mayer’s  (2001)  multimedia  principles. 


TABLE  14.3 

Minimalist  Instruction  and  Merrill’s  First  Principles 

Minimalist  Principles  (van  Meij,  1998) 

Provide  an  immediate  opportunity  to  act  (p.  22). 

Select  or  design  instructional  activities  that  are  real  tasks  (p.  29). 

Be  sure  the  components  of  the  task  reflect  the  task  structure  (p.  31). 

Prevent  mistakes  whenever  possible  (p.  35). 

Provide  error  information  that  supports  detection,  diagnosis,  and  recovery  (p.  38). 

Be  brief;  do  not  spell  out  everything  (p.  43). 

Provide  closure  for  chapters  (p.  44). 

Respect  the  integrity  of  the  user’s  activity  (p.  25). 

Provide  on-the-spot  error  information  (p.  41). 

Provide  error  information  when  actions  are  error  prone  or  when  correction  is  difficult  (p.  37). 
Encourage  and  support  exploration  and  innovation  (p.  23). 


First  Principles  (Merrill,  2002) 

Task-centered 

Task-centered 

Task-centered-components 

Demonstration-guidance 

Demonstration-guidance 

Demonstration 

Guidance  and  coaching 

Application 

Application-coaching 

Application-coaching 

Integration-reflect,  extrapolate 


(2)  link  the  new  information  with  existing  knowledge, 

(3)  organize  the  information,  (4)  assimilate  the  new 
knowledge  into  existing  knowledge,  and  (5)  strengthen 
the  new  knowledge  in  memory.  In  association  with 
these  tasks,  they  identify  17  elements  (prescriptive  prin¬ 
ciples  for  design)  of  a  training  lesson  (Table  14.4). 

Instructional  Principles  Based 
on  Learning  Principles 

Seidel  et  al.  (2004)  identified  factors  influencing  the 
acquisition  and  transfer  of  learning  as  empirically 
established  in  the  learning  literature  .  From  these  learn¬ 


ing  principles  they  derived  guidelines  for  more  effec¬ 
tive  instructional  design.  They  organized  their  princi¬ 
ples  around  four  domains:  the  cognitive  domain,  the 
affective  domain,  the  psychomotor  domain,  and  the 
interpersonal  domain.  Table  14.5  summarizes  their 
design  guidelines  for  the  cognitive  domain. 

4C/ID  Instructional  Design 

Perhaps  the  most  complete  recent  presentation  of 
instructional  strategies  is  the  work  of  van  Merrienboer 
(1997;  van  Merrienboer  and  Kirschner,  2007).  The 
model  presents  an  analysis  and  design  approach  that 
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TABLE  14.4 

The  Cognitive  Training  Model  and  Merrill’s  First  Principles 


Cognitive  Training  Model  Principles  (Foshay  et  al.,  2003,  p.  29) 

Answer  the  question  “What’s  in  it  for  me?”  for  the  learners. 

Specify  both  the  desired  behavior  and  the  knowledge  to  be  learned. 

Gain  and  focus  learners’  attention  on  the  new  knowledge. 

Bring  to  the  forefront  the  prerequisite  existing  (old)  knowledge  that  forms  the 
base  on  which  the  new  knowledge  is  built. 

Organize  text  presentation  to  help  learners  organize  new  knowledge. 

Tell  the  learners  “You  can  do  it!”  regarding  learning  new  knowledge. 

Demonstrate  real-life  examples  of  how  the  new  knowledge  works  when  it  is 
applied. 

Organize  and  limit  the  amount  of  new  knowledge  presented  to  match  human 
information  processing  capacity. 

Use  well-designed  illustrations  to  assist  learners’  organization  and  assimilation 
of  new  knowledge. 

Using  a  different  approach  for  each  type  of  knowledge,  present  the  new 
knowledge  in  a  way  that  makes  it  easiest  to  understand. 

Involve  learners  by  having  them  do  something  with  the  new  knowledge. 

Let  learners  know  how  well  they  have  done  in  using  the  new  knowledge,  what 
problems  they’re  having,  and  why. 

Present  the  structure  of  the  content  again,  including  the  entire  structure  of 
knowledge. 

Have  the  learners  use  the  new  knowledge  again,  this  time  to  prove  to 
themselves,  you,  and  their  employer  that  they  have  met  the  objectives  of  the 
training. 

Have  learners  use  new  knowledge  in  a  structured  way  on  the  job  to  ensure  they 
“use  it,  not  lose  it.” 


First  Principles  (Merrill,  2002) 

Task-centered  (show  the  task  to  be  completed) 

Task-centered  (they  are  more  accepting  of  traditional  objectives) 
Activation 

Activation-prior  knowledge 
Activation-structure 

Activation  (not  included  in  first  principles) 

Task-centered 

Demonstration 

Demonstration 

Demonstration 

Demonstration-consistency  (see  Gagne,  1985;  Merrill,  1994) 
Application 

Application-feedback  (could  be  both  intrinsic  and  extrinsic) 
Activation-structure  with  demonstration  guidance 
Application 


Integration-extrapolate 


TABLE  14.5 

Instructional  Principles  for  the  Cognitive  Domain  and  Merrill’s  First  Principles 


Instructional  Principles  (Seidel  et  al.,  2004,  p.  24) 

Provide  multiple  context  environments  to  facilitate  positive 
transfer  within  and  across  domains. 

Use  advanced  organizers  to  facilitate  integrative  skill 
acquisition  and  capitalize  on  prior  knowledge. 

Apply  the  operant  principles  of  minimizing  errors,  using 
small  steps,  and  providing  immediate  reinforcement  for 
acquisition  of  initial  elements  of  domain  knowledge. 

Use  part-task  training  to  break  up  complex  tasks  into 
manageable  chunks. 


First  Principles  (Merrill,  2002) 

Task-centered-progression  of  tasks 

Activation-structure  (coupled  with  the  structure-guidance- 
coaching-reflection  cycle) 

Demonstration-consistency  (especially  for  acquiring  information) 

Task-centered 

Demonstration-component  skills 


encompass  all  of  the  first  principles  of  instruction.* 
4C/ID  identifies  four  layers  of  activity  in  the  instruc¬ 
tional  development  process:  (1)  principled  skill 
decomposition,  (2)  analysis  of  constituent  skills  and 
related  knowledge,  (3)  selection  of  instructional  meth¬ 
ods,  and  (4)  development  of  a  learning  strategy.  The 


*  4C/ID  is  an  integrated  approach  for  which  individual  prescriptive 
statements  are  difficult  to  isolate.  We  have  tried  to  summarize  key 
aspects  of  this  model  and  relate  them  to  the  first  principles. 


first  principles  have  little  to  say  about  decomposing  a 
complex  skill  into  a  hierarchy  of  constituent  skills; 
however,  the  analysis  of  these  constituent  skills  in 
4C/ID  is  similar  to  the  analysis  of  a  problem  to  identify 
the  component  knowledge  and  skill  for  a  given  task 
(see  Figure  14.1).  “At  the  heart  of  this  training  method 
is  whole-task  practice,  in  which  more  and  more  com¬ 
plex  versions  of  the  whole  complex  cognitive  skill  are 
practiced”  (progression  of  whole  tasks)  (van  Merrien- 
boer,  1997,  p.  8).  The  constituent  skills  required  for 
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Components  for  a  Whole  Task 


1 .  How  to: 

2.  Kinds  of 

3.  Kinds  of 

4.  Kinds  of 

5.  How  to: 

6.  How  to: 


Series  of  tasks  leading  to  desired  consequence 
Specific  document  or  activity  resulting  from  each  task 
Defining  properties  for  each  artifact 
Ordering  properties  for  each  artifact 
Procedure  for  creating  or  selecting  each  artifact 
Example  scenario  for  carrying  out  procedure 


Figure  14.1  Components  for  a  whole  task.  (From  Merrill,  M.D.,  Learning  Objects  for  Task-Centered  Instruction,  2006, 
http://cito  .byuh  .edu/merrill/Merrill_  1  /Merrill- 1  .html .) 


performance  of  the  whole  task  are  presented  as  they 
are  required  by  the  task  (task-centered  approach).  Key 
aspects  of  the  just-in-time  information  presentation 
include  partitioning,  demonstration  (consistent  dem¬ 
onstration  and  guidance),  and  fading  (coaching).  The 
model  stresses  the  identification  of  cognitive  schemata 
that  underlie  the  performance  of  nonrecurrent  aspects 
of  complex  tasks.  To  promote  schema  induction,  in 
addition  to  teaching  the  recurrent  constituent  skills 
presented  just  in  time,  the  model  also  includes  presen¬ 
tation  of  heuristic  information  to  help  with  unfamiliar 
aspects  of  problems  (structure-activation).  The  “basic 
claim  of  the  4C/ID-model  is  that  its  application  leads 
to  reflective  expertise  and,  consequently,  to  increased 
transfer  performance”  (integration)  (van  Merrienboer, 
1997,  p.  73). 


DESIGNING  TASK-CENTERED 
INSTRUCTION 

Figure  14.2  illustrates  a  Pebble-in-the-Pond  approach 
to  instructional  development  that  assists  designers  to 
systematically  incorporate  the  first  principles  into  their 
instructional  design  (Merrill,  2002b).  The  steps  in  this 
approach  are  to:  (1)  specify  a  whole  task,  (2)  specify 
a  progression  of  whole  tasks,  (3)  specify  the  compo¬ 
nent  knowledge  and  skills  required  for  each  task,  (4) 
specify  an  instructional  strategy,  (5)  specify  the  user 
interface,  and  (6)  produce  the  course. 


Pebble-in-the-Pond  Instructional  Design 

Figure  14.2  The  Pebble-in-the-Pond  approach  to  instruction. 
(From  Merrill,  M.D.,  Perform.  Improve.,  41(7),  39-44,  2002. 
With  permission.) 

Merrill  (2006c)  elaborated  the  analysis  required  to 
specify  the  component  knowledge  and  skill  for  each 
task  in  terms  of  portrayals  of  specific  artifacts  and 
information  for  component  concepts  (kinds  of)  and 
component  procedures  (how  to)  (Merrill,  1997).  This 
component  analysis  (Figure  14.1)  consists  of  the  fol¬ 
lowing  steps:  (1)  find  a  portrayal  of  the  artifact  that  is 
a  consequence  of  completing  the  whole  task  (kind  of); 
(2)  identify  a  series  of  subtasks  (information)  leading 
to  the  desired  consequence  (how  to);  (3)  for  each  sub¬ 
task,  find  a  portrayal  of  the  artifact  that  is  a  conse¬ 
quence  of  completing  this  subtask  (kind  of);  (4)  iden¬ 
tify  the  defining  properties  of  each  artifact  (kind  of); 
(5)  identify  the  ordering  properties  of  each  artifact 


Whole  Task 

Progression 

Components 

Strategy 

Interface 

Production 
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Figure  14.3  Topic-centered  instructional  sequencing.  (From 
Mendenhall,  A.  et  al..  Introduction  to  Entrepreneurship:  How 
to  Start  Your  Own  Business,  2006,  http://cito.byuh.edu/entre- 
preneur/  main.swf.) 

(kind  of);  (6)  identify  the  procedure  for  creating  or 
selecting  the  artifact  (how  to);  and  (7)  identify  a  por¬ 
trayal  of  a  scenario  illustrating  this  procedure  (Merrill, 
2006c). 

Figure  14.3  illustrates  a  typical  topic-centered 
instructional  strategy.  In  this  strategy,  each  topic  is 
taken  in  turn.  Information  is  presented  and  demon¬ 
strated.  Periodic  evaluation  is  administered  to  assess 
the  information  that  is  presented.  Figure  14.4  illus¬ 
trates  a  task-centered  instructional  strategy  consistent 


with  the  first  principles  of  instruction  and  the  Pebble- 
in-the-Pond  approach  to  instructional  design.  In  this 
strategy,  a  whole  task  is  demonstrated;  some  level  of 
each  of  the  relevant  topics  is  presented  and  then  dem¬ 
onstrated  in  the  first  task.  A  second  whole  task  is  then 
presented.  The  learner  is  asked  to  apply  those  topics 
that  were  presented  to  the  new  task.  An  expanded 
version  of  the  topics  relevant  to  the  second  task  is 
presented  and  demonstrated  for  the  second  task.  This 
strategy  is  repeated  for  several  more  tasks  until  all  the 
topics  have  been  expanded  as  much  as  required  by  the 
final  tasks  and  the  student  is  able  to  apply  the  topics 
to  a  new  task  unaided. 

A  new  book  designed  to  help  designers  use  the 
4C/ID  model  adapts  the  Pebble-in-the-Pond  model  of 
content-first  instructional  development  as  a  recom¬ 
mended  application  of  the  ten  steps  to  designing  instruc¬ 
tion  for  complex  skills  (van  Merrienboer  and  Kirschner, 
2007).  Table  14.6  compares  the  ten  development  steps 
of  van  Merrienboer  and  Kirschner  to  the  ripples  in  the 
Pebble-in-the-Pond  model  for  instructional  design. 

CONCLUSION 

Considerable  agreement  exists  with  regard  to  the  pre¬ 
scriptive  instructional  design  principles  that  are  funda¬ 
mental  to  effective,  efficient,  and  engaging  instruction, 
and  the  first  principles  of  instructional  design  appear 


Task-Centered  Instructional  Strategy 

1 .  Show  a  new  whole  task. 

2.  Present  topic  components  specific  to  the  task. 

3.  Demonstrate  the  topic  components  for  the  task. 

4.  Show  another  new  whole  task. 

5.  Have  learners  apply  previously  learned  topic  components  to  the  task. 

6.  Present  additional  topic  components  specific  to  this  task. 

7.  Demonstrate  the  application  of  these  additional  topic  components. 

8.  Repeat  apply-present-demonstrate  cycle  (steps  4-7)  for  subsequent  tasks. 


Topic  1 
Topic  2 
Topic  3 
Topic  4 
Topic  5 


Learners  are 
able  to 
complete  a 
new  task 
without 
further 
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Figure  14.4  Task-centered  instructional  sequencing.  (Adapted  from  Mendenhall,  A.  et  al..  Introduction  to  Entrepreneurship:  How 
to  Start  Your  Own  Business,  2006,  http://cito.byuh.edu/entrepreneur/main.swf;  Merrill,  M.D.,  J.  Res.  Technol.  Educ.,  in  press.) 
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TABLE  14.6 

Pebble-in-the-Pond  and  Ten  Steps  to  Complex  Learning 


Ten  Steps  to  Complex  Learning 

Design  learning  tasks. 

Sequence  task  classes. 

Set  performance  objectives.0 
Design  supportive  information. 
Analyze  cognitive  strategies. 
Analyze  mental  models. 

Design  procedural  information. 
Analyze  cognitive  rules. 

Analyze  prerequisite  information. 
Design  part-task  practice. 

a  van  Merrienboer  and  Kirschner  (2007)  combine  analysis  and  strategy  as  they 
discuss  component  skills;  Pebble-in-the-Pond  separates  these  two  design  functions. 
b  Ten  Steps  has  only  a  minimal  discussion  on  interface,  delivery,  and  production; 
their  emphasis  is  on  the  design  aspects  of  instructional  development. 
c  Pebble-in-the-Pond  does  not  specify  assessment  criteria  for  the  tasks.  This  is  an 
area  where  the  model  should  be  improved. 

Source:  van  Merrienboer,  J.J.G.  and  Kirschner,  P.A.,  Ten  Steps  to  Complex  Learning, 
Lawrence  Erlbaum  Associates,  Mahwah,  NJ,  2007.  With  permission. 


Pebble-in-the-Pond 

Identify  a  whole  task  or  problem. 

Specify  a  progression  of  whole  tasks. 
Analyze  the  component  skills  for  each  task. 
Specify  an  instructional  strategy.3 
Determine  user  interface.15 
Produce  the  course. 


to  have  a  fair  amount  of  agreement.  The  limited  data 
available  indicate  that,  when  these  principles  are  imple¬ 
mented  in  instructional  products  and  environments,  the 
instructional  quality  increases;  however,  far  too  much 
instruction  seems  to  ignore  these  fundamental  princi¬ 
ples.  It  is  hoped  that  this  chapter  has  provided  a  starting 
point  for  more  rigorous  research  to  assess  the  validity 
of  these  principles  in  more  situations,  for  more  stu¬ 
dents,  and  for  a  greater  variety  of  subject  matters.  In 
the  meantime,  a  greater  effort  should  be  made  to  imple¬ 
ment  these  principles  in  our  instructional  products. 
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Part  III 

Technologies 


This  Technologies  part  of  the  Handbook  was  led  by 
J.  Michael  Spector.  The  various  chapters  address  both 
upstream  technologies  (analysis,  planning,  and 
design)  and  downstream  technologies  (development, 
deployment,  and  evaluation).  This  is  the  largest  part 
of  the  Handbook ,  with  sixteen  chapters  covering:  (1) 
programmed  technologies;  (2)  educational  hypter- 
text;  (3)  computer-mediated  technologies;  (4)  com¬ 
munications  technologies;  (5)  media  centers;  (6) 
knowledge -based  technologies;  (7)  flexible  learning; 
(8)  enabling  time  and  place  independence;  (9) 


blended  learning;  (10)  adaptive  technologies;  (11) 
technologies  for  generations  Y  and  Z;  (12)  synthetic 
learning  environments;  (13)  technologies  linking 
learning,  cognition,  and  instruction;  (14)  modeling 
technologies;  (15)  learning  objects;  and  (16)  open 
source  and  open  standards .  This  part  of  the  Handbook 
provides  readers  with  a  comprehensive  review  of 
recent  research  pertaining  to  a  variety  of  instructional 
technologies  that  can  be  linked  with  Strategies  in  Part 
II,  with  Models  in  Part  IV,  and  with  Design  and 
Development  in  Part  V. 
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ABSTRACT 

Programmed  technologies  of  instruction  have  played  an 
important  role  in  the  origins  and  development  of  the 
field  of  instructional  design  and  technology  (IDT).  This 
chapter  addresses  the  historical  evolution  of  pro¬ 
grammed  technologies,  their  underlying  psychological 


principals  and  characteristics,  and  the  extensive  research 
base  that  formed  the  early  foundations  of  the  IDT  field. 
The  most  prominent  models  of  programmed  technolo¬ 
gies  are  reviewed,  including  programmed  instruction 
(PI),  Personalized  System  of  Instruction  (PSI),  and  Pre¬ 
cision  Teaching  (PT).  Programmed  technologies  repre¬ 
sent  some  of  the  most  empirically  investigated  forms  of 
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instruction,  with  decades  of  studies  that  examined 
their  effectiveness.  The  modified  application  of  pro¬ 
grammed  technologies  in  current  models  of  teaching 
and  learning  reflects  the  long-standing  impact  of 
these  design  approaches  on  instructional  systems 
development. 


KEYWORD 

Programmed  technologies:  Process-based  methods 
and  approaches  to  support  learning  and  instruction, 
often  represented  in  the  form  of  algorithms  and 
implemented  in  computer  software. 


INTRODUCTION 

Mention  instructional  technology  to  anyone  outside 
the  field  and  the  term  evokes  mental  pictures  of  com¬ 
puter  hardware  and  software,  cameras,  projectors,  and 
the  like;  however,  instructional  technology  involves 
much  more  than  the  materials  and  equipment  used  in 
instruction.  These  are  merely  the  products  of  instruc¬ 
tion.  Instructional  technology  (IT)  is  an  applied  field 
and,  as  such,  implies  a  process,  perhaps  made  more 
clear  by  talking  about  instructional  design  and  tech¬ 
nology  (IDT).  The  applied  field  of  accounting  requires 
the  application  of  pure  mathematical  theories.  Cooking 
and  baking  require  the  application  of  chemical  and 
physical  theories  to  human  nutrition;  likewise,  the 
applied  field  of  instructional  technology  (referred  to 
as  IDT  hereafter)  requires  the  application  of  theories 
from  educational  psychology,  computer  science,  man¬ 
agement  science,  communications,  and  many  other 
disciplines. 

The  process  of  instructional  technology  involves 
analyzing  instructional  problems  and  needs,  selecting 
strategies  to  meet  those  needs,  designing  and  develop¬ 
ing  materials  to  support  those  strategies,  and  evaluat¬ 
ing  the  success  of  the  design  during  and  after  imple¬ 
mentation.  Often,  this  process  results  in  an 
instructional  product,  but  occasionally  it  results  in  yet 
another  process  — an  instructional  process  or  tech¬ 
nique.  These  instructional  techniques  can  be  simple  or 
elaborate  and  are  developed  through  the  application  of 
learning  theories  to  a  specific  instructional  problem. 
In  this  chapter,  these  process-based  instructional  tech¬ 
niques  are  referred  to  as  programmed  technolo¬ 
gies—  programmed  because  they  are  process  based  and 
do  not  necessarily  require  the  use  of  electronic  media 
for  delivery  (although  many  programmed  technologies 
are  capable  of  being  conveyed  via  electronic  media), 
and  technologies  in  the  sense  that  the  user  applies 


scientific  theory,  or  a  systematic  technique,  method,  or 
approach  to  solve  a  problem. 

This  chapter  addresses  several  of  the  most  well 
known  of  these  programmed  technologies,  including 
programmed  instruction,  Keller’s  Personalized  System 
of  Instruction,  and  Precision  Teaching.  In  each  case, 
learning  theories  form  the  basis  of  the  programmed 
technology,  serving  to  justify  its  use  and  application 
to  meet  instructional  needs  and  solve  instructional 
problems.  A  study  of  these  programmed  technologies 
provides  a  valuable  example  of  how  to  systematically 
apply  learning  theory  to  develop  an  instructional  pro¬ 
cess  or  an  instructional  product. 


PROGRAMMED  INSTRUCTION 

Probably  the  best  known  programmed  technology  is 
programmed  instruction  (PI),  considered  by  many  to 
be  a  precursor  to  contemporary  IDT  (McDonald  et 
al.,  2005).  A  close  examination  of  programmed 
instruction  can  build  an  insightful  appreciation  for 
IDT.  Programmed  instruction  was  an  integral  factor 
in  the  evolution  of  the  instructional  design  process 
and  served  as  the  foundation  for  the  procedures  used 
currently  to  develop  effective  learning  environments 
by  instructional  designers  and  performance  technol¬ 
ogy  professionals.  The  foundations  of  computer- 
mediated  instruction  are  based  on  PI  theory  and 
research,  and  the  term  programming  was  applied  to 
the  production  of  print-based  learning  materials  long 
before  it  was  used  to  refer  to  computer-based  learning 
materials.  PI  was  the  first  empirically  determined 
form  of  instruction;  it  played  a  prominent  role  in  the 
convergence  of  science  and  education  and  represents 
a  milestone  in  the  legitimacy  of  the  field.  Equally 
important  is  its  impact  on  the  evolution  of  the  instruc¬ 
tional  design  and  development  process.  Finally,  many 
of  the  same  design  factors  emphasized  by  those 
involved  in  the  early  development  of  PI  are  still  valid 
for  instructional  designers  today:  presenting  infor¬ 
mation  in  small  steps,  providing  immediate  knowl¬ 
edge  of  results,  providing  positive  reinforcement, 
allowing  self-pacing,  and  minimizing  the  rate  of 
error  (Deutsch,  1992). 

Historical  Origins  of  Programmed  Instruction 

Although  attempts  at  processes  resembling  programmed 
instruction  date  back  to  the  1920s  (Pressey,  1926),  the 
actual  term  is  probably  derived  from  B.  F.  Skinner’s 
(1954)  paper,  "The  Science  of  Learning  and  the  Art 
of  Teaching,”  presented  at  the  University  of  Pitts¬ 
burgh’s  conference  of  Current  Trends  in  Psychology 
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and  the  Behavioral  Sciences  on  March  12, 1954.  Skin¬ 
ner’s  remarks  reflected  his  reaction  to  a  1953  visit  to 
his  daughter’s  fourth-grade  arithmetic  class  (Vargas 
and  Vargas,  1992).  Skinner  (1954,  pp.  90-91)  argued 
that  schools  were  unable  to  accomplish  the  type  of 
teaching  that  eventually  leads  to  original  thinking 
because: 

•  Schools  relied  on  aversive  stimulation  or 
control;  as  Skinner  described  it,  children 
worked  to  “avoid  or  escape  punishment.” 

•  Schools  did  not  pay  attention  to  the  contin¬ 
gencies  of  reinforcement. 

•  Schools  lacked  a  systematic  plan  for  learn¬ 
ing  skills,  or,  in  Skinner’s  words,  “a  skillful 
program  which  moves  forward  through  a 
series  of  progressive  approximations  to  the 
final  complex  behavior  desired.” 

•  Schools  too  infrequently  provided  reinforce¬ 
ment. 

Skinner  suggested  a  systematic  plan— or  programmed 
instruction— as  the  vehicle  to  accomplish  the  changes 
that  needed  to  occur  in  classrooms,  and  in  his  descrip¬ 
tion  of  that  plan  he  made  two  statements  that  illustrate 
the  importance  of  instructional  design  and  its  relation¬ 
ship  to  technology.  He  stated  that  “education  is  perhaps 
the  most  important  branch  of  scientific  technology” 
(1954,  p.  93),  and  “in  the  present  state  of  our  knowl¬ 
edge  of  educational  practices,  scheduling  [of  behaviors 
and  consequences]  appears  to  be  most  effectively 
arranged  through  the  design  of  the  material  to  be 
learned”  (p.  94,  emphasis  added).  Skinner  was  at  the 
forefront  in  articulating  the  need  to  accomplish  this 
scheduling  of  behaviors  and  consequences  and  a  pro¬ 
gram  for  effective  and  efficient  learning  through  oper¬ 
ant  conditioning .  Operant  conditioning  is  a  form  of 
conditioning  that  reinforces  desired  behavior  and  it  is 
this  behaviorist  theory  that  forms  the  basis  for  pro¬ 
grammed  instruction. 

In  his  1954  work,  Skinner  went  on  to  discuss  the 
potential  for  mechanical  devices  to  provide  feedback, 
feeling  that  the  teacher’s  time  was  better  spent  in  the 
design  of  the  instruction  and  diagnosis  of  student 
progress.  In  his  article  “Teaching  Machines,”  pub¬ 
lished  in  Science  (1958a),  Skinner  pushed  harder  for 
the  use  of  technology  in  education  that  could  present 
instructional  material  prepared  by  programmers.  In 
this  work,  he  also  emphasized  that,  if  good  program¬ 
ming  was  to  become  a  scientific  technology  rather  than 
an  art,  it  would  depend  on  the  use  of  student  perfor¬ 
mance  data  in  making  revisions.  Again,  he  envisioned 
the  powerful  roll  that  machines  could  play  in  collecting 
these  data. 


A  discussion  of  the  history  of  programmed  instruc¬ 
tion  is  not  complete  without  reference  to  its  implemen¬ 
tation  in  teaching  machines  designed  to  guide  instruc¬ 
tional  events.  Skinner  (1958a)  and  others  (e.g..  Hartley 
and  Davies,  1978)  credited  Sidney  Pressey  with  devel¬ 
oping  the  first  teaching  machine.  Beginning  in  the 
1920s,  Pressey  designed  machines  for  administering 
tests.  Hartley  and  Davies  (1978)  correctly  pointed  out 
that  Pressey ’s  devices  were  used  after  the  instruction 
took  place,  but  more  important  to  Skinner  was  Pres¬ 
sey ’s  (1926)  understanding  that  not  only  could  such 
machines  test  and  score  but  they  could  also  teach. 

During  the  1950s,  educators  and  psychologists 
became  concerned  that  the  mass  schooling  precipitated 
by  increasing  demands  on  public  education  were  not 
meeting  an  individual’s  needs  for  personal  attention  in 
the  learning  process,  and  they  suggested  that  teaching 
machines  could  restore  the  “important  features  of  per¬ 
sonal  instruction”  (Skinner,  1986,  p.  103).  Additional 
teaching  machines  were  introduced  in  the  1960s, 
largely  as  a  result  of  the  success  of  programmed 
instruction.  A  variety  of  simple  machines  were  intro¬ 
duced,  including  Skinner’s  teaching  machine,  the  Por¬ 
ter  device,  the  Bell  device,  the  punchboard,  the  Subject 
Matter  Trainer  by  Briggs,  the  Arithmetic  Machine  by 
Skinner  and  Zeaman,  and  the  Polymath  by  Rothkopf 
(Ysewijn,  1993). 

The  most  notable  teaching  machine  to  implement 
programmed  instruction  was  the  first  computer  system 
completely  devoted  to  computer-assisted  instruction 
(CAI):  PLATO  (Programmed  Logic  for  Automated 
Teaching  Operations).  As  its  name  implies,  PLATO 
was  developed  using  PI  in  the  1960s  and  1970s. 
Although  extensively  modified  over  time  to  incorpo¬ 
rate  instructional  innovations  and  changing  educa¬ 
tional  philosophies  (Foshay,  1998),  PLATO  is  still 
regarded  as  one  of  the  most  successful  instructional 
software  systems  in  the  history  of  CAI  and  is  used 
widely  in  American  schools  today. 

Psychological  Principles 
of  Programmed  Instruction 

Programmed  Instruction  is  based  on  several  principles 
from  Skinner’s  operant  conditioning  theory,  including 
shaping,  priming  and  prompting ,  and  transfer  of  stim¬ 
ulus  control.  The  concept  of  shaping  is  based  on  the 
behavioral  belief  that  the  primary  factor  that  deter¬ 
mines  a  behavior  is  the  learner’s  environment  (physi¬ 
cal,  social,  etc.).  Shaping  involves  the  use  of  positive 
reinforcement  to  reward  successive  approximations  of 
the  desired  behavior,  eventually  creating  a  new 
response  that  did  not  previously  exist  in  the  learner’s 
repertoire. 
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Priming  and  prompting  are  two  terms  that  are  occa¬ 
sionally  confused.  Priming  is  meant  to  elicit  a  behavior 
that  is  not  likely  to  occur  otherwise  so  it  may  be 
reinforced.  Skinner  (1968a)  used  imitation  as  an  exam¬ 
ple  of  primed  behavior.  Like  all  behaviors  that  a 
teacher  reinforces,  to  be  sustained  it  would  have  to  be 
naturally  reinforced  in  the  environment.  Primes  must 
be  eliminated  for  the  behavior  to  be  learned.  Prompts 
are  stimulus-context  cues  that  elicit  a  behavior  so  it 
can  be  reinforced  (in  the  context  of  those  stimuli). 
Skinner  (1958a)  discussed  spelling  as  an  example, 
where  letters  in  a  word  are  omitted  from  various  loca¬ 
tions  and  the  user  is  required  to  fill  in  the  missing  letter 
or  letters.  The  letters  around  the  missing  element  serve 
as  prompts.  Prompts  are  gradually  removed,  or  faded, 
until  the  desired  response  can  be  completed  without 
any  prompts  (Skinner,  1958a). 

Transfer  of  stimulus  control  refers  to  the  practice  of 
fading  and  delaying  prompts  to  encourage  the  learner  to 
respond  to  more  naturally  occurring  stimuli;  for  exam¬ 
ple,  at  the  beginning  of  a  learning  sequence,  the  learner 
is  asked  to  make  responses  that  are  already  familiar  to 
him.  As  the  learner  proceeds  to  perform  subsequent 
activities  that  build  upon  but  are  different  from  these, 
learning  takes  place.  In  the  course  of  performing  these 
intermediate  activities,  the  student  transfers  his  original 
responses  to  new  subject  matter  content  and  also 
attaches  newly  learned  responses  to  new  subject  matter. 

Research  on  Programmed  Instruction 

Literally  thousands  of  studies  have  been  conducted  on 
the  topic  of  programmed  instruction.  As  such,  many 
compendiums  and  substantive  reviews  of  PI  research 
are  available ,  including  Galanter  (1959),  Glaser  (1965), 
Hartley  (1974),  Hughes  (1963),  Lockee  et  al.  (2004), 
Lumsdaine  and  Glaser  (1960),  Ofiesh  and  Meirhenry 
(1964),  Smith  and  Smith  (1966),  Stolurow  (1961),  and 
Taber  et  al.  (1965),  to  name  a  few  primary  references. 
The  following  section  provides  an  overview  of  the  key 
research  topics  and  studies  related  to  PI. 

Effect  of  Learner  Variables 

Some  of  the  general  issues  addressed  in  the  PI  research 
include  how  learner  variables  such  as  ability  and  atti¬ 
tudes  affect  learning  from  programmed  materials.  Sev¬ 
eral  studies  reported  that  there  was  little  or  no  correlation 
between  ability  level  and  achievement  on  programmed 
materials  (Detambel  and  Stolurow,  1956;  Ferster  and 
Sapon,  1958;  Porter,  1957).  Studies  designed  to  gauge 
learner  attitudes  toward  PI  of  learners  at  all  levels  and 
in  a  variety  of  settings  found  a  very  positive  attitude 
toward  this  instructional  approach  (Eigen,  1963; 


Engelmann,  1963;  Jones  and  Sawyer,  1949;  Smith, 
1962;  Smith  and  Smith,  1966;  Stolurow,  1963). 

Elements  of  Programmed  Instruction 

Many  studies  compared  elements  of  the  PI  model,  such 
as  the  mode  of  presentation,  the  effects  of  overt  vs. 
covert  responding,  prompting,  sequencing  of  content 
and  contingencies,  step  size  (level  of  difficulty  of  con¬ 
tent  presented  in  a  frame),  error  rate,  and  type  of 
response  options: 

•  Mode  of  presentation .  Researchers  found  no 
significant  difference  in  the  amount  of  learn¬ 
ing  between  linear  and  branching  programs 
(Anderson,  1967;  Coulson  and  Silberman, 
1960;  Holland,  1965;  Leith,  1966;  Roe, 
1962;  Silberman  et  al.,  1961),  with  the 
exception  of  one  study  by  Coulson  et  al. 
(1962),  who  found  branching  programs  to 
be  superior.  In  general,  branching  programs 
saved  significantly  more  time  in  instruction 
than  linear  programs  (Anderson,  1967; 
Coulson  and  Silberman,  1960;  Holland, 
1965;  Leith,  1966). 

•  Overt  vs.  covert  responses.  Overt  responses 
are  those  that  require  the  student  to  do  some¬ 
thing  (for  example,  writing  or  speaking  an 
answer),  whereas  covert  responses  are  those 
that  involve  thinking  about  or  reading  the 
material  and  are  therefore  not  able  to  be 
observed.  Holland  and  Porter’s  research 
(1961)  indicated  that  if  responses  were  not 
overt  (public),  they  often  ceased. 

•  Prompting.  Holland  ( 1965)  defined  a  prompt 
as  a  cue  given  prior  to  an  opportunity  to  give 
an  overt  response  that  can  be  reinforced  (e.g., 
leaving  blanks  in  a  sentence  to  be  filled  in  by 
the  learner).  A  few  studies  that  analyzed  the 
advantages  of  prompting  vs.  non-prompting 
in  a  program  sequence  found  no  significant 
difference  (Cook,  1961;  Cook  and  Spitzer, 
1960),  but  Angell  and  Lumsdaine  (1961) 
found  that  programs  should  include  both 
prompted  and  non-prompted  components. 
This  particular  component  of  PI  is  evident  in 
modem  instructional  techniques— for  exam¬ 
ple,  in  the  use  of  completion  problems  as 
guided  problem-solving  models  (van  Mer- 
rienboer  and  de  Croock,  1992;  van  Merrien- 
boer  and  Krammer,  1990).  Such  strategies 
assist  learners  in  focusing  on  key  features  of 
the  problem  scenario  and  ease  transfer  of 
learning  to  real-world  application. 
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•  Sequence.  One  of  Skinner’s  major  tenets  was 
the  “construction  of  carefully  arranged 
sequences  of  contingencies  leading  to  the 
terminal  performance”  (1953,  p.  169). 
Research  comparing  results  on  logical, 
ordered  program  sequences  vs.  nonlogical  or 
random  sequences  provides  mixed  results. 
Many  studies  indicate  that  the  effectiveness 
and  efficiency  of  ordered  sequences  is  sig¬ 
nificantly  better  than  unordered  (Gavurin 
and  Donahue,  1960;  Hickey  and  Newton, 
1964;  Miller,  1969).  Other  research  studies 
comparing  ordered  and  random  program 
sequences  do  not  support  Skinner’s  principle 
of  ordered  sequences  (Duncan,  1971;  Miller, 
1965;  Neidermeyer  et  al.,  1968;  Wager  and 
Broderick,  1974). 

•  Size  of  step.  In  studies  comparing  small  step 
size  (fewer  concepts  to  a  frame  of  instruc¬ 
tion)  to  large  step  size  (more  concepts  to  a 
frame  of  instruction),  the  majority  of  research 
found  that  smaller  step  sizes  produced  sig¬ 
nificantly  fewer  learner  errors  (Evans  et  al., 
1959;  Gropper,  1966),  especially  for  lower 
ability  students.  Results  from  several  other 
studies,  however,  warned  that  programs 
using  very  small  steps  could  introduce  a  “pall 
effect”  (Rigney  and  Fry,  1961,  p.  22)  in 
which  boredom  was  induced  by  the  material, 
particularly  with  brighter  students  (Briggs  et 
al.,  1962;  Feldhusen  et  al.,  1962). 

•  Error  rate.  A  major  tenet  of  programmed 
instruction  was  to  present  a  sequence  of 
instruction  that  had  a  "high  probability  of 
eliciting  desired  performance”  (Taber  et  al., 
1965,  p.  169);  thus,  care  was  to  be  taken  in 
designing  the  difficulty  of  that  sequence  so 
as  to  avoid  the  possibility  of  errors.  Many 
studies  support  the  concept  of  low  error  rate 
(Holland  and  Porter,  1961;  Keisler,  1959; 
Melaragno,  1960;  Meyer,  1960).  Gagne  and 
Dick  (1962),  however,  found  low  correla¬ 
tions  between  error  rate  and  learning. 

PERSONALIZED  SYSTEM 
OF  INSTRUCTION 

Historical  Origins  of  the 
Personalized  System  of  Instruction 

As  programmed  instruction  became  institutionalized 
practice,  variations  and  derivative  approaches  evolved, 
maintaining  the  general  theoretical  underpinnings  of 


behavioral  teaching  while  altering  specific  facets  of 
instruction  to  reflect  the  philosophical  and  psycholog¬ 
ical  beliefs  of  their  creators.  One  of  the  most  effective 
models  of  teaching,  the  Personalized  System  of 
Instruction  (PS I),  was  one  such  derivation.  Created  by 
Fred  Keller,  PS  I  was  often  referred  to  as  the  Keller 
Plan  (Keller  and  Sherman,  1974). 

In  his  seminal  paper  entitled,  “Good-bye, 
Teacher...,”  Keller  (1968)  provided  a  brief  history  of 
the  development  of  the  PSI  model,  based  on  a  practical 
need  to  establish  a  new  department  of  psychology  at 
the  University  of  Brasilia  and  to  create  the  first  course 
by  the  given  deadline.  His  colleagues  were  likeminded 
in  their  desire  to  try  something  new  and  were  also 
interested  in  applying  reinforcement  theory  as  a  means 
of  instructional  innovation.  PSI  courses  were  inher¬ 
ently  different  than  traditional  courses  in  that  they  were 
self-paced— there  were  no  classes  per  se.  Students 
engaged  in  instructional  units,  typically  print-based,  at 
a  time  and  place  of  their  choice.  According  to  Keller 
(Keller,  1968),  the  distinguishing  characteristics  of  the 
PSI  model  are: 

•  The  go-at-your-own-pace  feature,  which 
permits  a  student  to  move  through  the  course 
at  a  speed  commensurate  with  his  ability  and 
other  demands  upon  his  time 

•  Unit  perfection  requirement  for  advance, 
which  lets  the  student  go  ahead  with  new 
material  only  after  demonstrating  mastery  of 
that  which  proceeded 

•  The  use  of  lectures  and  demonstrations  as 
vehicles  of  motivation  rather  than  sources  of 
critical  information 

•  The  related  stress  upon  the  written  word  in 
teacher-student  communication 

•  The  use  of  proctors  which  permits  repeated 
testing,  immediate  scoring,  almost  unavoid¬ 
able  tutoring,  and  a  marked  enhancement  of 
the  personal-social  aspect  of  the  educational 
process 

Keller  himself  noted  the  similarities  between  PSI 
and  PI,  particularly  focusing  on  “analysis  of  the  task, 
the  same  concern  with  terminal  performance,  the  same 
opportunity  for  individualized  progression,  and  so  on” 
(Keller,  1977,  p.  114).  He  indicated  that  the  primary 
difference  between  the  two  instructional  approaches 
were  the  larger  “principle  steps  of  advance”  (p.  114), 
with  PSI  focusing  on  the  broader  understanding  of  a 
concept  rather  than  discrete  responses  to  individual 
prompts  or  questions.  Features  of  PSI  are  evident  in 
more  recent  models  of  customized,  computer-based 
models  of  instruction,  such  as  intelligent  tutoring 
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systems  (ITS).  Like  PSI,  ITS  determines  student  knowl¬ 
edge  of  a  particular  domain  or  subject  through  strategic 
assessment  approaches  and  provides  a  personalized 
curriculum  based  on  the  gap  between  the  targeted 
learning  outcomes  and  the  learner’s  current  knowledge 
of  the  content  (Burns  and  Capps,  1988).  However,  ITS 
functions  in  a  more  “supportive”  capacity,  demonstrat¬ 
ing  the  evolution  of  automated  instruction  from 
“strong  systems”  designed  to  replace  human  instruc¬ 
tional  tasks  to  “weak  systems”  that  serve  to  extend 
human  performance  (Spector  and  Ohrazda,  2004). 

Psychological  Principles  of  the 
Personalized  System  of  Instruction 

According  to  Sherman  (1992,  p.  59),  instruction  must 
support  a  “three-term  contingency,”  including  the  pro¬ 
vision  of  presentation,  performance,  and  conse¬ 
quences,  each  constantly  adjusted  to  meet  the  needs 
of  every  individual  student.  Keller  (1968)  advocated 
this  individualized  approach  to  instruction,  the  antith¬ 
esis  of  lecture-driven  models  that  left  students  behind 
if  they  were  absent,  confused,  or  somehow  not  able  to 
accommodate  the  pace  of  the  course  curriculum.  PSI 
was  devised  as  a  process  that  would  not  leave  any 
contingency  to  chance— chance  produces  the  normal 
curve  (Sherman,  1974).  Taveggia  (1976)  noted  that  the 
requirement  of  unit  perfection  provides  a  connection 
to  classical  learning  theory.  The  notion  of  reinforce¬ 
ment  through  evaluation  and  feedback  at  the  comple¬ 
tion  of  each  unit  is  a  straightforward  extension  of  Skin¬ 
nerian  theory. 

Research  on  the  Personalized 
System  of  Instruction 

Student  Achievement  and  Attitudes 

Like  programmed  instruction,  PSI  is  well  grounded  in 
empirical  research,  with  thousands  of  studies  support¬ 
ing  its  effectiveness  across  a  wide  range  of  disciplines. 
Numerous  meta-analyses  have  been  conducted  on  PSI, 
including  those  by  Hursh,  1976;  Johnson  and  Ruskin, 
1977;  Kulik  et  al„  1976,  1979,  1990;  Robin,  1976; 
Ryan,  1974;  and  Taveggia,  1976.  Consistent  across 
these  reviews  are  performance  gains  of  students 
engaged  in  PSI  coursework.  Kulik  et  al.  (1979,  p.  317) 
stated  that  “PSI  final  exams  average  about  8  points 
higher  than  exams  in  conventional  classes,  using 

Glass’s  index  the  effect  size  is  .5 _ A  somewhat  larger 

PSI  effect  is  found  when  achievement  exams  are 
administered  several  months  after  the  end  of  a  course, 
an  improvement  of  about  14  percentage  points.”  Dif¬ 
ferences  in  student  ratings  of  PSI  and  control  classes 


are  also  pronounced.  Students  rate  PSI  classes  as  more 
enjoyable,  more  demanding,  and  higher  in  overall 
quality  and  contribution  to  student  learning  than  con¬ 
ventional  classes  (Hursh,  1978;  Kulik  et  al.,  1979). 

Research  on  Personalized  System 
of  Instruction  Components 

Various  facets  of  the  PSI  approach  have  been  investi¬ 
gated.  Hursh  (1976)  provided  a  detailed  overview  of 
many  PSI  features,  including  the  construction  of  units 
(unit  size,  unit  format),  the  use  of  study  questions 
(presence  vs.  absence,  question  format),  quizzing 
(quizzing  routine,  question  format,  and  mastery  crite¬ 
rion),  and  student  pacing.  The  use  of  proctors,  another 
key  characteristic  of  PSI,  has  also  been  widely  inves¬ 
tigated.  Farmer  et  al.  (1972)  examined  the  effects  of 
differing  amounts  of  proctoring  and  found  no  differ¬ 
ence  in  student  performance  with  the  exception  of  the 
absence  of  proctoring,  which  generated  a  negative 
impact  on  achievement;  however,  in  their  analysis  of 
three  PSI  variables  (learner  pacing,  the  mastery 
requirement,  and  proctoring),  Femald  et  al.  (1975) 
found  that  students  learned  best  with  a  high  amount 
of  contact  with  a  proctor.  Mixed  results  possibly  reflect 
some  of  the  challenges  of  PSI  research. 

Personalized  System  of  Instruction  Research  Flaws 

Hursh  (1976,  p.  92)  pointed  to  sources  of  confounding 
in  PSI  effectiveness  research,  including  differences  in 
“instructors,  curriculum  materials,  grading  criteria, 
testing  formats,  student  selection,  and  student  expec¬ 
tations.”  Like  much  of  the  research  on  PI,  varied  inter¬ 
pretations  and  applications  of  the  features  of  the  PSI 
model  may  impact  the  quality  and  generalizability  of 
some  PSI  studies. 

PRECISION  TEACHING 

Historical  Origins  of  Precision  Teaching 

Precision  teaching  is  an  approach  to  instruction  that 
falls  under  the  general  behavioral  notion  of  B .  F.  Skin¬ 
ner,  with  particular  emphasis  on  the  free  operant,  flu¬ 
ency,  and  rate  of  responding  as  a  measure  (standard 
celeration).  PT  is  uniquely  associated  with  Ogden 
Lindsley,  a  student  of  B .  F.  Skinner’s  at  Harvard  Uni¬ 
versity  in  the  mid-1950s.  Like  Skinner,  Lindsley  was 
influenced  by  Claude  Bernard’s  (1865)  argument  that 
science  proceeded  from  inductive  (bottom  up)  reason¬ 
ing  rather  than  deductive.  Certainly  both  men  were 
influenced  by  Pavlov’s  (1927)  terminology,  and  many 
of  his  notions  of  measurement. 
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In  the  1950s,  Lindsley  began  his  career  at  the  Har¬ 
vard  Medical  School  and  Metropolitan  State  Hospital, 
where  he  founded  the  Harvard  Behavioral  Research 
Laboratory.  In  his  laboratory,  Lindsley  became  con¬ 
vinced  that  free-operant  conditioning  should  focus  on 
the  frequency  of  responding  rather  than  percentage 
correct.  Free-operant  conditioning  refers  to  an  envi¬ 
ronment  where  “students  are  free  to  respond  at  their 
own  pace  without  having  restraints  placed  on  them  by 
the  limits  of  the  materials  or  the  instructional  proce¬ 
dures”  (Lindsley,  1950,  p.  10).  His  advance  in  this 
regard  was  unusual.  Skinner  used  controlled  operants 
(discrete  trials),  rather  than  free  operants.  Lindsley 
(1966)  did  not  agree  with  Skinner’s  work  with  teaching 
machines  and  programmed  instruction  for  three  rea¬ 
sons:  (1)  there  was  no  opportunity  to  repeat  a  response 
in  the  presence  of  its  stimulus— once  a  frame  passed, 
it  was  passed;  (2)  the  response  measure  was  percentage 
of  frames  correct  (as  opposed  to  frequency  of 
response);  and  (3)  no  provision  was  made  for  external 
reinforcement  because  teaching  machine  programs 
were  designed  on  the  premise  that  getting  the  answer 
correct  was  sufficient  reinforcement.  Lindsley  argued 
that  if  that  were  true  then  all  learners  would  demon¬ 
strate  progress  (but  at  different  rates). 

Lindsley  (1990)  became  convinced  him  that  edu¬ 
cation  had  to  change  and  argued  that  teaching 
machines  and  PI  did  not  work  well  enough:  “I  knew 
that  the  real  power  of  learning  enhanced  by  free-oper¬ 
ant  conditioning  lay  in  frequency  of  responding  (by 
allowing  the  student  to  be  both  accurate  and  fluent) 
and  standard  self-recording.  When  educators  would 
not  heed  my  caution  and  could  not  see  my  vision  for 
dramatic  learning  opportunities,  I  decided  the  ethical 
thing  to  do  was  to  close  my  hospital  laboratory  and 
devote  myself  to  education”  (pp.  10-11).  In  1965, 
Lindsley  became  the  Director  of  Educational  Research 
at  the  University  of  Kansas  Medical  School,  and  in 
1971  he  became  a  Professor  of  Education  working 
with  preservice  and  in-service  special  education  teach¬ 
ers.  Precision  teaching  as  a  formal  system  emerged  in 
1964  and  1965. 

Psychological  Principles 
of  Precision  Teaching 

Precision  teaching  is  about  “basing  educational  deci¬ 
sion  on  changes  in  continuously  self-maintained  per¬ 
formance  frequencies  display  on  standard  celeration 
charts”  (Lindsley,  1992,  p.  51).  As  such,  precision 
teaching  does  not  prescribe  what  should  be  taught 
(curriculum)  nor  how  to  teach  it  (instruct);  rather,  it  is 
“a  precise  and  systematic  method”  (West  and  Young, 
1992,  p.  114)  of  evaluating  curricula  and  instruction. 


In  a  nutshell  then,  precision  teaching  was  an  individ¬ 
ualized  behavioral  approach.  It  relied  on  the  basic  prin¬ 
ciples  of  operant  conditioning  in  a  free  operant  envi¬ 
ronment.  Learning  plans  were  written  to  address 
specific  reinforcers  for  certain  targeted  behaviors,  or 
pinpoints.  Lindsley  (1968)  devised  what  he  called  the 
“dead-man  test”  to  decide  appropriate  pinpoints.  A 
good  pinpoint  involves  observable  actions  and  move¬ 
ments.  A  bad  pinpoint  lacks  either  movement  or  action, 
is  completely  internal  and  low  level,  or  involves  the 
sheer  passage  of  time.  In  general,  if  a  dead  man  can 
perform  well  on  the  pinpoint  behavior,  then  it  is  a  bad 
one.  Terms  such  as  appreciate ,  know ,  and  understand 
are  poor  pinpoints  because  they  reflect  no  actual  mea¬ 
surable  behavior. 

Recording  the  frequency  of  behaviors  is  a  hallmark 
of  precision  teaching;  according  to  Skinner,  “my  major 
contributions  are  rate  of  response  and  the  cumulating 
response  records”  (1968).  Skinner  felt  that  frequency 
was  a  “universal  datum”  that  permitted  comparison 
across  individuals  and  behaviors;  for  example,  one 
could  compare  “the  acceleration  in  dressing  produced 
by  smiling  at  a  retarded  child  with  the  slight  change 
in  the  mathematics  curriculum  of  a  gifted  teenager. 
And  we  can  say  that  the  rewarding  smile  for  the 
retarded  child  was  one  half  as  accelerating  as  the  cur¬ 
riculum  change  for  the  teenager”  (Lindsley,  1972,  p. 
2) .  Lindsley  also  pointed  out  that  the  acceleration  num¬ 
ber  could  be  used  to  determine  cost-benefit  ratios  and 
could  be  used  to  replace  test  scores. 

Research  on  Precision  Teaching 

Much  like  PI  and  PSI,  the  effectiveness  of  the  PT 
approach  is  demonstrated  through  empirical  research 
and  represents  true  data-based  decision  making  regard¬ 
ing  instructional  design.  Binder  and  Watkins  (1990) 
contended  that,  despite  the  fact  that  PT  may  be  one  of 
the  “most  thoroughly  validated  and  consistently  effec¬ 
tive  methods  yet  developed  in  English  speaking 
schools”  (p.  74),  educators  and  policymakers  either 
lack  awareness  of  the  method  or  maintain  political  or 
philosophical  objections  to  it.  Although  PT  research 
was  not  as  widely  published  as  PI  and  PSI  studies,  a 
tremendous  amount  of  student  data  was  collected  in 
the  1960s  and  1970s  that  supports  the  efficacy  of  the 
PT  method. 

Lindsley  (1991)  noted  a  particularly  impressive 
compilation  of  PT  student  data  collected  through  a 
computerized  database  system  established  in  1967  by 
the  Behavior  Bank.  Over  1 1 ,000  student  records  sub¬ 
mitted  by  teachers  were  put  into  the  database  and 
accessed  by  scientists  for  research  studies  related  to 
PT.  The  analysis  of  this  extensive  dataset  indicated 
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several  major  findings,  the  most  notable  of  which  is 
that  “behavior  frequencies  celerate,  bounce,  and 
spread  in  multiples  and  should  be  charted  on  multiply 
scales”  (Lindsley,  1991,  p.  262).  Also,  another  impor¬ 
tant  finding  is  that  many  different  stimuli  and  rewards 
were  equally  effective  in  producing  the  same  behavior; 
none  emerged  as  the  “best”  strategy  for  a  given  learn¬ 
ing  behavior. 

An  often-cited  case  study  illustrating  PT  effective¬ 
ness  is  the  Morningside  Academy  in  Seattle.  Begin¬ 
ning  in  1980,  the  private  institution  targeted  fluency 
by  combining  PT  with  Direct  Instruction  and  the  Tie- 
mann-Markle  instructional  design  strategy  (Lindsley, 
1992).  Morningside  guaranteed  a  tuition  refund  to  stu¬ 
dents  who  did  not  progress  two  grade  levels  per  year. 
As  of  1989,  no  refunds  had  been  given.  The  academy 
had  similar  success  with  an  adult  literacy  program  in 
support  of  the  Job  Training  and  Partnership  Act, 
exceeding  by  over  ten  times  the  expected  federal  gains 
(Lindsley,  1992). 


CONCLUDING  REMARKS 

Although  trends  in  educational  philosophy  and  learn¬ 
ing  theory  have  shifted  away  from  behavioral  sciences 
to  more  cognitive  and  constructivist  approaches,  the 
influence  of  programmed  technologies  continues  in  the 
instructional  design  processes  that  serve  as  the  stan¬ 
dards  for  our  field  (Dick  et  al.,  2000;  Gagne  et  al., 
1992;  Gustafson  and  Branch,  2002;  Kemp  et  al.,  1998; 
Smith  and  Ragan,  1999).  Trends  in  instructional  design 
and  technology  indicate  that,  even  though  the  system¬ 
atic  instructional  design  process  has  been  embraced  at 
varying  levels  across  different  venues  (Reiser  and 
Dempsey,  2002) ,  its  behavioral  origins  are  still  evident , 
and  notions  of  programmed  technologies  are  found  in 
existing  practice.  The  instructional  innovations 
described  herein  served  to  transform  several  functions 
of  instructional  design  into  the  cohesive  process  prac¬ 
ticed  today.  These  functions  include  the  conduct  of  a 
needs  assessment,  the  establishment  of  clearly  defined 
and  measurable  objectives,  the  process  of  task  analy¬ 
sis,  the  creation  of  assessment  instruments  and 
approaches  that  reflect  the  specified  outcomes,  the  pro¬ 
vision  of  opportunities  for  practice  and  feedback,  and 
the  evaluation  of  the  instructional  program  or  product. 
Perhaps  the  most  prominent  effect  of  the  programmed 
technologies  tradition  on  education  as  a  whole  is  the 
convergence  of  the  science  of  learning  with  the  prac¬ 
tice  of  teaching,  a  point  originating  from  the  first  dis¬ 
cussion  of  PI  from  Skinner  (1954)  himself  in  “The 
Science  of  Learning  and  the  Art  of  Teaching.” 


Programmed  technologies  are  still  evident  in  the 
contemporary  landscape  of  educational  innovations. 
Features  of  the  aforementioned  instructional  models 
are  prominent  in  today’s  advanced  learning  systems, 
especially  in  the  area  of  performance  support.  Elec¬ 
tronic  performance  support  systems,  knowledge 
management  systems,  and  expert  systems  are  exam¬ 
ples  of  current  technologies  that  provide  customized 
feedback  and  guidance  based  on  user  input— -“weak 
systems”  (Spector  and  Ohrazda,  2004,  p.  686)  spe¬ 
cifically  designed  to  enhance  human  performance. 
As  educational  researchers  seek  to  assess  the  efficacy 
and  value  of  more  modern  instructional  approaches, 
revisiting  the  tremendous  empirical  base  that  dem¬ 
onstrates  the  effectiveness  of  programmed  technol¬ 
ogies  for  learning  can  provide  insight  and  direction 
to  IDT  research  designs  and  research  plans  for  the 
21st  century. 
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ABSTRACT 

The  widespread  development  of  digital,  linked,  text- 
based  information  repositories  has  prompted  consider¬ 
able  research  on  learning  from  hypertext.  The  prolifer¬ 
ation  of  the  Internet  has  made  the  World  Wide  Web, 
with  its  hyperlinked  Web  pages,  wikis,  and  blogs,  the 
preferred  information  source  for  21st-century  world  cit¬ 
izens.  As  we  increasingly  rely  on  hypertext-based  infor¬ 
mation  resources.  It  is  critical  that  we  focus  on  how 
reading  hyperlinked  text  affects  the  learning  process  as 
we  increasingly  rely  on  hypertext-based  informational 


resources.  Grounded  in  cognitive  flexibility  theory  and 
the  construction-integration  model,  this  chapter  is 
focused  on  current  empirical  research  that  furthers  our 
understanding  of  learning  from  hypertext.  Clearly, 
one’s  goals  for  reading  a  given  hypertext  play  an  impor¬ 
tant  role  in  determining  what  is  learned  from  it.  The 
degree  to  which  these  goals  are  self  or  externally 
imposed,  the  specificity  of  goals,  and  the  interactions 
among  goals,  system  structure,  and  individual  charac¬ 
teristics  of  learners  all  influence  what  is  learned  when 
reading  hypertext.  Further,  strategies  readers  use  to  nav¬ 
igate  through  hypertext  content,  and  the  inclusion  of 
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navigational  scaffolds,  also  appear  to  influence  learn¬ 
ing.  Finally,  individual  characteristics,  such  as  prior 
knowledge  and  cognitive  style,  also  appear  to  affect 
learning  from  hypertext,  while  metacognitive  aspects 
such  as  active  knowledge  construction  and  self-regu¬ 
lated  learning,  coupled  with  coherence-promoting 
hypertext  design  elements,  appear  to  hold  great  prom¬ 
ise  for  future  research. 


KEYWORDS 

Hierarchical  hypertext:  A  hypertext  structure  in  which 
content  is  ordered  relative  to  the  concepts  pre¬ 
sented.  The  concepts  presented  on  a  screen  link  to 
superordinate  (more  general)  concepts  or  subordi¬ 
nate  (more  specific)  concepts. 

Hypertext:  Text-based  informational  screens  that  are 
presented  using  a  computer;  informational  screens 
are  connected  to  each  other  using  links. 

Linear  hypertext:  A  hypertext  structure  in  which  links 
allow  the  reader  to  move  forward  and  backward 
through  the  content,  as  if  turning  the  pages  of  a  book. 

Relational  hypertext:  A  hypertext  structure  with  links 
that  allow  the  reader  to  access  information  on  other 
screens  that  have  some  logical,  conceptual,  or  hier¬ 
archical  connection  to  the  content  on  the  current 
screen. 

Self-regulated  learning:  When  learners  set  their  own 
goals  for  learning,  then  attempt  to  plan,  monitor, 
regulate,  and  control  their  cognition,  motivation, 
behavior,  and  context. 

User  control:  The  degree  to  which  the  reader  can  deter¬ 
mine  the  sequence  and  pacing  of  information 
accessed  in  a  hypertext. 


INTRODUCTION 

The  development  and  widespread  implementation  of 
hypertext-based  information  presentation  systems  have 
prompted  researchers  to  investigate  how  the  use  of 
such  systems  affects  knowledge  acquisition.  Although 
the  majority  of  research  on  hypertext  is  not  specifically 
related  to  learning,  a  growing  body  of  empirical 
research  addresses  learning  with  hypertext.  This  chap¬ 
ter  provides  a  conceptual  review  of  empirical  research 
reported  during  the  past  5  years  and  updates  a  previous 
Handbook  of  Research  on  Educational  Communica¬ 
tions  and  Technology  chapter  (Shapiro  and  Nieder¬ 
hauser,  2004). 

Established  theories  that  help  explain  reading  and 
learning,  such  as  the  construction-integration  model 
(Kintsch,  1988)  and  cognitive  flexibility  theory  (Spiro 


et  al.,  1988,  1992),  provide  the  theoretical  underpin¬ 
nings  of  the  learning-from- hypertext  process.  Accord¬ 
ing  to  Shapiro  and  Niederhauser  (2004),  the  construc¬ 
tion-integration  model  (CIM)  of  text  processing 
suggests  a  three-stage  process  of  text  comprehension. 
The  first  two  processes— character/word  decoding  and 
the  construction  of  a  textbase  (a  hierarchical  proposi¬ 
tional  representation  of  the  textual  information)  — are 
thought  to  be  invariant  across  media.  The  third  process, 
creation  of  a  situation  model,  is  central  to  our  under¬ 
standing  of  learning  with  hypertext. 

Creating  a  situation  model  is  the  process  whereby 
the  learner  constructs  a  conceptual  representation  of 
the  content  in  the  text  by  integrating  new  information 
from  the  text  (textbase)  with  his  or  her  prior  knowl¬ 
edge.  At  its  best,  this  active  integration  of  the  textbase 
with  prior  knowledge  constitutes  meaning  making— a 
deep,  complex,  and  coherent  understanding  of  the 
material  presented  in  the  text.  The  active  cognition 
inherent  in  this  integrative  process  drives  educators’ 
and  researchers’  belief  that  hypertext  can  serve  as  a 
tool  to  promote  high-level  conceptual  learning. 

Cognitive  flexibility  theory  (CFT)  also  contributes 
to  our  understanding  of  learning  with  hypertext.  CFT 
suggests  that  a  hypertext-based  environment  allows 
the  reader  to  utilize  the  linking  capabilities  inherent 
in  hypertext  to  construct  his  or  her  own  unique  path¬ 
way  through  the  text  (Jacobson  and  Spiro,  1995;  Law¬ 
less  and  Kulikowich,  1996).  Because  the  learner  can 
access  a  given  information  screen  from  multiple  other 
hyperlinked  screens,  he  or  she  will  encounter  the 
same  information  from  multiple  perspectives  depend¬ 
ing  on  the  paths  taken.  A  rich  understanding  of  com¬ 
plex  conceptual  landscapes  will  emerge  when  the 
learner  has  made  numerous  traversals  of  the  domain 
from  different  perspectives  (Jacobson  and  Spiro, 
1995).  Thus,  the  linked  structure  of  hypertext  allows 
the  reader  to  access  information  from  different 
paths  — providing  opportunities  for  learners  to  inte¬ 
grate  new  information  from  the  text  into  their  existing 
conceptions  (create  a  situation  model)  and  make  con¬ 
nections  among  concepts  that  are  not  likely  to  occur 
when  sequentially  reading  traditional  text.  This  pro¬ 
cess  of  criss-crossing  the  conceptual  landscape 
(Jacobson  and  Spiro,  1995;  Spiro  and  Jehng,  1990; 
Spiro  et  al.,  1987)  requires  the  reader  to  engage  in 
deeper  levels  of  semantic  processing  to  compare  and 
contrast  the  linked  information  by  analyzing  the  con¬ 
tent  in  terms  of  meaningful  properties. 

The  increased  learner  control  associated  with  criss¬ 
crossing  places  additional  demands  on  the  reader’s 
cognitive  resources.  Chief  among  these  are  the  meta¬ 
cognitive  demands  associated  with  determining  the 
structure  of  the  textbase  in  the  hypertext,  integrating 
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the  textbase  with  prior  knowledge,  and  navigating 
through  the  hypertext  in  a  way  that  promotes  the  devel¬ 
opment  of  a  complex  and  conceptually  coherent  rep¬ 
resentation  of  the  content.  To  navigate  effectively,  the 
learner  must  monitor  whether  he  or  she  understands 
what  has  been  read,  determine  whether  additional 
information  is  needed,  and  decide  which  of  the  avail¬ 
able  links  might  be  useful  for  locating  the  needed 
information  in  the  hypertext. 

As  noted  in  by  Shapiro  and  Niederhauser  (2004, 
pp.  606-607): 

CIM  is  informative  to  hypertext  research  because  it  offers 
an  explanation  of  the  relevance  of  user  behavior.  Specif¬ 
ically,  it  explains  the  research  that  points  to  user  behaviors 
such  as  link  choice,  navigation  patterns,  and  metacogni- 
tive  practice  as  mediators  of  learning.  CFT  offers  an 
explanation  of  meaningful  learning  on  the  part  of 
advanced  learners.  It  successfully  explains  why  the  explo¬ 
ration  of  identical  texts  can  result  in  more  flexible,  trans¬ 
ferable  knowledge  from  a  hypertext  than  a  traditional  text. 

It  adds  to  our  understanding  of  [hypertext-assisted  learn¬ 
ing]  because  it  offers  a  unique  explanation  of  how  mental 
representations  are  constructed,  reconstructed,  and  altered 
by  exposure  to  dynamic  information  structures. 

It  is  these  dynamic  information  structures  that  make 
hypertext-based  systems  different  from  traditional  print- 
based  media.  Hypertext  supports  nonlinear  access  to 
information  through  the  use  of  navigational  elements 
such  as  menus,  concept  maps,  and  hotwords  embedded 
in  the  text.  These  features  structure  the  content  being 
presented  and  force  the  reader  to  choose  his  or  her  own 
path  through  the  text.  Requiring  users  to  take  this 
responsibility  gives  rise  to  issues  concerning  users’ 
goals,  motivations,  and  purposes  for  reading:  their  prior 
knowledge  about  the  content;  and  the  metacognitive 
strategies  they  use  to  monitor  their  understanding  of  the 
content  and  make  decisions  about  the  order  of  informa¬ 
tion  access.  Thus,  essential  hypertext-specific  variables 
include  the  effects  of:  (1)  reader  goals,  (2)  navigation 
patterns,  (3)  learner  characteristics,  and  (4)  active  cog¬ 
nitive  engagement  when  learning  with  hypertext. 


READER  GOALS 

Goal-directed  learning  appears  to  have  a  powerful 
influence  when  learning  with  hypertext  (Jonassen  and 
Wang,  1993).  When  reading  hypertext,  a  reader’s 
learning  goal  may  be  influenced  by  several  factors, 
including  implicit  and  explicit  teacher/researcher  goals 
(e.g.,  to  answer  specific  questions,  for  general  mean¬ 
ing,  to  find  certain  information),  features  and  informa¬ 
tion  inherent  in  the  hypertext  itself  (e.g.,  goal  state¬ 


ments,  embedded  questions),  and  learner  goals  (e.g., 
interest,  need  to  know).  Gerjets  and  Scheiter  (2003) 
reported  a  series  of  studies  that  demonstrated  the 
effects  on  cognitive  load  of  the  interrelationships 
among  teacher  goals,  learner  goals,  and  learners’  pro¬ 
cessing  strategies  — and  the  resulting  influences  on 
learning.  They  concluded  that  the  relationship  between 
instructional  design  (system  structure)  and  pattern  of 
extraneous  and  germane  cognitive  load  is  not  as  deter¬ 
ministic  as  many  cognitive  load  researchers  claim. 
Rather,  the  relation  is  moderated  by  instructional 
(teacher-imposed)  goals,  learner  goals,  and  the  learn¬ 
ers’  processing  strategies.  Differentiating  goals  and 
processing  strategies  appear  to  be  important  aspects  of 
the  development  of  our  understanding  of  the  processes 
associated  with  learning  with  hypertext. 

Specificity  of  Goals 

Much  of  the  hypertext-assisted  learning  research  to 
date,  however,  has  focused  on  the  use  of  explicit 
teacher/researcher  goals.  Curry  et  al.  (1999)  conducted 
a  study  to  examine  the  effect  of  providing  a  specific 
learning  objective  to  guide  the  reading  of  a  hypertext. 
Fifty  university  students  read  a  60-frame  hierarchical 
hypertext  on  Lyme  disease.  Half  of  the  students 
received  a  specific  task  to  guide  their  learning.  They 
were  given  a  scenario  about  a  man  with  physical  symp¬ 
toms  and  a  probable  diagnosis  and  were  told  to  use 
the  hypertext  to  determine  the  accuracy  of  the  infor¬ 
mation  in  the  scenario.  The  other  half  were  told  to  read 
the  text  carefully,  because  they  would  be  asked  a  series 
of  questions  at  the  end.  Although  no  differences  were 
found  on  recall  measures,  the  concept  maps  that  stu¬ 
dents  drew  did  show  differences.  Students  in  the  spe¬ 
cific  goal  condition  constructed  more  relational  maps, 
which  the  authors  felt  demonstrated  a  more  sophisti¬ 
cated  conceptual  representation  of  the  content. 

More  recent  research  also  examined  the  effects  of 
specificity  of  goal-setting  orientation  on  learning  with 
hypertext  (Gall,  2006;  Moos  and  Azevedo,  2006;  Zum- 
bach  and  Reimann,  2002).  Results  indicated  that  goal 
orientation  in  these  studies  also  influenced  the  type  of 
learning  that  occurred.  Gall  (2006)  examined  the  dif¬ 
ferential  learning  of  Air  Force  Academy  cadets  who 
used  a  200-article  title-linked  relational  hypertext 
under  three  orienting  conditions:  browsing  to  learn  the 
content,  searching  for  factual  information  to  answer 
specific  questions,  and  looking  for  connections  among 
articles  to  answer  conceptual  questions.  Subjects  who 
used  the  hypertext  to  find  connections  among  concepts 
performed  best  on  a  measure  of  conceptual  under¬ 
standing.  Cadets  without  a  focused  goal  orientation 
(those  in  the  browsing  condition)  learned  the  least. 
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Not  all  specific  learning  goals  promote  deep,  mean¬ 
ingful  learning,  however.  Azevedo  and  his  colleagues 
(Azevedo  et  al.,  2001 , 2002)  gave  one  group  of  subjects 
the  explicit  goal  of  answering  specific  questions  about 
the  human  circulatory  system,  while  other  subjects 
were  able  to  generate  their  own  goals  while  reading  a 
Microsoft  Encarta-based  relational  hypermedia  system. 
Some  subjects  in  the  question-answering  groups 
increased  the  sophistication  of  their  mental  models  of 
circulation,  but  many  actually  showed  a  decrease  in 
sophistication.  No  subjects  in  the  learner-generated 
goal  condition  showed  a  decrease  in  the  quality  of  their 
mental  models,  and  nearly  all  showed  an  increase. 
Moreover,  those  in  the  self-generated  goal  condition 
demonstrated  more  effective  use  of  metacognitive  strat¬ 
egies. 

Results  form  other  studies  that  examined  reader- 
regulated  goal  structures,  however,  contradicted  these 
results.  Zumbach  and  Reimann  (2002)  presented  stu¬ 
dents  with  either  a  seven-module,  computer-based  linear 
hypertext  (an  externally  imposed  explicit  goal)  or  a  rela¬ 
tional  hypertext  that  contained  all  of  the  content  in  the 
tutorial  presented  in  a  linked  format.  Subjects  who 
received  the  hypertext  treatment  were  given  either  strat¬ 
egy  instruction  in  analyzing,  comparing,  and  evaluating 
the  causal  relationships  of  facts  and  information  (to 
prompt  internally  imposed  goals)  or  a  goal-based  sce¬ 
nario  that  required  them  to  role-play  a  newspaper  editor 
who  had  to  use  the  information  in  the  hypertext  to  write 
a  commentary  (externally  imposed  general  goal).  Sixty 
adult  participants  were  randomly  assigned  to  one  of  the 
3  treatment  conditions  (20  per  treatment)  and  assessed 
on  factual  knowledge  (multiple-choice  test),  structural 
knowledge  (computerized  concept-mapping  tool),  and 
application/inference/transfer  of  knowledge  (“argumen¬ 
tation”  task).  Findings  indicated  that  subjects  in  the  tuto¬ 
rial  condition  performed  best  on  the  factual  information 
task;  subjects  in  the  goal-based  scenario  condition  per¬ 
formed  best  on  the  structural  knowledge  and  applica¬ 
tion/inference/transfer  task;  and,  contrary  to  results  from 
Azevedo  and  colleagues’  (2001, 2002)  studies,  subjects 
in  the  strategy  group,  despite  explicit  instructions,  failed 
to  apply  the  metacognitive  strategy,  yielding  disappoint¬ 
ing  results  for  this  group. 

Relationships  among  Goals,  System 
Structure,  and  Learner  Characteristics 

Subjects  in  Curry  and  colleagues’  (1999)  and  Gall’s 
(2006)  studies  may  have  benefited  from  a  specific  goal 
because  it  capitalized  on  the  features  offered  by  hyper¬ 
text.  The  specific  fact-finding  goal  assigned  in  the  stud¬ 
ies  by  Azevedo  et  al.  (2001,  2002)  and  Gall  (2006) 
may  be  less  compatible  with  hypertext-assisted  learn¬ 


ing.  Early  in  the  history  of  hypertext  in  educational 
settings,  Landow  (1992)  considered  the  importance  of 
matching  learning  goals  to  the  uniqueness  of  the  tech¬ 
nology.  He  pointed  out  that  hypertext  and  printed  text 
have  different  advantages  and  that  hypertext  assign¬ 
ments  should  be  written  that  complement  hypertext, 
suggesting  that  goals  such  as  fact  retrieval  squander 
the  richness  of  hypertext  because  fact  finding  is  not 
aided  by  multiple  links. 

Shapiro  (1998b)  examined  the  affects  of  matching 
system  structure,  goals,  and  prior  knowledge  when  she 
matched  specific  learning  goals  with  the  use  of  a  clear, 
conceptually  based  hypertext  system.  She  found  that 
the  ability  of  low  prior  knowledge  (PK)  learners  to 
meet  their  goals  was  mediated  by  a  given  structure’s 
compatibility  with  the  learning  goal.  Learning  was 
enhanced  for  participants  with  low  PK  of  the  content 
domain  who  were  specifically  instructed  to  learn  about 
that  content  and  were  provided  with  an  advance  orga¬ 
nizer  that  represented  the  conceptual  structure  of  the 
content  presented  in  the  hypertext.  This  result  speaks 
to  the  potential  of  a  well-defined,  goal-appropriate 
structure  for  initial  learning  by  novices. 


NAVIGATION  PATTERNS 

Several  researchers  have  attempted  to  identify  patterns 
of  reader  navigation  as  they  read  hypertext.  In  an  early 
study  of  navigation  patterns,  researchers  observed  sub¬ 
jects  reading  hypertext  and  identified  six  distinct  strat¬ 
egies:  skimming,  checking,  reading,  responding, 
studying,  and  reviewing  (Horney  and  Anderson- 
Inman,  1994).  In  another  effort,  Castelli  et  al.  (1998) 
examined  the  relationships  among  a  battery  of  psycho¬ 
logical  factors  and  a  series  of  navigation  indices.  Based 
on  their  examinations,  the  authors  identified  seven  cat¬ 
egories  of  hypertext  users  and  related  the  kinds  of 
cognitive  characteristics  associated  with  the  various 
patterns.  Such  studies,  however,  simply  addressed 
what  readers  did,  not  the  relationship  between  reading 
patterns  and  learning. 

Other  investigations  have  examined  how  individual 
navigation  patterns  relate  to  learning.  Lawless  and  her 
colleagues  (Lawless  et  al.,  2002)  followed  up  on  their 
previous  research,  which  identified  three  distinct  nav¬ 
igation  patterns  in  hypertext  users— knowledge  seek¬ 
ers,  feature  explorers,  and  apathetic  users— by  exam¬ 
ining  whether  these  navigation  patterns  were  age 
dependent  (Lawless  and  Brown,  1997;  Lawless  and 
Kulikowich,  1996).  They  used  hypertext  with  60 
fourth,  fifth,  and  sixth  graders  and  found  similar  nav¬ 
igation  patterns  to  those  identified  in  their  previous 
studies  with  adults,  but  that  the  largest  group  of 
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younger  readers  in  the  study  were  knowledge  seekers, 
while  the  largest  group  of  adult  subjects  were  feature 
explorers  (Lawless  and  Kulikowich,  1998).  They 
hypothesized  that  the  primary  reason  for  this  discrep¬ 
ancy  was  related  to  differing  level  of  domain  knowl¬ 
edge— the  adult  readers  had  lower  prior  knowledge 
and  probably  less  interest  in  the  content  presented  than 
did  the  younger  readers. 

Navigational  Scaffolding 

Recent  research  has  examined  how  incorporating  sup¬ 
port  mechanisms  that  help  users  grasp  the  overall 
scope  and  structure  of  the  content  affects  hypertext- 
assisted  learning.  Various  factors  have  been  examined, 
including  the  use  of  hierarchically  structured  overview 
navigation  menus  (Brinkerhoff  et  al.,  2001);  preview 
pop-up  windows  that  gave  a  synopsis  of  linked  infor¬ 
mation  (Cress  and  Knabel,  2003);  expanding  hyper¬ 
text,  which  allowed  the  reader  to  have  more  detailed 
information  without  losing  the  linear  flow  of  informa¬ 
tion  (Lee  and  Tedder,  2004);  embedded  annotations 
that  provided  additional  information  about  the  topic 
on  the  screen  (Lee  and  Calandra,  2004);  navigational 
prompts  to  direct  reader  through  the  content  (Puntam- 
bekar  and  Stylianou,  2005);  navigable  concept  maps 
(Puntambekar  et  al.,  2003);  and  graphical  overviews 
(de  Jong  and  van  der  Hulst,  2002).  Results  from  these 
studies  typically  provided  support  for  the  idea  that 
scaffolding  a  user’s  understanding  of  scope  and  struc¬ 
ture  helped  organize  the  content  and  orient  the  reader. 
As  we  will  see  later  in  this  chapter,  however,  organi¬ 
zation  of  the  content  interacts  with  other  learning  vari¬ 
ables  such  as  the  reader’s  navigation  strategies,  goals, 
and  prior  knowledge;  the  complexity  of  the  content; 
and  the  coherence  of  the  information  being  linked. 

Active  vs.  Passive  Navigation 

Additional  research  has  examined  how  the  sequence  in 
which  the  information  is  read  affects  learning  of  the 
content  (Eveland  et  al.,  2004;  Niederhauser  et  al.,  2000; 
Salmeron  et  al.,  2005)— that  is,  the  effects  of  reading 
in  a  sequential  hierarchical  fashion  vs.  more  idiosyn¬ 
cratic  jumping  around  in  the  hypertext.  Su  and  Klein 
(2006)  used  a  hypertext  on  the  Internet  and  World  Wide 
Web  to  examine  learning  of  354  undergraduate  college 
students.  Students  were  presented  with  one  of  three 
navigational  forms  of  the  hypertext  program:  (1)  hier¬ 
archically  structured  content  list,  (2)  concept  map,  and 
(3)  embedded  hyperlink.  They  determined  that  subjects 
who  used  the  content  list  hypertext  performed  better 
on  a  test  of  factual  knowledge  than  did  subjects  who 
used  the  embedded  hyperlink  hypertext. 


Eveland  and  colleagues  (2004)  also  demonstrated 
that  users  who  read  a  more  structured  hypertext  learned 
factual  information  but  also  showed  that  readers  who 
read  a  nonlinear  hypertext  gained  a  better  conceptual 
understanding  of  the  textbase.  They  presented  172  col¬ 
lege  student  and  non-student  adult  subjects  with  a 
complex  hypertext  containing  health  information.  One 
form  of  the  hypertext  allowed  only  hierarchical  navi¬ 
gation;  the  other  provided  numerous  relational  hyper¬ 
links  among  the  information  nodes.  Results  indicated 
that  subjects  in  the  hierarchical  condition  performed 
better  on  a  factual  knowledge  measure  than  those  in 
the  relational  condition,  while  subjects  in  the  relational 
condition  outperformed  hierarchical  condition  subjects 
in  terms  of  knowledge  structure  density  (as  measured 
with  concept  mapping). 

Niederhauser  and  colleagues  (2000)  provided  their 
subjects  with  a  hypertext  that  allowed  both  hierarchical 
and  relational  navigation.  Subjects  were  provided  with 
navigational  tools  that  supported  both  navigation 
options  (a  hierarchical  map  navigation  tool  and  hot- 
word  links  to  conceptually  related  content).  Subjects 
were  allowed  to  read  the  hypertext,  which  described 
behaviorist  and  constructivist  learning  theories,  using 
navigation  options  of  their  choice.  Reading  the  83- 
screen  hypertext  was  part  of  a  regular  class  assignment 
for  39  university  students  over  a  2-week  period,  and 
students  were  tested  on  their  learning  of  the  content 
as  a  class  assignment.  Subjects’  navigation  pattern 
preferences  were  determined  by  examining  server  log 
files  of  the  sequence  of  screen  access. 

Examination  of  navigation  patterns  showed  that 
some  students  navigated  in  a  passive  manner.  These 
students  adopted  a  linear  approach  by  systematically 
moving  through  each  frame  for  the  content  on  one  the¬ 
ory,  then  moving  through  the  content  on  the  second 
theory  in  the  same  manner.  Other  students  adopted  a 
more  active  strategy  as  they  read  a  screen  on  one  theory, 
then  used  a  link  to  compare  that  information  with  the 
other  theory,  proceeding  through  the  text  using  this  com¬ 
pare  and  contrast  strategy.  Results  indicated  that  stu¬ 
dents  who  read  the  text  in  a  more  sequential  fashion  had 
higher  scores  on  a  combined  measure  (multiple-choice 
test  of  factual  content  and  essay  that  required  students 
to  compare  and  contrast  the  major  themes  in  the  hyper¬ 
text).  Increased  cognitive  load  was  hypothesized  as 
being  the  reason  why  students  who  used  the  linking 
features  did  not  perform  as  well  on  the  post-tests. 

Zumbach  et  al.  (2001)  also  examined  the  influence 
of  active  and  passive  navigation  strategies  in  a  yoke- 
controlled  experimental  design  in  which  one  group  of 
1 1  subjects  was  allowed  to  navigate  freely  through  a 
300-page  relational  hypertext  about  marine  pollution 
accidents,  and  a  yoked  group  of  11  students  moved 
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sequentially  through  the  same  screens  as  their  coun¬ 
terparts.  Results  showed  no  significant  difference  on 
knowledge  measures  between  the  active  and  passive 
hypertext  users  and  what  appeared  to  be  slightly  better 
performance  on  the  measure  of  conceptual  understand¬ 
ing  by  the  passive  users.  The  authors  concluded  that 
‘“active  access’  to  [hyperj-text-based  information  does 
not  always  lead  to  better  cognitive  and  metacognitive 
outcomes  than  a  pre-sequenced  presentation  (‘passive 
access’)”  (Zumbach  et  al.,  2001,  p.  316). 

Finally,  using  a  24-node  hierarchical  hypertext  on 
atmosphere  pollution  with  41  undergraduate  students, 
Salmeron  and  colleagues  (2005)  demonstrated  that, 
given  a  similar  number  of  nodes  read,  subjects  who 
read  the  contents  in  a  high  coherent  order  formed  a 
better  situation  model  of  the  text;  that  is,  subjects 
tended  to  develop  a  better  conceptual  understanding 
of  the  content  when  they  followed  links  that  accessed 
information  that  had  shared  arguments  and  maintained 
the  coherence  of  the  text  representation. 

The  Role  of  Coherence 

Navigation  patterns  that  promote  coherence  appear  to 
be  an  important  aspect  of  learning  with  hypertext. 
McNamara  and  Shapiro  (2005)  addressed  the  impor¬ 
tance  of  coherence  when  learning  with  hypertext  by 
offering  a  series  of  guidelines  for  hypertext  developers . 
Using  the  following  guidelines  may  help  learners  rec¬ 
ognize  the  relationships  among  ideas  scattered  across 
a  hypertext:  (1)  provide  ill-defined  global  structure  for 
advanced  learners,  (2)  provide  well-defined,  goal- 
appropriate  global  structure  for  domain  novices,  (3) 
create  an  environment  that  encourages  a  metacognitive 
approach  to  the  content,  and  (4)  highlight  links  that 
denote  very  important  inter-document  relationships. 

In  the  study  by  Salmeron  and  colleagues  (2005) 
described  earlier,  researchers  also  addressed  the  rela¬ 
tionship  between  coherence  of  reading  order  and  level 
of  prior  knowledge.  They  used  their  atmosphere  pol¬ 
lution  hypertext  with  82  university  students,  this  time 
including  two  different  overviews  of  the  content  as 
navigation  aids.  The  first  overview  contained  a  6  x  4 
matrix  of  nodes  with  high  conceptual  coherence  read¬ 
ing  right  to  left  and  top  to  bottom.  The  second  had  a 
similar  array,  except  the  nodes  were  arranged  with  low 
conceptual  coherence.  Subjects  were  required  to  read 
all  screens  but  were  not  allowed  to  reread  any  screen. 
Findings  revealed  that  participants  with  low  domain 
knowledge  tended  to  develop  better  situation  models 
when  reading  the  content  in  a  high  coherence  order, 
while  participants  with  high  domain  knowledge 
learned  more  from  a  low  coherence  order.  Reading 
order  did  not,  however,  affect  textbase  construction. 


In  sum,  the  need  to  navigate  through  a  hypertext 
is  a  defining  feature  that  differentiates  reading  and 
learning  in  a  hypertext  environment  from  reading 
and  learning  with  traditional  printed  text.  Initial  nav¬ 
igation  strategies  may  be  adopted  due  to  interest, 
motivation,  and  intrinsic  or  extrinsic  goals  of  the 
reader.  Several  authors  (Niederhauser  et.  al.,  2000; 
Shapiro,  1999;  Tergan,  1997;  Yang,  1997;  Zumbach 
et  al.,  2001)  have  discussed  issues  of  cognitive  load 
when  examining  navigation  patterns  in  hypertext- 
assisted  learning.  (For  more  about  instructional 
issues  associated  with  cognitive  load,  see  Paas  and 
van  Merrienboer,  1994;  Sweller,  1988;  Sweller  et  al., 
1998.)  When  users’  navigation  patterns  do  not  sup¬ 
port  coherence,  cognitive  load  associated  with  nav¬ 
igating  through  the  text  may  interfere  with  a  reader’s 
ability  to  make  sense  of  the  content.  The  reader  may 
then  adopt  compensatory  strategies  to  simplify  the 
learning  task;  thus,  coherence  may  influence  what 
the  reader  learns  from  the  text  and  may  be  influenced 
by  the  navigational  opportunities  afforded  by  the 
hypertext. 


LEARNER  CHARACTERISTICS 

Individual  characteristics,  such  as  prior  knowledge 
(Hofman  and  van  Oostendorp,  1999;  Potelle  and 
Rouet,  2002;  Shapiro,  1998a,b,  1999;  Shin  et  al., 
1994),  metacognition  (Azevedo,  2005;  Graesser  et 
al.,  2005),  and  cognitive  style  (Dunser  and  Jirasko, 
2005;  Graff,  2003;  2005;  Lin  and  Davidson-Shivers, 
1996),  play  an  important  role  in  learning  with  hyper¬ 
text. 

Prior  Knowledge 

Research  on  organizing  tools  and  system  structure 
indicates  that  low  PK  learners  benefit  from  well- 
defined  structures  (such  as  hierarchies)  when  the  learn¬ 
ing  goal  is  to  achieve  simple,  factual  knowledge  (text- 
base).  Nonhierarchical  hypertext  structures  (e.g., 
alphabetically  structured  list  or  an  unprincipled  net¬ 
work  of  links)  appear  to  confuse  low  PK  learners  and 
inhibit  learning,  even  for  factual  information  presented 
on  individual  screens  (Potelle  and  Rouet,  2002).  Cau¬ 
tion  is  warranted  in  overgeneralizing  these  findings, 
especially  as  they  relate  to  the  use  of  use  of  complex 
conceptual  maps.  Evidence  suggests  that  providing 
complex  concept  maps  for  low  PK  learners  may  over¬ 
whelm  readers  and  hinder  understanding  if  use  of  the 
map  draws  attention  away  from  the  content  of  the  text 
and  focuses  learners  on  the  macro  structure  (Hofman 
and  van  Oostendorp,  1999). 
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Simply  providing  hierarchical  structure  is  probably 
not  sufficient  to  ensure  learning  for  low  PK  learners. 
The  use  of  explicit  pointers  to  conceptual  relationships 
(an  arrangement  of  thematic  clusters)  was  related  to 
an  increase  in  problem-solving  ability.  Any  device  or 
structure  that  explicates  the  conceptual  relationships 
between  topics  and  increases  text  coherence  can  ben¬ 
efit  learning  for  low-knowledge  learners  (McNamara 
and  Shapiro,  2005;  Shapiro,  1999). 

Well-structured  hypertexts  may  offer  low  PK 
learners  an  introduction  to  the  ways  in  which  topics 
relate  to  one  another  and  an  easy-to-follow  introduc¬ 
tion  to  a  domain,  especially  when  the  structure  is  com¬ 
patible  with  the  learning  goal.  Well-defined  structures 
may  also  help  low-knowledge  learners  to  stay  oriented 
while  exploring  the  information;  however,  as  Spiro  and 
colleagues  (1987)  noted,  it  is  dangerous  to  oversim¬ 
plify  a  topic  for  learners.  Providing  rigid  structures, 
especially  for  ill- structured  domains  (such  as  history 
or  psychology),  can  impose  arbitrary  delineations  that 
may  impede  progress  as  a  learner  advances  in  knowl¬ 
edge.  For  this  reason,  ill- structured  hypertexts  may 
also  offer  advantages. 

Demonstrating  consistent  benefits  for  ill-structured 
hypertext,  however,  has  been  challenging  for  several 
reasons.  Reasonably  sensitive,  valid,  and  reliable  mea¬ 
sures  for  the  types  of  gains  that  are  likely  promoted 
through  the  use  of  ill- structured  hypertext  (deep  and 
connected  conceptual  learning)  have  only  recently  been 
developed  and  employed.  Further,  although  many 
hypothesize  that  high  PK  users  may  be  able  to  take 
advantage  of  ill-structured  hypertexts  and  develop  a 
high-level,  interconnected  conceptual  understanding  of 
the  textbase,  interactions  among  variables  make  it  dif¬ 
ficult  to  sort  out  the  unique  effects  of  system  structure 
from  numerous  intervening,  and  potentially  confound¬ 
ing,  variables. 

Providing  less  obvious  organizational  structures 
challenges  learners  to  seek  their  own  coherence  within 
the  system.  The  overall  effect  is  to  promote  active  cog¬ 
nitive  engagement  and  improve  learning;  however,  ill- 
structured  systems  may  not  always  be  effective  for 
advanced  learners.  As  pointed  out  in  earlier  sections, 
reader  goals  and  motivation  can  affect  whether  learners 
actively  attempt  to  integrate  the  textbase  into  their  prior 
knowledge.  A  passive  learner  with  unclear  goals  will 
likely  learn  little  from  any  hypertext  system,  whereas 
the  active  cognitive  engagement  inherent  in  criss-cross¬ 
ing  the  conceptual  landscape  and  creating  a  situation 
model  may  promote  deep  conceptual  understanding  of 
the  content. 

In  keeping  with  prior  research  on  traditional  text- 
based  learning,  promoting  active  engagement  with  the 
content  appears  to  be  essential  for  developing  a  situa¬ 


tion  model.  Providing  a  structure  that  is  highly  orga¬ 
nized  or  simple  to  follow  may  cause  readers  to  become 
passive.  The  task  for  hypertext  designers  is  to  create 
hypertext  that  sufficiently  challenges  beginning  learn¬ 
ers  while  not  overburdening  them  to  the  point  where 
learning  is  compromised. 

Cognitive  Style 

Early  hypertext  research  literature  on  cognitive  style 
failed  to  demonstrate  much  in  the  way  of  predictive 
or  explanatory  power  (Dillon  and  Gabbard,  1998). 
These  studies  typically  examined  hypertext-based 
learning  relative  to  concepts  such  as  field  dependence 
and  independence  (Fin  and  Davidson-Shivers,  1996). 
Recent  work,  however,  has  utilized  style  dimensions 
that  show  greater  potential  for  predicting  behavior 
and  performance  (Dunser  and  Jirasko,  2005;  Graff, 
2003, 2005). 

Graff  (2003)  examined  learning  from  three  64- 
node  hypertexts  — linear,  hierarchical,  and  rela¬ 
tional-relative  to  two  measures  of  the  users’  learning 
styles.  Ninety-six  undergraduate  university  students 
were  grouped  according  to  scores  on  a  wholist-ana- 
lytic  questionnaire  (wholist/intermediate/analytic), 
which  identified  an  individual’s  tendency  to  process 
information  as  an  integrated  whole  or  as  discrete  parts. 
Results  indicated  that  the  intermediate  group  in  the 
relational  condition  tended  to  outperform  all  other 
group/condition  configurations  on  measures  of  detail 
and  number  of  words  on  a  post-test  essay.  Using  the 
same  dataset,  Graff  then  grouped  students  based  on 
scores  from  a  verbal-imager  questionnaire  (verbal/ 
bimodal/imager),  which  identified  an  individual’s  ten¬ 
dency  to  process  information  either  in  words  (working 
with  verbal  information)  or  in  images  (visual  and  spa¬ 
tial  information).  Subjects  in  the  bimodal  group  in  the 
hierarchical  condition  outperformed  all  other  group/ 
condition  configurations  on  measures  of  detail  and 
number  of  words  on  a  post-test  essay  (although  the 
verbal-relational  group  approached  significance  on 
both  measures,  as  well). 

Graff  cautiously  proposed  support  for  the  relation¬ 
ship  between  his  two  measures  of  cognitive  style,  the 
hypertext  architectures  he  employed,  and  learning— 
indicating  that  subjects  in  the  intermediate  group  pos¬ 
sess  both  wholist  and  analytic  characteristics  and  may 
have  been  more  flexible  in  their  processing  of  the 
relational  hypertext.  Further,  he  drew  on  Paivio’s 
(1971)  dual-coding  model  to  suggest  that  bimodals 
were  able  to  process  information  in  verbal  and  imag- 
inal  subsystems  simultaneously  to  incorporate  the  text- 
based  media  while  maintaining  their  orientation  in  the 
hierarchically  organized  structure. 
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In  a  second  study  using  the  same  materials  and 
55  undergraduate  subjects  (Graff,  2005),  cognitive 
style  was  assessed  using  an  intuitive-analytic  cogni¬ 
tive  style  measure  that  was  similar  to  the  wholist- 
analytic  measure  described  above.  Dependent  mea¬ 
sures  included  scores  on  recall  and  concept  map  com¬ 
plexity  and  concept  map  density.  For  recall,  a  clear 
relationship  between  cognitive  style  and  hypertext 
structure  was  found.  Analysts  scored  highest  in  the 
hierarchical  condition,  intermediates  in  the  relational 
condition,  and  intuitives  in  the  linear  condition.  No 
significant  effects  were  found  for  cognitive  style  on 
the  concept  map  measures.  It  may  be  that  there  were 
no  differences  between  groups  on  the  conceptual 
models  they  developed,  or  perhaps  concept  maps  do 
not  assess  conceptual  learning  in  ways  that  effec¬ 
tively  differentiate  learners.  Identifying  and  develop¬ 
ing  valid  and  reliable  measures  of  conceptual  knowl¬ 
edge  are  important  and  ongoing  challenges  for 
hypertext  researchers. 


ACTIVE  COGNITIVE  ENGAGEMENT 

Reading  hypertext  differs  from  reading  traditional 
printed  text  in  the  degree  to  which  the  reader  must 
make  choices  about  how  to  proceed  through  the  text, 
ostensibly  increasing  reader  interest  and  engaging  the 
reader  in  deeper  processing  of  the  information  (Patter¬ 
son,  2000).  A  fundamental  shift  in  the  reading  process 
relates  to  the  hypertext  reader  having  to  create  his  or 
her  own  path  through  the  text.  Actively  engaged  read¬ 
ers  tend  to  feel  a  greater  sense  of  control  over  what 
they  read  and  how  they  read  it.  Results  of  their  choices 
are  instantaneous,  and  readers  become  part  of  the 
meaning  construction  as  they  create  an  individualized 
text  that  is  not  as  tied  to  the  author’s  intended  message, 
as  is  the  case  with  traditional  printed  text.  Printed  text 
tends  to  formalize  the  role  of  the  author,  while  hyper¬ 
text  challenges  our  assumptions  about  the  roles  of  the 
author  and  the  reader;  thus,  the  educational  use  of 
hypertext  appears  to  have  great  potential  because  it 
forces  the  reader  to  actively  participate  in  creating 
meaning  from  the  text. 

A  critical  variable  influencing  whether  studying 
will  lead  to  knowledge  acquisition  is  the  degree  to 
which  students  become  actively  involved  in  trying 
to  make  sense  of  the  material  (Anderson-Inman  and 
Tenny,  1989).  This  sense-making  activity,  which  is 
inherent  in  constructing  a  situation  model,  relates  to 
Piaget’s  (1985)  notion  of  equilibration— an  essential 
element  in  the  learning  process.  In  their  study  of 
traditional  text-based  learning,  Mannes  and  Kintsch 
(1987,  p.  93)  noted  that  refraining  from  “providing 


readers  with  a  suitable  schema  and  thereby  forcing 
them  to  create  their  own  ...  might  make  learning 
from  texts  more  efficient.”  This  is  a  central  principle 
in  cognitive  flexibility  theory  and  provides  a  foun¬ 
dation  for  the  use  of  complex,  ill- structured  hyper¬ 
texts  to  promote  deep  and  connected  conceptual 
learning. 

Active  Knowledge  Construction 

As  mentioned  earlier,  Shapiro  (1998a)  compared  hier¬ 
archical  and  unstructured  systems  in  a  study  of  Amer¬ 
ican  history  learning.  Participants  in  that  study  per¬ 
formed  better  on  several  measures  when  presented 
with  the  unstructured  system.  Subjects  in  the  unstruc¬ 
tured  condition  significantly  outperformed  those  in  the 
hierarchical  condition  on  measures  that  included  how 
well  they  integrated  information  from  the  hypertext 
into  an  essay  that  demonstrated  their  learning.  It 
appeared  that  subjects  in  the  hierarchical  group  were 
able  to  navigate  more  passively  because  the  highly 
structured  nature  of  the  system  kept  them  oriented  in 
the  information  space.  As  a  consequence,  because  a 
well-organized  and  structured  model  was  presented, 
readers  did  not  have  to  actively  create  conceptual  con¬ 
nections  within  the  content  or  integrate  and  reflect  on 
the  content  to  make  decisions  about  which  links  to 
access.  Those  in  the  unstructured  system  condition, 
however,  had  to  more  carefully  regulate  their  learning 
and  be  more  principled  in  their  movements  through 
the  information.  Taken  together,  the  essay  and  naviga¬ 
tion  results  suggested  that  the  less  structured  system 
forced  readers  to  engage  in  more  active  processing  and 
to  systematically  monitor  and  regulate  their  learning 
to  develop  a  coherent  situation  model. 

Self-Regulated  Learning 

It  should  be  clear  by  now  that  hypertext  is  a  cognitively 
demanding  mode  of  learning.  As  such,  a  users’  ability 
to  supervise  his  or  her  learning  processes  would  seem 
to  be  essential  in  this  context.  Self-regulated  learning 
(SRL)  occurs  when  learners  establish  their  own  goals, 
then  monitor,  regulate,  and  control  their  thoughts  and 
behaviors  as  they  seek  to  achieve  those  goals  (Graesser 
et  al.,  2005;  Pintrich,  2000;  Winne,  2001 ;  Zimmerman, 
2001).  Relative  to  learning  with  hypertext,  SRL  mod¬ 
els  assume  that  learners  (1)  are  active,  constructive 
participants  in  the  learning  process;  (2)  are  capable 
of  monitoring,  controlling,  and  regulating  aspects  of 
their  own  cognition,  motivation,  behavior,  and  con¬ 
text;  (3)  have  a  goal  or  criterion  against  which  judg¬ 
ments  are  made  about  whether  the  learning  process 
should  continue  or  if  some  modification  in  strategy  or 
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metacognitive  monitoring  is  needed;  and  (4)  self-reg¬ 
ulation  of  cognition,  motivation,  and  behavior  medi¬ 
ates  personal  characteristics,  contextual  characteris¬ 
tics,  and  learning  (Azevedo,  2005;  Pintrich,  2000; 
Zimmerman,  2001).  A  series  of  studies  have  examined 
the  affects  of  SRL  on  learning  with  hypertext. 

There  is  evidence  that  students  who  demonstrate 
more  SRL  behavior  while  learning  with  hypertext  tend 
to  develop  more  sophisticated  mental  models  (Aze¬ 
vedo  et  al.,  2004).  Twenty-four  undergraduate  students 
spent  45  minutes  studying  the  human  circulatory  sys¬ 
tem  using  a  relational  hypermedia  encyclopedia. 
Gains  from  pre-  to  post-test  were  used  to  separate 
students  into  two  groups  (median  split)  based  on  the 
extent  of  conceptual  change  evidenced  by  their  ability 
to  identify  parts  of  the  heart  and  diagram  the  path  of 
blood  through  the  body,  as  well  as  by  a  free-recall 
measure.  Results  indicated  that  the  group  that  made 
greater  conceptual  gains  engaged  in  more  self-regu¬ 
lated  learning  behavior— particularly  in  the  areas  of 
monitoring  and  their  use  of  effective  strategies.  In 
terms  of  strategies,  the  group  that  made  greater  gains 
tended  to  use  proportionately  more  effective  strategies 
(e.g.,  summarizing,  rereading,  knowledge  elaboration) 
than  ineffective  strategies  (e.g.,  goal-free  searching, 
copying  information).  Students  who  made  little,  or  no, 
gains  tended  not  to  use  SRL  or  used  it  ineffectively. 
These  findings  suggest  that  effective  use  of  SRL  is  an 
important  consideration  when  using  hypertext  to  pro¬ 
mote  the  development  of  high-level  conceptual  under¬ 
standings.  Additional  research  has  examined  ways  to 
promote  readers’  use  of  SRL  when  learning  with 
hypertext. 

Azevedo  and  colleagues  (2002)  examined  the 
affects  of  prompting  SRL  use.  They  used  the  same 
human  circulatory  hypermedia  system  that  was 
described  previously  with  40  undergraduate  students 
who  were  (1)  trained  in  the  use  of  SRL  techniques 
themselves,  (2)  paired  with  a  human  tutor  who  was 
trained  in  SRL  techniques  (Winne,  1995,  2001)  and 
who  sat  with  the  subject  during  the  study  session  and 
prompted  the  use  of  SRL,  (3)  asked  to  self-generate 
and  complete  a  personal  goal,  or  (4)  given  a  series  of 
factual  questions  to  answer.  In  the  coregulation  condi¬ 
tion,  the  tutor  encouraged  metacognitive  strategies  by 
providing  a  variety  of  prompts;  specifically,  the  tutor 
encouraged  self-questioning,  content  evaluation,  judg¬ 
ments  of  learning,  planning,  goal  setting,  prior  knowl¬ 
edge  activation,  and  other  activities.  In  the  SRL  strat¬ 
egy  instruction  condition,  subjects  were  trained  to  do 
the  same  thing  as  the  tutor,  but  as  independent  learners. 
The  other  two  conditions  provided  no  metacognitive 
prompts,  tutors,  or  training.  Analyses  revealed  that  the 
sophistication  of  learners’  mental  models  shifted  to  a 


significantly  greater  degree  when  provided  with  tutors 
or  metacognitive  training  than  when  they  were  simply 
given  learning  goals  and  no  training.  Both  the  tutor 
group  and  the  strategy  instruction  group  demonstrated 
the  greatest  use  of  effective  learning  strategies  and  the 
least  incidence  of  ineffective  strategies;  however, 
although  post  hoc  tests  were  not  reported,  it  did  not 
appear  that  the  tutor  group  significantly  outperformed 
the  strategy  instruction  group. 

Given  that  having  a  one-on-one  tutor  to  prompt 
SRL  is  not  practical,  an  additional  study  singled  out 
the  effectiveness  of  SRL  training  and  examined  it  more 
directly.  In  a  study  of  131  undergraduate  students, 
Azevedo  and  Cromley  (2004)  had  subjects  in  the  treat¬ 
ment  condition  participate  in  a  30-minute  training  ses¬ 
sion  on  how  to  regulate  learning  of  the  circulatory 
system  when  using  a  hypermedia  environment.  Sub¬ 
jects  in  the  control  condition  received  no  SRL  training. 
All  subjects  were  given  a  global  learning  goal:  “Make 
sure  you  learn  about  the  different  parts  and  their  pur¬ 
pose,  how  they  work  both  individually  and  together, 
and  how  they  support  the  human  body”  (p.  526). 
Results  from  this  study  supported  findings  from  the 
previous  study:  (1)  Students  in  the  SRL  training  con¬ 
dition  gained  a  deeper  understanding  of  the  content, 
(2)  providing  students  with  an  overall  learning  goal 
and  not  SRL  training  did  not  promote  increased  con¬ 
ceptual  understanding,  and  (3)  students  in  the  SRL 
training  condition  used  SRL  strategies  more  frequently 
and  used  them  more  effectively. 

Explicit  training  in  SRL  techniques,  however,  is 
not  the  only  way  to  encourage  readers  to  monitor  their 
learning  in  hypertext-based  environments.  Automated 
processes  and  tools  that  can  be  integrated  into  hyper- 
text-based  learning  environments  include  learner-con¬ 
trolled  question-answer  facilities  and  artificial-intelli¬ 
gence-based  animated  conversational  agents  (Graesser 
et  al.,  2005;  Kauffman,  2004).  Kauffman  (2004)  devel¬ 
oped  a  hypertext  system  that  integrated  questions 
designed  to  encourage  metacognition.  Subjects  were 
presented  with  a  hypertext  on  educational  measure¬ 
ment.  Half  of  the  subjects  were  assigned  to  work  with 
a  system  that  presented  automated  self-monitoring 
prompts  in  the  form  of  questions.  The  prompts 
appeared  each  time  a  reader  moved  from  one  node  to 
another.  If  students  were  unable  to  correctly  answer 
the  question,  they  were  encouraged  to  go  back  and 
review  the  page  they  had  just  read.  The  second  group 
of  subjects  were  able  to  click  freely  on  link  buttons 
and  move  to  a  new  page  without  answering  any  ques¬ 
tions  about  their  understanding. 

Both  groups  performed  comparably  on  the 
declarative  knowledge  test;  however,  students  in  the 
metacognitive  prompt  condition  outperformed  their 
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counterparts  on  a  post-test  that  assessed  their  ability 
to  apply  what  they  learned  to  real-world  problems  (a 
measure  of  situation  model  learning).  Interestingly, 
the  groups  did  not  differ  in  their  awareness  of  meta¬ 
cognition.  Thus,  providing  automated  self-regulation 
prompts  was  an  effective  means  of  encouraging  deep 
learning,  even  if  subjects  were  unaware  of  how  the 
prompts  altered  their  thinking  about  their  own  learn¬ 
ing.  It  should  also  be  noted  that,  because  of  the  small 
size  of  this  hypertext,  there  were  few  link  buttons  and 
subjects  received  only  three  or  four  prompts  during 
the  learning  period.  That  clear  improvement  in  learn¬ 
ing  was  observed  after  such  a  modest  intervention 
speaks  to  the  promise  of  using  such  prompts. 

The  active  participation  that  takes  place  when  read¬ 
ing  hypertext  clearly  places  additional  cognitive 
demands  on  the  reader.  As  in  traditional  reading  of 
printed  text,  the  learner  must  engage  basic  lower  level 
processes  (such  as  letter  recognition  and  decoding 
words)  and  higher  level  processes  (such  as  relating 
new  information  with  prior  knowledge).  Further,  read¬ 
ing  hypertext  requires  additional  metacognitive  func¬ 
tioning  such  as  choosing  what  to  read  and  deciding  on 
the  sequence  for  reading  information.  As  Niederhauser 
and  colleagues  (2000)  pointed  out,  less  proficient  com¬ 
puter  users  must  use  cognitive  resources  simply  to 
operate  the  computer  (e.g.,  working  the  mouse,  press¬ 
ing  keys,  activating  on  screen  buttons).  Given  these 
considerations  and  the  cognitive  load  issues  previously 
raised  in  this  chapter,  it  is  not  surprising  that  explicit 
strategies  for  managing  cognitive  resources  and  regu¬ 
lating  the  learning  process  seem  to  be  effective. 
Although  Azevedo’s  work  has  been  almost  exclusively 
with  low-prior-knowledge  learners  in  hypermedia 
learning  environments,  there  appears  to  be  clear  prom¬ 
ise  for  the  use  of  SRL  techniques  for  enhancing  learn¬ 
ing  with  hypertext. 


CONCLUSION 

Hypertext  research  continues  to  be  a  challenging  but 
fruitful  area.  The  numerous  interacting  and  at  times 
confounding  variables  contribute  to  the  difficulty  of 
building  our  knowledge  base  for  learning  with  hyper¬ 
text,  and  measures  of  learning  for  conceptual  knowl¬ 
edge  continue  to  pose  challenges  in  this  and  many 
other  areas.  Yet,  a  core  of  rigorous  and  insightful 
research  has  emerged  during  the  past  25  years  since 
technological  advances  made  computer-based  hyper¬ 
text  possible.  Current  theoretically  advanced  work  in 
coherence  and  self-regulated  learning  holds  great 
promise  for  the  future  . 
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ABSTRACT 

This  chapter  reviews  research  on  the  application  of 
computer- mediated  technologies  to  learning.  This 
includes  traditional  computer-based  training,  multime¬ 
dia,  hypertext  and  hypermedia,  interactive  simulation, 
intelligent  tutoring  systems,  inquiry -based  information 
retrieval,  animated  pedagogical  agents,  virtual  environ¬ 
ments  with  agents,  serious  games,  and  collaborative 
learning  environments.  Most  of  these  systems  encour¬ 
age  active  learning,  knowledge  construction,  inquiry, 
and  exploration  on  the  part  of  the  student,  as  opposed 
to  being  exposed  to  information  delivery  systems. 


Unfortunately,  the  learning  strategies  of  most  students 
are  extremely  limited,  so  the  systems  must  provide 
modeling  of  effective  strategies,  intelligent  scaffolding, 
and  accurate  feedback.  Available  research  has  con¬ 
firmed  that  students  learn  from  most  of  these  advanced 
learning  environments  compared  to  classroom  lectures, 
reading  textbooks,  and  noninteractive  control  condi¬ 
tions;  however,  empirical  research  on  learning  is  con¬ 
spicuously  absent  on  some  of  these  environments,  such 
as  virtual  environments  with  agents  and  serious  games. 
We  advocate  a  research  roadmap  for  the  future  that 
systematically  investigates  a  broad  landscape  of  learn¬ 
ing  technologies  (T),  pedagogical  mechanisms  (M), 
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learning  goals  (G),  and  learner  characteristics  (L).  A 
well-understood  TMGL  landscape  will  provide  a  prin¬ 
cipled  foundation  for  assigning  the  right  learning  envi¬ 
ronment  to  the  right  learner  at  the  right  time.  The  costs 
of  developing  advanced  learning  environments  are 
often  high,  so  we  encourage  designers  to  follow  stan¬ 
dards  for  reusing  learning  objects ,  lessons ,  and  systems 
developed  by  the  research  community.  One  major  tech¬ 
nical  challenge  will  be  to  develop  authoring  tools  to 
make  it  easy  for  designers,  instructors,  and  students  to 
develop  new  content  that  incorporates  these  advanced 
learning  technologies. 


KEYWORDS 

Animated  pedagogical  agents:  Talking  heads  with 
speech,  facial  expressions,  and  gestures  that  imple¬ 
ment  pedagogical  strategies. 

Hypertext/hypermedia:  Pages  of  text  and  other  media 
with  hot  spots  that  users  can  click  on  to  access 
other  pages. 

Inquiry  learning:  Students  actively  learn  by  asking 
questions  and  interpreting  answers  or  by  formulat¬ 
ing  and  testing  hypotheses. 

Intelligent  tutoring  systems:  Intelligent  computer  sys¬ 
tems  that  model  the  learner’s  knowledge  and  skills 
at  a  fine-grained  level  and  that  adaptively  respond. 

Interactive  simulation:  Learners  manipulate  compo¬ 
nents  and  parameters  of  a  complex  system  and 
observe  what  happens  when  system  output  is  gen¬ 
erated. 

Serious  games:  Games  that  help  students  learn  new 
content,  strategies,  and  skills  that  are  relevant  to 
academic  and  practical  subject  matter. 


INTRODUCTION 

A  revolution  in  technology -based  training  has  occurred 
since  the  advent  of  computers  50  years  ago.  In  1960, 
none  of  the  following  kinds  of  learning  environments 
discussed  in  this  chapter  existed:  adaptive  computer- 
based  training,  dynamic  multimedia,  hypertext,  hyper¬ 
media,  interactive  simulation,  intelligent  tutoring  sys¬ 
tems,  inquiry-based  information  retrieval,  animated 
pedagogical  agents,  virtual  environments  with  agents, 
serious  games,  and  collaborative  learning  environ¬ 
ments.  Most  of  these  environments  were  not  available 
20  years  ago  and  are  still  not  mainstream  technologies 
in  schools  today;  however,  the  World  Wide  Web  has 
exemplars  or  mature  technologies  for  all  these  learning 
environments,  so  they  are  potentially  available  to  all 
Web  users. 


The  current  revolution  in  technology-based  train¬ 
ing  requires  more  research  in  the  cognitive,  learning, 
and  social  sciences.  Most  students  do  not  know  how 
to  use  advanced  learning  environments  effectively,  so 
modeling,  scaffolding,  and  feedback  on  their  optimal 
use  are  necessary.  Indeed,  students  often  do  not  even 
know  how  to  get  started.  When  researchers  assess 
learning  gains  from  these  learning  environments 
(LEs),  the  results  are  often  disappointing  because  the 
developers  of  the  systems  have  not  had  sufficient  train¬ 
ing  in  cognitive  science,  learning  science,  pedagogy, 
curriculum  design,  human-computer  interaction, 
social  interaction,  discourse  processing,  and  other  rel¬ 
evant  fields  that  are  coordinated  in  interdisciplinary 
teams.  Far  too  many  LEs  are  launched  without  the 
required  empirical  testing  on  usability,  engagement, 
and  learning  gains.  The  pace  of  new  technologies  hit¬ 
ting  the  market  is  so  fast  that  there  typically  is  not 
enough  time  to  adequately  test  the  systems.  The  only 
way  to  cope  with  the  pace  of  technological  innovations 
is  to  develop  theoretical  models  and  tools  to  forecast 
the  quality  of  LE  designs  before  or  during  their  poten¬ 
tial  development. 

The  role  of  technology  in  education  and  training 
has  always  had  its  critics.  Cuban  (1986,  2001)  docu¬ 
mented  that  technology  has  historically  had  a  negligible 
impact  on  improvements  in  education,  pointing  to  the 
television  and  the  radio  as  exemplar  disappointments. 
Clark  (1983)  argued  that  it  is  the  pedagogy  underlying 
a  LE,  not  the  technology  per  se .  that  typically  explains 
learning  gains.  That  conclusion,  of  course,  motivates 
the  need  to  investigate  how  particular  technologies  are 
naturally  aligned  with  particular  pedagogical  princi¬ 
ples,  theories,  models,  hypotheses,  or  intuitions;  for 
example,  a  film  on  how  to  design  a  nuclear  power  plant 
would  not  be  a  technology  that  is  naturally  aligned  with 
a  pedagogical  theory  that  emphasizes  active  discovery 
learning.  Reading  texts  on  the  Web  about  negotiation 
strategies  is  not  well  aligned  with  a  social  learning 
theory  that  embraces  modeling-scaffolding-fading. 
There  somehow  needs  to  be  a  systematic,  principled 
foundation  for  creating  technologies  that  mesh  with 
theoretical  learning  mechanisms. 

The  first  giant  step  is  to  map  out  a  landscape  of 
learning  technologies  and  learning  theories  (Bransford 
et  al.,  2000;  Jochems  et  al„  2004;  Jonassen,  2004; 
Merrill,  2002;  O’Neil  and  Perez,  2003).  Any  given 
technology  (T)  affords  a  number  of  cognitive,  peda¬ 
gogical,  and  social  mechanisms  (M)  (Gee,  2003; 
Kozma,  1994;  Norman,  1988).  Examples  of  these 
mechanisms  are  mastery  learning  with  presentation- 
test-feedback-branching,  building  on  prerequisites, 
practice  with  problems  and  examples,  multimedia 
learning,  modeling-scaffolding-fading,  reciprocal 
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teaching,  problem-based  learning,  inquiry  learning, 
and  collaborative  knowledge  construction.  Nearly  all 
of  these  mechanisms  emphasize  that  the  learner 
actively  constructs  knowledge  and  builds  skills,  as 
opposed  to  merely  being  exposed  to  information  deliv¬ 
ered  by  the  LE.  In  addition  to  these  TM  mappings,  it 
is  essential  to  consider  the  goals  (G)  of  the  LE;  that 
is,  is  the  LE  designed  for  quick  training  on  shallow 
knowledge  about  an  easy  topic  or  for  deep  learning 
about  explanations  of  a  complex  system?  It  is  essential 
to  consider  the  characteristics  of  the  learners  (L),  such 
as  whether  they  have  high  vs.  low  knowledge  of  the 
subject  matter,  high  vs.  low  verbal  ability,  high  vs.  low 
prior  experiences  with  technology,  and  so  on.  The 
resulting  TMGL  landscape  of  cells  must  be  explored. 
Some  cells  are  promising  conditions  for  learning,  oth¬ 
ers  are  impossible,  and  groups  of  cells  give  rise  to 
interesting  interactions.  We  advocate  a  long-term 
research  roadmap  that  identifies  an  adequate  TMGL 
landscape  for  education  and  training.  This  would  pro¬ 
vide  a  practical  foundation  for  assigning  the  right  LE 
to  the  right  student  at  the  right  time. 

The  TMGL  landscape  would  also  allow  research¬ 
ers  to  select  research  projects  that  strategically  cover 
cells  that  require  attention;  for  example,  not  enough 
research  has  been  done  on  learning  gains  from  serious 
games  that  afford  active  discovery  learning  in  adults 
with  low  reading  ability.  In  contrast,  a  wealth  of 
research  exists  with  regard  to  learning  gains  from  intel¬ 
ligent  tutoring  systems  on  algebra  and  physics,  span¬ 
ning  the  gamut  of  learner  characteristics,  pedagogical 
mechanisms,  and  learning  goals  (Anderson  et  al., 
1995;  Doignon  and  Lalmagne,  1999;  VanLehn  et  al., 
2002).  As  another  example,  currently  being  debated 
are  the  conditions  under  which  animated  pedagogical 
agents  are  effective  in  improving  learning  and  motiva¬ 
tion,  so  the  corresponding  cells  would  need  attention. 
An  executive  with  high  knowledge  and  high  stress 
would  probably  not  have  the  patience  to  learn  from  an 
embodied  animated  conversational  agent,  whereas 
such  an  environment  might  fit  the  bill  for  a  child  who 
has  persistent  motivational  problems. 

The  LEs  must  be  evaluated  from  the  standpoint  of 
learning  gains,  usage,  engagement,  and  return  on 
investment.  Regarding  learning  gains,  the  outcome 
variables  include  tests  of  retention  for  shallow  vs.  deep 
knowledge,  problem  solving,  and  transfer  of  knowl¬ 
edge  or  skill  to  different  but  related  contexts.  Meta¬ 
analyses  have  revealed  that  computerized  learning 
environments  fare  well  compared  to  classroom  instruc¬ 
tion  and  other  naturalistic  control  conditions  (Corbett, 
2001;  Dodds  and  Lletcher,  2004;  Wisher  and  Lletcher, 
2004);  these  studies  have  reported  effect  sizes  of  .39 
for  computer-based  training,  .50  for  multimedia,  and 


1.08  for  intelligent  tutoring  systems.  An  effect  size  is 
defined  as  the  difference  in  performance  between  the 
treatment  and  control  conditions,  measured  in  standard 
deviation  units  (called  sigmas).  Precious  little  data 
exist  with  regard  to  learning  gains  from  most  other 
classes  of  LEs,  however,  so  research  is  needed  in  these 
arenas.  Whereas  learning  gains  are  routinely  reported 
in  published  studies,  data  are  often  incomplete  with 
regard  to  usage  (attrition),  engagement  (including  how 
much  the  learners  like  the  system),  system  develop¬ 
ment  time,  study  time,  instructor-student  interaction 
time,  and  costs.  The  latter  measures  are  needed  for 
systematic  and  practical  assessments  of  return  on 
investment  (Spector,  2005). 

CLASSES  OF  LEARNING 
TECHNOLOGIES 

This  section  examines  classes  of  computer-mediated 
learning  technologies.  Lor  each,  we  will  identify 
salient  theoretical  frameworks,  empirical  findings,  and 
opportunities  for  future  research. 

Adaptive  Computer-Based  Training 

A  prototypical  computer-based  training  (CBT)  system 
involves  mastery  learning.  The  learner  (1)  studies  mate¬ 
rial  presented  in  a  lesson,  (2)  gets  tested  with  a  multi¬ 
ple-choice  test  or  another  objective  test,  (3)  gets  feed¬ 
back  on  the  test  performance,  (4)  restudies  the  material 
if  the  performance  in  step  2  is  below  threshold,  and  (5) 
progresses  to  a  new  topic  if  performance  exceeds 
threshold.  The  order  of  topics  presented  and  tested  can 
follow  different  pedagogical  models,  such  as  ordering 
on  prerequisites  (Gagne,  1985),  a  prestructured  top- 
down  hierarchical  organization  (Ausubel  et  al.,  1978), 
a  knowledge  space  model  that  attempts  to  fill  learning 
deficits  and  correct  misconceptions  (Doignon  and  Lal¬ 
magne,  1999),  or  other  models  that  allow  dynamic 
sequencing  and  navigation  (O’Neil  and  Perez,  2003). 

The  materials  presented  in  a  lesson  can  vary  con¬ 
siderably  in  CBT — organized  texts  with  figures ,  tables , 
and  diagrams  (essentially  books  on  the  Web);  multi- 
media;  problems  to  solve;  example  problems  with 
solutions  worked  out;  and  other  classes  of  learning 
objects .  CBT  has  been  extensively  studied  over  the  last 
few  decades  and  has  evolved  into  a  mature  technology 
that  is  ripe  for  scaling  up  at  an  economical  cost.  As 
mentioned  earlier,  meta-analyses  show  effect  sizes  of 
.39  sigma  compared  to  classrooms  (Dodds  and 
Lletcher,  2004).  The  amount  of  time  that  learners 
spend  studying  the  material  in  CBT  has  a  .35  correla¬ 
tion  with  learning  performance  (Taraban  et  al.,  2001) 
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and  can  be  optimized  by  contingencies  that  distribute 
practice.  Learning  researchers  have  explored  relations 
between  learning  activities  and  characteristics  of  the 
learners,  such  as  subject-matter  knowledge,  cognitive 
abilities,  learning  style,  age,  gender,  and  so  on.  Such 
aptitude-treatment  interactions  underscore  the  impor¬ 
tance  of  adapting  the  learning  materials  to  the  learner’s 
cognitive  and  experiential  profile. 

The  nature  of  the  feedback  in  CBT  merits  careful 
attention  (Kulhavy  and  Stock,  1989;  Moreno  and 
Mayer,  2005;  Shute,  2006).  Testing  influences  the 
course  of  learning  in  formative  evaluation  but  simply 
scales  a  learner’s  mastery  in  summative  evaluation 
(Hunt  and  Pellegrino,  2002;  Shute,  2006).  A  test  score 
alone  is  adequate  feedback  for  informing  the  learner 
on  how  well  they  are  doing  but  is  not  useful  for  clar¬ 
ifying  specific  deficits  in  knowledge  or  skill.  We  need 
a  better  understanding  of  the  conditions  under  which 
the  learner  benefits  from  feedback  in  the  form  of  cor¬ 
rect  answers,  explanations  of  why  correct  answers  are 
correct,  identification  of  misconceptions,  explanations 
of  the  misconceptions,  and  other  forms  of  elaboration. 
Researchers  also  need  to  identify  conditions  in  which 
it  is  best  to  withhold  feedback  so  learners  acquire  self- 
regulated  learning  strategies. 

The  nature  of  the  test  format  is  an  important  deter¬ 
minant  of  the  magnitude  of  learning  gains.  Most  mul¬ 
tiple-choice  questions  in  actual  courses,  e-learning 
facilities,  and  commercial  test  banks  tap  shallow  levels 
of  comprehension  rather  than  deep  levels  (Ozuru  et  al., 
2006;  Wisher  and  Graesser,  2007).  Shallow  questions 
quiz  the  learner  on  explicit  information  in  the  lessons, 
definitions  of  terms,  properties  of  concepts,  steps  in 
procedures,  and  other  forms  of  perception-based  and 
memory-based  processes  that  require  little  or  no  rea¬ 
soning.  Deep-level  questions  require  the  learner  to 
understand  causal  mechanisms,  logical  justification  of 
claims,  explanations  of  complex  systems,  mental  mod¬ 
els,  inferences,  and  applications  (Bloom,  1956;  Chi  et 
al.,  1994;  Graesser  and  Person,  1994).  An  emphasis  in 
training  on  shallow  knowledge  has  the  unfortunate 
consequence  of  learners  settling  for  shallow  standards 
of  comprehension  (Baker,  1985;  Dwyer,  2005;  Otero 
and  Graesser,  2001).  Experimental  investigations  need 
to  manipulate  the  quality  of  questions  affiliated  with 
a  course  and  measure  the  resulting  impact  on  retention, 
problem  solving,  and  transfer  performance. 

Conventional  CBT  has  two  potential  disadvan¬ 
tages,  both  of  which  require  confirmation  with  addi¬ 
tional  research.  First,  some  populations  of  learners  are 
not  engaged  in  the  learning  process  provided  by  CBT, 
particularly  LEs  that  lack  multimedia.  Conventional 
electronic  page-turning  CBT  is  fine  for  motivated 
learners  who  desire  training  on  moderate  to  easy  mate¬ 


rial  in  the  minimum  amount  of  time  but  not  for  those 
who  lack  motivation  and  require  more  entertainment. 
Second,  CBT  seems  more  appropriate  for  acquiring 
inert  knowledge  than  active  application  of  knowledge 
(Bereiter  and  Scardamalia,  1985;  Bransford  et  al., 
2000)  and  for  shallow  knowledge  rather  than  deep 
knowledge.  Other  types  of  LEs  appear  to  be  more 
appropriate  for  enhancing  engagement,  active  applica¬ 
tion  of  knowledge  and  skills,  and  depth  of  mastery. 

Multimedia 

Material  can  be  delivered  in  different  presentation 
modes  (verbal,  pictorial) ,  sensory  modalities  (auditory, 
visual),  and  delivery  media  (text,  video,  simulations). 
The  impact  of  different  forms  of  multimedia  has  been 
extensively  investigated  by  Mayer  and  his  colleagues 
(Mayer,  2005).  Meta-analyses  reported  by  Dodds  and 
Fletcher  (2004)  indicated  an  effect  size  of  .50  sigma 
for  multimedia  learning,  whereas  it  is  considerably 
higher  (1.00  sigma)  in  the  meta-analyses  reported  by 
Mayer  (2005).  In  many  of  these  studies,  retention, 
problem  solving,  and  transfer  of  training  are  facilitated 
by  multimedia  because  the  separate  modalities  offer 
multiple  representations  (Paivio,  1986),  conceptually 
richer  and  deeper  representations  (Craik  and  Lockhart, 
1972),  multiple  retrieval  routes,  and  more  cognitive 
flexibility  (Spiro  et  al.,  1991).  It  is  important,  however, 
that  the  multimedia  presentation  does  not  present  a 
large  cognitive  load  and  split  the  learner’s  attention 
(Kalyuga  et  al.,  1999;  Sweller  and  Chandler,  1994); 
for  example,  a  picture  on  the  screen  with  a  voice  that 
explains  highlighted  aspects  of  the  picture  provides 
multiple  codes  without  overloading  working  memory. 
If  text  on  the  screen  redundantly  echoes  the  spoken 
explanations,  then  cognitive  overload,  interference, 
and  a  split  attention  effect  (between  print  and  the  pic¬ 
ture)  may  result.  Inputs  within  the  same  sensory 
modality  interfere  with  each  other  more  than  inputs 
from  separate  modalities. 

Mayer  (2005)  has  documented  and  empirically 
confirmed  a  number  of  principles  that  predict  when 
different  forms  of  multimedia  will  facilitate  learning. 
Among  these  multimedia  learning  principles  are  the 
principles  of  modality,  coherence,  redundancy,  and 
individual  differences.  These  principles  are  based  on  a 
cognitive  model  that  specifies  the  processes  of  select¬ 
ing,  organizing,  and  integrating  information.  Mayer’s 
multimedia  learning  model  attempts  to  predict  when 
and  how  to  highlight  a  text  or  diagram  with  arrows, 
lines,  color,  sound,  spoken  messages,  and  so  on. 

One  counterintuitive  result  of  research  with  multi- 
media  is  that  noninteractive  animations  of  a  complex 
process  often  have  no  impact  on  learning  (Lowe,  2004; 
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Rieber,  1996;  Tversky  et  al.,  2002).  The  animations 
run  a  number  of  risks:  not  being  easy  to  understand, 
being  transient,  moving  too  quickly,  presenting  dis¬ 
tracting  material,  placing  demands  on  working  mem¬ 
ory,  and  depicting  processes  in  a  fashion  other  than 
what  the  learner  would  otherwise  actively  construct 
(Hegarty,  2004).  In  contrast,  a  static  picture  remains 
on  the  screen  for  inspection  (off-loading  working 
memory  via  an  external  memory),  is  available  for 
active  construction  of  interpretations  at  the  learner’s 
leisure,  and  potentially  stimulates  a  mental  construc¬ 
tion  of  the  dynamic  process  (Hegarty  et  al.,  2003). 
Although  some  researchers  have  documented  learning 
gains  from  animations,  a  persistent  question  is  whether 
information  equivalence  between  the  simulation  and 
control  conditions  exists. 

A  formal  cognitive  model  is  needed  that  predicts 
the  impact  of  particular  forms  of  multimedia  on  learn¬ 
ing  at  varying  levels  of  depth.  What  is  desired,  for 
example,  is  a  GOMS  (Goals,  Operators,  Methods,  and 
Selection  Rules)  model  (Card  et  al.,  1983;  Gray  et  al., 
1993)  of  multimedia  learning  that  has  the  theoretical 
scope,  analytical  precision,  and  predictive  power  that 
GOMS  provided  for  the  field  of  human-computer 
interaction  in  the  1980s  and  1990s.  A  satisfactory 
model  would  consider  the  cognitive  representations  of 
the  content,  the  processes  needed  to  perceive  and  inter¬ 
pret  the  multimedia  presentations,  the  knowledge  of 
the  learner,  and  the  tasks  the  learner  needs  to  perform. 
A  fine-grained  cognitive  model  appears  to  be  needed 
to  resolve  some  inconsistent  findings  in  the  literature 
and  to  make  a  priori  predictions. 

More  research  is  needed  to  resolve  a  number  of 
other  questions  about  multimedia.  How  can  learners 
be  trained  to  interpret  complex  multimedia  displays? 
What  sort  of  semiotic  theory  is  needed  to  explain  how 
pictures/icons  are  interpreted  and  integrated  with  ver¬ 
bal  input?  How  can  cognitive  theories  inform  graphic 
artists?  How  can  multimedia  presentations  be  tailored 
to  the  profile  of  the  learners,  including  those  with 
disabilities?  How  can  different  forms  of  content  be 
represented  with  different  types  of  multimedia?  Given 
that  most  research  on  multimedia  is  based  on  experi¬ 
ments  in  which  material  is  presented  for  less  than  an 
hour,  how  well  does  the  existing  multimedia  research 
scale  up  to  LEs  that  are  used  for  several  weeks?  Will 
the  razzle-dazzle  of  exotic  multimedia  end  up  being 
too  exhausting  to  the  learner  over  the  long  haul? 

Hypertext  and  Hypermedia 

These  systems  provide  a  large  space  of  Web  pages  with 
texts ,  pictures ,  animations ,  and  other  media.  Each  page 
has  hot  spots  for  the  learner  to  click  and  explore.  The 


learner  has  free  rein  to  maneuver  through  the  hyper- 
text/hypermedia  space,  which  of  course  is  an  ideal 
environment  for  active  learning  and  inquiry.  Unfortu¬ 
nately,  most  learners  do  not  have  the  skills  of  self- 
regulation  and  metacognition  to  intelligently  search 
through  the  space  (Azevedo  and  Cromley,  2004;  Con¬ 
klin,  1987;  Winne,  2001),  so  they  get  lost,  get  side¬ 
tracked  by  seductive  details,  and  lose  sight  of  the  pri¬ 
mary  learning  goals.  These  liabilities  of  this 
technology  have  resulted  in  mixed  reports  of  learning 
gains  from  hypertext/hypermedia  compared  with  a  pre¬ 
designed  sequence  of  materials  by  an  expert  author 
(Azevedo  and  Cromley,  2004;  Dillon  and  Gabbard, 
1998;  Rouet,  2006).  Learners  benefit  from  a  naviga¬ 
tional  guide  that  trains,  models,  and  scaffolds  good 
inquiry  strategies  (Azevedo  and  Cromley,  2004). 
Another  aid  is  an  interface  that  shows  learners  an  over¬ 
view  of  the  media  space  as  well  as  the  locations  that 
they  have  visited;  a  graphical  interface  or  labeled  hier¬ 
archy  may  be  suitable  for  providing  this  global  context 
(Lee  and  Baylor,  2006).  More  research  is  necessary  on 
training  learners  how  to  effectively  use  hypertext/ 
hypermedia  to  achieve  specific  learning  goals.  We 
also  need  research  that  assesses  and  increases  the 
likelihood  that  designers  of  these  environments  use 
principles  of  cognition,  human  factors,  semiotics,  and 
human-computer  interaction.  Many  designers  con¬ 
gest  the  Web  pages  with  excessive  options,  clutter, 
and  seductive  details  (i.e.,  feature  bloat),  which  over¬ 
load  the  cognitive  system  and  distract  learners  with 
low  ability. 

Interactive  Simulation 

Interactive  simulation  allegedly  produces  more  learn¬ 
ing  than  having  students  simply  view  simulations.  In 
interactive  simulation,  the  student  can  actively  control 
input  parameters  and  observe  the  results  on  the  system. 
The  learner  can  slow  down  animations  to  inspect  the 
process  in  detail,  zoom  in  on  important  subcomponents 
of  a  system  during  the  course  of  a  simulation,  observe 
the  system  from  multiple  viewpoints,  and  systemati¬ 
cally  relate  inputs  to  outputs  (Kozma,  2000).  Some 
studies  have  indeed  shown  advantages  of  interactive 
simulation  on  learning,  whereas  others  have  shown  no 
gains  of  interactive  simulation  over  various  control 
conditions  (Deimann  and  Keller,  2006;  Jackson  et  al., 
2006;  Stern  et  al.,  2006;  van  der  Meij  and  de  Jong, 
2006a,b).  The  empirical  results  are  therefore  mixed  and 
in  need  of  a  meta-analysis,  assuming  that  a  sufficient 
number  of  empirical  studies  have  been  conducted. 

Unfortunately,  simulations  tend  to  have  complex 
content  and  complex  interfaces  that  are  unfamiliar  to 
learners.  Students  with  low  domain  knowledge  or 
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computer  expertise  have  trouble  getting  started  and 
managing  the  human-computer  interface.  Students 
with  more  knowledge  and  expertise  often  do  not  under¬ 
stand  how  to  strategically  interact  with  the  simulation 
to  advance  learning.  Consequently,  designers  of  these 
systems  are  sometimes  disappointed  in  how  little  or 
ineffectively  the  simulations  are  used.  Training,  mod¬ 
eling,  and  scaffolding  of  the  use  of  complex  simula¬ 
tions  must  be  provided  before  they  can  be  used  effec¬ 
tively.  A  game  environment  with  points  and  feedback 
(as  in  the  case  of  Flight  Simulator;  see  http://www. 
microsoft.com/games/flightsimulator/)  is  presumably 
motivating  and  effective  in  promoting  learning  gains. 
Researchers  need  to  investigate  the  cognitive  and  moti¬ 
vational  mechanisms  that  encourage  intelligent  inter¬ 
actions  with  these  simulations. 

Intelligent  Tutoring  Systems 

Intelligent  tutoring  systems  (ITSs)  track  the  knowl¬ 
edge  states  of  learners  in  fine  detail  and  adaptively 
respond  with  activities  that  are  sensitive  to  these 
knowledge  states.  The  processes  of  tracking  knowl¬ 
edge  (called  user  modeling )  and  adaptively  responding 
to  the  learner  ideally  incorporate  computational  mod¬ 
els  in  artificial  intelligence  and  cognitive  science,  such 
as  production  systems,  case-based  reasoning,  Bayesian 
networks ,  theorem  proving,  and  constraint  satisfaction 
algorithms.  Successful  systems  have  been  developed 
for  mathematically  well-formed  topics,  including  alge¬ 
bra;  geometry;  programming  languages  (Cognitive 
Tutors)  (Anderson  et  al.,  1995;  Koedinger  et  al.,  1997); 
physics  (Andes,  Atlas,  and  Why/ Atlas)  (VanLehn  et 
al.,  2002,  2007);  electronics  (SHERLOCK)  (Lesgold 
et  al.,  1992);  and  information  technology  (Mitrovic  et 
al.,  2004).  These  systems  show  impressive  learning 
gains  (1.00  sigma,  approximately),  particularly  for 
deeper  levels  of  comprehension.  ITSs  are  expensive  to 
build  but  are  now  in  the  phase  of  scaling  up  for  wide¬ 
spread  use.  One  challenge  in  getting  more  widespread 
use  of  these  systems  is  that  instructors  do  not  know 
what  systems  are  available,  how  to  access  and  use 
them,  and  how  to  integrate  the  ITSs  in  course  curricula. 
A  second  challenge  lies  in  the  authoring  of  new  subject 
matter  content  in  a  timely  fashion  that  keeps  pace  with 
the  growth  of  knowledge. 

Some  of  the  recent  ITSs  have  attempted  to  handle 
knowledge  domains  that  are  not  mathematically  pre¬ 
cise  and  well  formed.  The  Intelligent  Essay  Assessor 
(Foltz  et  al.,  2000;  Landauer  et  al.,  2000)  and  e-Rater 
(Burstein,  2003)  grades  essays  on  science,  history,  and 
other  topics  as  reliably  as  experts  of  English  compo¬ 
sition.  Learners  can  get  very  quick  feedback  on  the 
quality  of  their  essays  and  on  particular  deficiencies. 


Summary  Street  (Kintsch  et  al.,  2000)  helps  the  learner 
summarize  texts  by  identifying  idea  gaps  and  irrelevant 
information.  AutoTutor  (Graesser  et  al.,  2004b,  2005a) 
helps  college  students  learn  about  computer  literacy, 
physics,  and  critical  thinking  skills  by  holding  conver¬ 
sations  in  natural  language.  AutoTutor  shows  learning 
gains  of  approximately  .80  sigma  compared  with  read¬ 
ing  a  textbook  for  an  equivalent  amount  of  time 
(Graesser  et  al.,  2004b;  VanLehn  et  al.,  2007).  These 
systems  automatically  analyze  language  and  discourse 
by  incorporating  recent  advances  in  computational  lin¬ 
guistics  (Jurafsky  and  Martin,  2000)  and  information 
retrieval,  notably  latent  semantic  analysis  (Dumais, 
2003;  Landauer  et  al.,  2007;  Millis  et  al.,  2004). 

There  are  three  major  reasons  for  encouraging 
more  research  and  development  on  ITSs  with  tutorial 
dialog  in  a  natural  language.  The  first  is  the  need  for 
intelligent  training  on  subject  matters  that  involve  con¬ 
ceptualizations  and  verbal  reasoning  that  are  not  math¬ 
ematically  well  formed.  Second,  natural  language  dia¬ 
log  is  a  frequent  form  of  communication,  as  in  the  case 
of  chat  rooms,  multi-user  domains  (MUDs),  object- 
oriented  MUDs  (MOOs),  games,  and  instant  messag¬ 
ing  (Kinzie  et  al.,  2005;  Looi,  2005);  indeed,  the  major¬ 
ity  of  teenagers  in  the  United  States  use  instant  mes¬ 
saging  every  day.  Third,  the  revolutionary  advances  in 
computational  linguistics,  corpus  analyses,  speech  rec¬ 
ognition,  and  discourse  processing  (Graesser  et  al., 
2003a)  have  made  it  possible  to  make  significant 
progress  in  developing  natural  language  dialog  sys¬ 
tems.  Two  points  of  caution  must  be  noted  in  this  line 
of  research  and  development.  It  is  important  to  focus 
on  making  the  conversational  systems  more  responsive 
to  the  learner’s  ideas,  threads  of  reasoning,  and  ques¬ 
tions  as  opposed  to  merely  coaching  the  learner  in 
following  the  tutor’s  agenda.  Second,  it  is  necessary  to 
have  a  fine-grained  assessment  of  what  aspects  of  nat¬ 
ural  language  dialog  facilitate  learning,  engagement, 
and  motivation.  Learners  get  irritated  with  conversation 
partners  who  do  not  seem  to  be  listening  at  a  suffi¬ 
ciently  deep  level  (Mishra,  2006;  Walker  et  al.,  2003). 

Inquiry-Based  Information  Retrieval 

One  type  of  inquiry  learning  consists  of  asking  ques¬ 
tions  and  searching  for  answers  in  an  information 
repository  (Graesser  et  al.,  2004a;  Wisher  and 
Graesser,  2007).  High-knowledge  individuals  some¬ 
times  do  not  have  the  patience  to  wade  through  learning 
materials  but  prefer  to  actively  ask  questions  and  seek 
answers  to  achieve  their  goals.  Query -based  informa¬ 
tion  retrieval  occurs  when  Google  is  used  to  access 
information  on  the  Web.  The  queries  do  not  need  to  be 
well  formed  semantically  and  syntactically  because  the 
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system  uses  keyword  search  algorithms.  The  responses 
are  not  direct  answers  to  queries  but  rather  are  Web 
pages  and  documents  that  may  contain  the  answers. 
More  recently,  advances  in  computational  linguistics 
have  made  it  possible  for  users  to  parse  and  interpret 
well-formed  questions  and  for  answers  to  the  questions 
to  be  returned  (Harabagiu  et  al.,  2002;  Voorhees,  2001). 
The  information  repositories  have  varied  from  focal 
topics  (e.g.,  terrorism  and  finances  in  The  Wall  Street 
Journal)  to  open  searches  on  the  Web. 

Formal  evaluations  of  these  question  answering 
systems  have  been  conducted  with  respect  to  their 
accuracy  in  accessing  all  of  the  desired  information 
and,  importantly,  not  returning  irrelevant  information 
(see  informedia.cs.cmu.edu/aquaint/index.html).  The 
performance  of  these  query-based  information 
retrieval  systems  has  been  quite  impressive  for  short- 
answer  questions  (who,  what,  when,  where)  but  not 
for  questions  that  require  lengthy  answers  (why,  how) . 
For  the  latter  types  of  questions,  the  best  that  can  be 
accomplished  is  returning  a  paragraph  from  the  text 
that  might  contain  the  answer.  What  has  been  rare  in 
these  evaluations  of  question-and-answer  systems  is 
performance  in  the  context  of  LEs.  In  one  study,  the 
accuracy  of  fetching  paragraphs  was  quite  respectable 
in  an  LE  on  research  ethics;  95%  of  the  paragraphs 
were  judged  by  the  learner  as  being  relevant  and  50% 
as  being  informative  (Graesser  et  al.,  2004a).  More 
research  is  needed  to  assess  the  questions  that  students 
ask  during  learning  and  the  fidelity  of  the  answers 
delivered  by  the  question-and-answer  facilities  in  the 
learning  environments. 

One  challenge  that  limits  the  utility  of  query-based 
retrieval  systems  is  that  most  learners  ask  very  few 
questions,  and  most  of  the  questions  they  ask  are  shal¬ 
low  (Graesser  et  al.,  2005b;  Graesser  and  Person, 
1994).  Questions  are  typically  asked  when  learners 
experience  cognitive  disequilibrium  as  a  result  of 
obstacles  to  goals,  contradictions,  anomalous  informa¬ 
tion,  difficult  decisions,  and  salient  knowledge  gaps 
(Graesser  and  Olde,  2003).  Even  then,  though,  most 
learners  need  to  be  trained  on  how  to  ask  good  ques¬ 
tions.  Such  training  of  question-asking  skills  does 
improve  question  quality  and  also  comprehension 
(King,  1994;  Rosenshine  et  al.,  1996).  Learners  need 
to  be  exposed  to  good  models  of  question  asking, 
inquiry,  and  curiosity.  A  deep,  curious  learner  is  some¬ 
thing  rarely  seen  in  classrooms  and  the  real  world. 

A  different  sense  of  inquiry  learning  is  manifested 
in  LEs  that  stimulate  hypothetical  reasoning  and  the 
scientific  method,  such  as  Inquiry  Island  (White  and 
Frederiksen,  2005).  Learners  are  presented  with 
authentic  challenges  that  motivate  them  to  generate 
hypotheses,  plans  on  testing  hypotheses,  reports  to 


colleagues,  revisions  of  hypotheses,  and  so  on.  Ideally, 
the  learners  will  be  intrinsically  motivated  by  the  prob¬ 
lem  and  the  affordances  of  the  LE  to  the  point  of 
engaging  in  the  inquiry  process;  however,  it  is  neces¬ 
sary  to  investigate  the  process  of  scaffolding  effective 
inquiry  for  a  wide  range  of  learner  profiles.  Many  LEs 
fail  in  stimulating  genuine  inquiry  in  most  learners,  so 
this  is  an  area  greatly  in  need  of  research.  The  time 
course  of  learning  from  these  LEs  involves  weeks, 
months,  or  years  (hardly  one-hour  training  sessions), 
so  the  research  is  expensive  and  requires  several 
months  or  years  for  adequate  evaluations. 

Animated  Pedagogical  Agents  (Agents) 

Embodied  animated  conversational  agents  have 
become  very  popular  in  information  and  communica¬ 
tion  technologies,  but  the  most  serious  applications 
have  been  in  learning  technologies  (Atkinson,  2002; 
Baylor  and  Kim,  2005;  Cole  et  al.,  2003;  Graesser  et 
al.,  2005a;  Johnson  et  al.,  2000;  McNamara  et  al., 
2004;  Moreno  and  Mayer,  2004;  Reeves  and  Nass, 
1996).  These  agents  speak,  point,  gesture,  walk,  and 
exhibit  facial  expressions.  Some  are  built  in  the  image 
of  humans,  whereas  others  are  animals  or  cartoon  char¬ 
acters.  The  potential  power  of  these  agents,  from  the 
standpoint  of  LEs,  is  that  they  can  mimic  face-to-face 
communication  with  human  tutors,  instructors,  men¬ 
tors,  peers,  or  people  who  serve  other  roles.  Ensembles 
of  agents  can  model  social  interaction.  Single  agents 
can  model  individuals  with  different  knowledge,  per¬ 
sonalities,  physical  features,  and  styles.  Both  single 
agents  and  ensembles  of  agents  can  be  carefully  cho¬ 
reographed  to  mimic  virtually  any  social  situation: 
curious  learning,  negotiation,  interrogation,  argu¬ 
ments,  empathetic  support,  helping,  and  so  on.  As  a 
result,  agent  technologies  can  potentially  have  a  revo¬ 
lutionary  impact  on  social  science  research. 

Researchers  have  investigated  the  conditions  in 
which  single  agents  promote  learning  either  alone  or 
in  the  presence  of  other  media  (Mayer,  2005);  for 
example,  is  it  better  to  have  information  presented  in 
print  or  spoken  by  agents?  Are  realistic  agents  better 
than  cartoon  agents?  Does  the  attractiveness  or  con¬ 
versational  style  of  the  agent  matter?  These  and  other 
similar  questions  can  be  related  to  the  previous 
research  on  multimedia,  discourse,  and  social  psychol¬ 
ogy  that  was  conducted  before  the  onslaught  of  agent 
technologies.  It  is  of  course  important  to  make  sure 
that  the  agent  does  not  create  cognitive  overload,  a 
split  attention  effect,  or  a  distraction  from  other  infor¬ 
mation  on  the  display  that  has  higher  importance 
(Moreno  and  Mayer,  2004).  It  is  important  to  make 
sure  that  the  agent  is  not  so  realistic  that  the  learner 
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has  too  high  an  expectation  with  regard  to  its  intelli¬ 
gence  (Norman,  1994;  Shneiderman  and  Plaisant, 
2005).  Available  research  suggests  that  it  is  the  content 
of  what  is  expressed,  rather  than  the  aesthetic  quality 
of  the  speech  or  face,  that  is  most  important  in  pre¬ 
dicting  learning  (Graesser  et  al.,  2003b).  Research  also 
suggests  that  it  is  possible  to  create  social  presence 
from  facial  icons  with  expressions,  a  minimalist  form 
of  the  persona  effect. 

There  are  four  directions  that  merit  more  attention 
in  future  research.  First,  ensembles  of  agents  can  model 
learning  processes,  so  researchers  can  investigate  how 
learning  is  systematically  affected  by  different  theories 
of  social  interaction;  for  example,  dyads  can  exist 
between  peer  learners,  between  teacher  and  student,  or 
between  tutor  and  student,  as  well  as  triads  among 
teachers,  tutors,  and  peers  (McNamara  et  al.,  2004). 
Learners  can  learn  vicariously  from  such  interactions 
(Craig  et  al.,  2000).  The  possibilities  are  endless.  Sec¬ 
ond,  researchers  can  explore  the  processes  that  design¬ 
ers  and  learners  go  through  when  they  create  agents 
with  the  tool  kits  that  have  been  developed.  In  addition 
to  understanding  these  design  processes,  researchers 
will  accumulate  a  broader  population  of  agents  to  test 
in  their  studies  (i.e.,  beyond  Microsoft  agents),  includ¬ 
ing  those  with  diverse  physical  appearances,  personal¬ 
ities,  and  styles  that  resonate  with  specific  learner  pop¬ 
ulations  (Baylor  and  Kim,  2005).  Third,  researchers  can 
develop  agents  that  deeply  interpret  what  learners 
express  in  tutorial  dialog  or  other  forms  of  human- 
computer  interaction.  This  direction  requires  integra¬ 
tion  of  advances  from  computational  linguistics,  cog¬ 
nitive  science,  and  artificial  intelligence.  Fourth, 
researchers  can  investigate  alternative  ways  that  agents 
can  be  responsive  to  the  learner  as  the  learner  makes 
contributions  that  vary  in  quality.  This  is  already  being 
done  in  AutoTutor  (Graesser  et  al.,  2005a),  which  holds 
a  mixed  initiative  dialog  with  the  learner.  AutoTutor 
has  dialog  moves  that  are  responsive  to  the  learner’s 
knowledge  states:  short  feedback  (positive,  neutral, 
negative),  pumps  for  information  (“What  else?”),  hints, 
answers  to  learner  questions,  and  corrections  of  student 
misconceptions.  Similarly,  McNamara’s  iSTART  sys¬ 
tem  has  groups  of  agents  that  adaptively  respond  to 
learners  who  generate  self-explanations  while  reading 
science  texts  (McNamara  et  al.,  2004).  These  respon¬ 
sive  agents  require  more  intelligence  than  the  prepre¬ 
pared  choreographed  agents. 

Virtual  Environments  with  Agents 

Virtual  environments  are  getting  progressively  closer 
to  real-world  settings  and  therefore  should  have  excel¬ 
lent  transfer  to  the  real  world  but  at  costs  that  can  be 


much  more  economical  than  learning  in  field  settings. 
The  addition  of  agents  to  scaffold  learners’  interac¬ 
tions  with  these  virtual  realities  has  the  potential  to 
form  an  ideal  meshing  of  technology  and  pedagogy 
(Winn,  2003).  Outstanding  examples  of  virtual  envi¬ 
ronments  with  agents  are  those  developed  at  the  Uni¬ 
versity  of  Southern  California,  namely  Mission 
Rehearsal  (Gratch  et  al.,  2002)  and  Tactical  Iraqi 
(Johnson  and  Beal,  2005).  These  virtual  worlds  are 
very  close  to  authentic  interactions  in  war  scenarios 
or  when  soldiers  interact  with  citizens  in  another  cul¬ 
ture  with  a  different  language.  The  learner  holds  a 
dialog  in  natural  language,  with  speech  recognition 
and  multiple  agents.  These  award-winning  virtual 
environments  are  major  milestones  and  have  involved 
major  investments  by  the  military.  Unfortunately, 
these  learning  environments  are  not  currently  on  the 
Web,  so  the  feasibility  of  transporting  simpler  ver¬ 
sions  on  simpler  platforms  remains  a  question.  More 
modest  virtual  environments  with  agents  are  available 
in  MOOs  (Slator  et  al.,  2004).  As  mentioned  earlier, 
there  is  a  glaring  deficit  in  the  literature  when  it  comes 
to  evaluating  the  impact  of  these  virtual  environments 
on  learners  (Winn,  2003).  More  empirical  evaluations 
of  these  systems  on  learning  gains,  usability,  learner 
impressions,  and  the  fidelity  of  specific  computational 
modules  are  necessary. 

Serious  Games 

The  game  industry  has  certainly  captured  the  imagi¬ 
nation  of  this  generation  of  young  adults,  with  reve¬ 
nues  larger  than  the  movie  industry.  Serious  teenage 
gamers  play  games  approximately  20  hours  per  week 
(Yee,  2006).  Nearly  all  of  the  rich  taxonomy  of  games 
could  be  integrated  with  education  and  training  (e.g., 
first-person  shooter  games,  multiparty  games,  simula¬ 
tions  of  cities).  A  large-scale  game  such  as  America’s 
Army  is  extremely  appealing  to  both  the  young  and 
old  because  it  is  engaging  and  simultaneously  weaves 
in  serious  content  about  the  Army.  The  challenge  of 
combining  entertainment  and  pedagogical  content  is 
the  foundational  question  of  serious  games  (Brody, 
1993).  Social  scientists  need  to  investigate  the  mech¬ 
anisms  necessary  for  this  to  succeed. 

The  components  of  games  have  been  analyzed  at 
considerable  depth  (Gee,  2003;  Salen  and  Zimmer¬ 
man,  2004),  but  more  research  is  needed  on  the  impact 
of  these  components  on  learning  gains,  engagement, 
and  usability  (Cameron  and  Dwyer,  2005;  Conati, 
2002;  Lawrence,  2004;  Malone  and  Lepper,  1987; 
Moreno  and  Mayer,  2005;  Virvou  et  al.,  2005).  Pre¬ 
sumably,  the  success  of  a  game  can  be  attributed  to 
such  factors  as  feedback,  progress  markers,  engaging 
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content,  fantasy,  competition,  challenge,  uncertainty, 
curiosity,  control,  and  other  factors  that  involve  cog¬ 
nition,  emotions,  motivation,  and  art.  Investigating  the 
relationships  between  game  features  and  outcome 
measures  should  be  an  important  priority  for  future 
researchers  because  scientific  data  are  sparse  and  the 
impact  of  games  on  society  is  enormous. 

An  adequate  understanding  of  games  requires 
research  on  the  relations  between  emotions  and  learn¬ 
ing.  Connections  between  emotions  and  complex 
learning  are  receiving  more  attention  in  the  fields  of 
psychology  (Dweck,  2002;  Lepper  and  Henderlong, 
2000),  education  (Meyer  and  Turner,  2006),  neuro¬ 
science  (Damasio,  2003),  and  computer  science  (Kort 
et  al.,  2001;  Picard,  1997).  Some  of  the  recent  intelli¬ 
gent  tutoring  systems  automatically  infer  and  track 
student  emotions  and  motivational  states  on  the  basis 
of  the  tutorial  interaction,  facial  expressions,  speech 
intonation,  posture,  and  other  communication  channels 
(De  Vicente  and  Pain,  2002;  D’Mello  et  al.,  2005; 
Graesser  et  al.,  2006;  Litman  and  Forbes-Riley,  2004). 
Pervasive  affective  states  during  complex  learning 
include  confusion, boredom,  flow/engagement,  curios¬ 
ity/interest,  delight/eureka,  and  frustration  from  being 
stuck  (Burleson  and  Picard,  2004;  Craig  et  al.,  2004; 
Csikszentmihalyi,  1990;  Graesser  et  al.,  2006;  Kort  et 
al.,  2001). 

Meyer  and  Turner  (2006)  identified  three  major 
theories  that  link  emotions  and  learning,  which  they 
refer  to  as  academic  risk  taking,  flow  theory,  and  goal 
theory.  The  academic  risk  theory  contrasts  students 
who  like  to  be  challenged  with  difficult  tasks,  take  risks 
of  failure,  and  manage  negative  emotions  with  those 
students  who  tackle  easier  tasks,  take  fewer  risks,  and 
minimize  learning  situations  where  they  fail  and  expe¬ 
rience  negative  emotions.  According  to  flow  theory, 
the  learner  is  in  a  state  of  flow  (Csikszentmihalyi, 
1990)  when  the  learner  is  so  deeply  engaged  in  learn¬ 
ing  the  material  that  time  and  fatigue  disappear.  Met¬ 
calfe  and  Kornell  (2005)  predict  that  the  flow  experi¬ 
ence  is  believed  to  occur  when  the  learning  rate  is  high 
and  the  learner  has  achieved  a  high  level  of  mastery 
at  the  region  of  proximal  learning.  Goal  theory  empha¬ 
sizes  the  role  of  goals  in  predicting  emotions.  Out¬ 
comes  that  achieve  goals  result  in  positive  emotions, 
whereas  outcomes  that  jeopardize  goal  accomplish¬ 
ment  result  in  negative  emotions  (Dweck,  2002). 
Obstacles  to  goals  are  particularly  diagnostic  of  both 
learning  and  emotions.  The  affective  state  of  confusion 
correlates  with  learning  gains  perhaps  because  it  is  a 
direct  reflection  of  deep  thinking  (Craig  et  al.,  2004; 
Guhe  et  al.,  2004).  Confusion  is  diagnostic  of  cognitive 
disequilibrium,  a  state  that  occurs  when  learners  face 
obstacles  to  goals,  contradictions,  incongruities,  anom¬ 


alies,  uncertainty,  and  salient  contrasts  (Graesser  et  al., 
2006).  Cognitive  equilibrium  is  restored  after  thought, 
reflection,  problem  solving,  and  other  effortful  delib¬ 
erations. 

An  emotion-sensitive  tutor  would  presumably 
enhance  intelligent  learning  environments  (D’Mello  et 
al.,  2005;  Graesser  et  al.,  2006;  Lepper  and  Hender¬ 
long,  2000).  If  the  learner  is  frustrated,  for  example, 
the  tutor  would  generate  hints  to  advance  the  learner 
in  constructing  knowledge  or  would  make  supportive 
empathetic  comments  to  enhance  motivation.  If  the 
learner  is  bored,  the  tutor  would  present  more  engaging 
or  challenging  problems  for  the  learner  to  work  on. 
Suitable  adaptive  moves  of  the  tutor  would  be  desired 
for  learners  who  experience  confusion,  delight,  sur¬ 
prise,  and  so  on.  One  of  the  research  frontiers  of  the 
future  is  to  discover  optimal  links  between  student 
emotions  and  learning. 

Computer-Supported  Collaborative  Learning 

In  computer-supported  collaborative  learning 
(CSCL),  groups  of  learners  collaboratively  construct 
knowledge  on  a  topic  in  pursuit  of  project  goals  that 
typically  are  provided  by  instructors  (Lee  et  al., 
2006);  for  example,  in  Knowledge  Forum  (Bereiter, 
2002;  Scardamalia  and  Bereiter,  1994)  students  cre¬ 
ate  messages  that  others  can  review,  elaborate,  cri¬ 
tique,  and  build  on.  CSCL  systems  support  threads 
of  conversations  that  involve  formulating  arguments, 
problem  solving,  planning,  report  writing,  and  count¬ 
less  other  tasks  (Gunawardena  et  al.,  1997).  Because 
the  length  of  most  of  these  conversational  threads  is 
short  (2.2  to  2.7  turns  per  thread)  (Hewitt,  2005), 
attempts  have  been  made  to  design  these  systems  to 
lengthen  the  threads.  Some  evidence  suggests  that 
CSCL  facilitates  deeper  learning,  critical  thinking, 
shared  understanding,  and  long-term  retention  (Garri¬ 
son  et  al.,  2001;  Johnson  and  Johnson,  1991),  but  the 
scale  of  these  distributed  learning  environments  makes 
it  very  difficult  to  perform  systematic  evaluations  of 
them. 

Social  and  behavioral  scientists  have  the  founda¬ 
tion  necessary  to  step  in  and  improve  these  CSCL 
systems  in  several  ways  (Clark  and  Brennan,  1991; 
Dillenbourg  and  Traum,  2006;  Looi,  2005;  Mazur, 
2004;  Soller  et  al.,  1998;  Wang,  2005).  How  do  learn¬ 
ers  figure  out  how  to  use  the  complex  interfaces  on 
these  multiparty  computer-mediated  communication 
systems?  How  does  a  potential  contributor  learn  how 
and  when  to  speak?  How  is  knowledge  grounded  in 
these  distributed  systems?  How  can  moderators  guide 
the  group  of  learners  in  productive  directions? 
Research  on  CSCL  is  very  much  an  open  frontier. 
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CONCLUDING  REMARKS 

Many  are  convinced  that  learning  gains  from  technol¬ 
ogies  are  best  attributed  to  the  underlying  pedagogies 
rather  than  the  technologies  per  se.  At  the  same  time, 
we  all  recognize  that  various  technologies  afford  dif¬ 
ferent  pedagogies.  One  of  the  key  challenges  for  the 
future  is  to  explore  how  technologies  and  pedagogies 
can  be  effectively  coordinated  in  a  fashion  that  con¬ 
siders  the  learning  goals  and  the  learners’  profiles.  The 
TMGL  landscape  has  a  large  space  of  empty  cells  that 
will  potentially  guide  researchers  for  many  decades. 
Progress  will  only  be  achieved  when  social,  cognitive, 
and  behavioral  scientists  are  part  of  an  interdiscipli¬ 
nary  team  of  LE  designers,  developers,  and  deliverers. 

The  LEs  significantly  vary  in  development  costs, 
of  course.  We  have  periodically  been  asked  how  expen¬ 
sive  it  is  to  develop  and  test  these  different  types  of 
learning  technologies,  so  we  have  generated  some  ball¬ 
park  estimates  of  such  costs.  Our  estimates  should  be 
considered  with  caution  because  other  experts  may 
have  very  different  viewpoints,  and  there  is  wide  vari¬ 
ance  in  these  costs  depending  on  detailed  parameters 
of  the  relevant  cells  in  the  TMGL  landscape.  Approx¬ 
imate  costs  are  $10,000  for  an  hour  training  session 
with  conventional  computer-based  training;  $100,000 
for  a  10-hour  course  with  conventional  computer-based 
training  and  rudimentary  multimedia;  $1,000,000  for 
an  information-rich,  hypertext/hypermedia  system; 
$10,000,000  for  an  intelligent  tutoring  system  without 
the  benefit  of  a  suite  of  authoring  tools  and  utilities; 
and  $100,000,000  for  a  serious  game  (not  a  frivolous 
commercial  game)  on  the  Web  for  thousands  of  users. 

Given  that  training  systems  have  costs  and  that 
some  have  nontrivial  costs,  one  mission  has  been  to 
find  ways  to  cut  the  price,  development  time,  and  other 
resources  associated  with  building  these  systems  with¬ 
out  sacrificing  the  quality  of  the  learning  experience. 
This  mission  has  been  pursued  by  the  Advanced  Dis¬ 
tributed  Learning  initiative  (for  example,  see  www.adl- 
net.org;  Dodds  and  Fletcher,  2004;  Duval  et  al.,  2004; 
Fletcher,  2003).  Learning  content  is  standardized  by 
being  decomposed,  packaged,  and  organized  into 
learning  objects  that  have  to  conform  to  the  standards 
of  the  Sharable  Content  Object  Reference  Model 
(SCORM).  Each  learning  object  has  meta-tags  that 
identify  the  relevant  contexts  of  its  application.  A 
SCORM-conformant  learning  object  can  be  used  in 
most  learning  management  systems,  so  the  content  is 
sharable,  interoperable,  reusable,  and  extendable.  This 
represents  a  substantial  savings  in  costs:  Once  content 
is  created  in  a  SCORM-conformant  fashion,  it  can  be 
used  throughout  the  e-leaming  world.  One  of  the  chief 
challenges  now  is  to  get  designers  of  courseware  to 


reuse  the  SCORM-conformant  learning  objects  (Brusi¬ 
lovsky  and  Nijhawan,  2002;  Sampson  and  Karampip- 
eris,  2006).  This  is  accomplished  by  building  and 
indexing  large  repositories  of  SCORM-conformant 
content,  as  in  the  case  of  CORDRA  (Rehak,  2005; 
http://cordra.net),  and  to  somehow  market  and  encour¬ 
age  such  repositories  to  be  used.  A  second  major  chal¬ 
lenge  is  to  develop  SCORM  standards  for  the  more 
advanced  LEs  now  that  SCORM  is  mainstream  for 
computer-based  training  and  most  multimedia. 

Other  ways  to  reduce  the  costs  of  building  LEs  are 
available— for  example,  authoring  tools  for  the  easy 
preparation  of  course  content  for  CBT  and  multimedia. 
Better  authoring  tools  are  necessary,  however,  to  build 
new  course  content  with  the  more  advanced  LEs  (Mur¬ 
ray  et  al.,  2003).  The  existing  authoring  tools  for 
advanced  systems  are  very  difficult  to  learn  and  use; 
these  authoring  tools  are  so  complex  that  only  the  most 
advanced  cognitive  scientists  and  computer  scientists 
can  use  them— often  only  the  original  designers  of  the 
systems.  To  allow  these  tools  to  be  more  widely  used 
by  individuals  with  varying  backgrounds,  it  will  be 
necessary  to  conduct  systematic  research  on  human 
factors  and  human-computer  interaction  that  can  be 
applied  to  developing  course  content  with  authoring 
tools;  otherwise,  it  is  difficult  to  see  how  these  advanced 
systems  will  scale  up  to  handle  the  large  volume  of 
training  needs  throughout  the  military,  for  example. 

A  salient  side  benefit  of  the  authoring  tools  is  that 
they  can  be  viewed  as  LEs  themselves;  that  is,  learning 
a  complex  system  at  a  deep  level  can  be  achieved  by 
building  an  advanced  LE  on  the  system  with  an  author¬ 
ing  tool.  One  of  the  exciting  developments  of  the  future 
is  to  explore  the  process  of  learning  deep  content  and 
strategies  by  creating  learning  environments  with  these 
authoring  tools.  This  not  only  will  be  engaging  to  many 
students  but  will  also  empower  them  for  handling  the 
technologies  of  the  future. 
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ABSTRACT 

This  chapter  reports  recent  research  on  using  text-based 
computer-mediated  communication  (CMC)  in  educa¬ 
tional  settings.  CMC  takes  many  forms,  and  the  tools 
available  are  many.  An  outline  of  CMC  features  and 
their  affordances  is  presented  as  a  means  to  draw  the 
attention  of  those  designing,  evaluating,  or  describing 
communication  environments  to  the  features— techni¬ 
cal  and  social— that  are  the  essence  of  the  activity  and 


away  from  the  particular  applications  that  provide 
them.  The  goals  of  research  using  CMC  are  widely 
varied.  Some  research  focuses  on  how  a  particular 
activity  is  connected  to  a  measure  of  learning.  Other 
research  focuses  on  the  nature  of  CMC  and  how  various 
forms  differ  from  one  another.  With  the  tacit  assump¬ 
tions  that  people  learn  from  thoughtful  evaluation  and 
respond  to  the  ideas  of  others,  some  research  looks  at 
ways  to  increase  participation  or  guide  learners  to  com¬ 
municate  in  particular  ways.  Because  CMC  is  not 
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affected  by  time  or  place,  it  is  also  used  as  a  means 
for  teacher  training  and  reduction  of  the  isolation  that 
many  educators  feel. 


KEYWORDS 

Blog:  From  the  term  “Web  log”;  originally  a  Web- 
based  diary  but  often  now  used  to  refer  to  the 
software  designed  to  support  it  which  presents  Web 
pages  in  a  reverse  chronological  order. 

Computer-mediated  communication  (CMC):  Commu¬ 
nication  between  two  or  more  individuals  with  text- 
based  tools  such  as  e-mail,  instant  messaging,  or 
computer-based  conferencing  systems. 

Learning  management  system  (LMS):  A  computer- 
based  system  to  support  or  replace  classroom- 
based  learning;  examples  include  Blackboard, 
Moodle,  Sakai,  and  WebCT. 


INTRODUCTION  AND  SCOPE 

A  widening  array  of  communications  technologies  can 
be  found  in  education  today.  In  addition  to  the  ubiq¬ 
uitous  Internet  and  its  set  of  general-purpose  commu¬ 
nication  tools,  learning  management  systems  (LMSs) 
such  as  Blackboard,  Moodle,  and  Sakai  are  also 
increasingly  prevalent,  so  the  question  educators  ask 
is  not  whether  or  not  to  use  these  tools,  but  how.  This 
chapter  explores  different  types  of  communication 
technologies,  their  affordances,  and  their  uses;  the 
goals  of  research  using  these  tools;  and  areas  for  fur¬ 
ther  research. 


TECHNOLOGY  AFFORDANCES 

Historically,  new  media  have  first  been  used  as  a  means 
to  replace  some  other  medium;  for  example,  the  most 
obvious  educational  use  of  motion  pictures  and  later 
television  was  to  allow  more  people  to  enjoy  the  ben¬ 
efits  of  attending  high-quality  lectures.  A  series  of  stud¬ 
ies  follows  to  determine  whether  the  new  medium  is 
working  as  well  as  the  one  it  has  replaced  (Cuban, 
1986).  One  argument  is  that  media  are  merely  delivery 
vehicles,  and  it  does  not  matter  what  kind  of  truck 
delivers  your  groceries;  the  other  perspective  is  that  we 
should  look  instead  at  the  affordances  of  each  medium 
and  how  to  exploit  them  (Clark,  1983,  1994;  Kozma, 
1994).  Russel  (2001),  and  the  companion  website,  pro¬ 
vide  an  extensive  annotated  bibliography  of  media 
comparison  studies  as  evidence  that  distance  education 
can  be  as  effective  as  face-to-face  instruction.  These 


issues  are  further  discussed  in  the  second  edition  of 
this  volume  (Romiszowski  and  Mason,  2004).  This 
section  focuses  on  particular  features  of  CMC  and  their 
affordances  and  is  intended  to  help  those  designing  or 
reporting  CMC  research  to  focus  on  the  activity  rather 
than  the  particular  tools  used.  It  is  more  informative  to 
say,  for  example,  that  participants  used  threaded  dis¬ 
cussion  than  to  list  a  specific  tool  that  may  have  dozens 
of  features  unused  in  the  activity  reported. 

Text  Handling 

Computer-mediated  communication  text-handling  fea¬ 
tures  can  be  divided  into  synchrony,  retrieval,  and  stor¬ 
age.  Synchrony,  whether  the  communication  is  real 
time  or  asynchronous,  has  a  profound  effect  on  the 
nature  of  a  conversation.  Honeycutt  (2001)  tested  a 
theory  of  grounding  proposed  by  Clark  and  Brennan 
(1991).  Different  media  have  different  costs— for 
speakers,  formulation  and  production;  for  addressees, 
reception  and  understanding;  for  both,  delay,  fault,  and 
repair.  As  an  example,  the  immediacy  of  feedback 
makes  the  costs  to  repair  a  faulty  understanding  in  a 
face-to-face  discussion  relatively  low.  Honeycutt 
assigned  students  into  groups  of  three;  each  group  used 
chat  to  discuss  writing  revisions  on  one  assignment 
and  e-mail  in  another  assignment  in  a  2-hour  class. 
Content  analysis  showed  that,  when  using  e-mail,  stu¬ 
dents  made  significantly  more  references  to  docu¬ 
ments,  their  contents,  and  rhetorical  contexts  than 
when  using  chat.  Using  synchronous  chats,  students 
made  greater  reference  to  both  writing  and  response 
tasks  than  when  using  e-mail. 

In  a  similar  design,  Davidson-Shivers  et  al.  (2001) 
randomly  assigned  groups  of  graduate  students  to  com¬ 
municate  synchronously  (chat)  or  asynchronously 
(threaded  discussion)  for  one  week;  they  then  switched 
to  the  other  tool  in  a  subsequent  week  for  a  different 
topic.  Though  students  reported  liking  both  types  of 
discussion,  students  especially  liked  the  convenience 
of  asynchronous  threaded  discussion.  Results  showed 
that  students  using  chat  were  more  likely  to  respond 
or  react  to  other  messages  than  during  the  threaded 
discussion. 

Although  infrequently  studied  explicitly,  another 
significant  feature  in  text  handling  is  whether  the  infor¬ 
mation  is  pushed  to  the  reader,  as  with  e-mail,  or  must 
be  sought  out,  as  with  threaded  discussion.  As  the 
amount  of  electronic  communication  continues  to 
expand,  tools  for  managing  its  flow  will  become 
increasingly  important. 

What  happens  after  the  communication  is 
received— how  the  application  archives  messages  — 
not  only  affects  how  people  communicate  but  is  also 
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critical  to  those  who  study  communication.  The  need 
for  archiving  sometimes  outweighs  the  type  of  com¬ 
munication  researchers  set  out  to  study.  Tidwell  and 
Walther  (2002)  used  e-mail  to  function  as  an  instant 
messaging  tool  because  e-mail  facilitated  recording 
the  data  that  they  wanted  to  analyze.  Robust  archival 
mechanisms  can  work  to  the  advantage  of  both  the 
discussant  and  the  researcher;  threaded  discussion 
forums  provide  good  support  for  those  communicat¬ 
ing  and  keep  a  complete  record  of  the  communication 
for  researchers.  Threaded  discussion  makes  it  easier 
for  users  to  follow  a  conversation,  and,  because  it  can 
be  calculated  mechanically,  for  researchers  thread 
length  is  a  convenient  variable  to  track.  As  the  num¬ 
ber  of  messages  in  an  online  conversation  increases, 
especially  for  many  speakers,  it  becomes  more 
important  to  have  a  means  to  track  which  messages 
have  been  read  already  and  to  follow  the  thread  of 
the  conversation. 

Participant  Selection 

Another  feature  of  CMC  systems  is  how  easily  partic¬ 
ipants  are  selected  and  how  easily  these  groupings  are 
maintained.  With  e-mail,  the  writer  must  make  a  con¬ 
scious  decision  regarding  who  to  send  a  message  to; 
with  threaded  discussion,  the  writer  has  no  control  or 
responsibility  over  the  recipient  list.  When  publishing 
on  the  Web,  the  writer  typically  cannot  know  who  will 
read  what  is  written.  Barton  (2005)  argues  that  the 
open  nature  of  blogs  and  other  Web-based  discussion 
environments  are  a  means  to  help  teach  students  the 
importance  of  public  discussion  to  college  composi¬ 
tion  courses.  Richardson  (2003)  described  how  his 
high-school  students  used  blogs  to  discuss  a  book  not 
only  with  each  other  but  also  with  the  author.  Public 
e-mail  lists  have  been  shown  to  reduce  isolation  among 
physical  education  educators,  a  finding  that  is  likely 
replicable  with  many  educators  (Pennington  and  Gra¬ 
ham,  2002). 

An  interesting  result  of  the  openness  of  chat  rooms 
and  discussions  on  the  Internet  is  that  they  allow 
researchers  access  to  groups  that  are  otherwise  difficult 
to  study.  A  common  methodology,  especially  among 
those  interested  in  public  health  issues  surrounding 
HIV/AIDS ,  is  to  survey  members  of  online  groups  to 
learn  more  about  their  practices  as  a  means  of  evalu¬ 
ating  those  forums  as  tools  for  public  health  education 
(Tikkanen  and  Ross,  2003;  Wang  and  Ross,  2002). 

Open  groups  are  susceptible  to  unwanted  commu¬ 
nication.  Whitney  (2004)  documents  how  jESSE,  a 
discussion  list  for  library  and  information  science  edu¬ 
cation,  went  from  being  a  valuable  resource  to  an 
unusable  one  as  the  percentage  of  unsolicited  commer¬ 


cial  e-mail  (spam)  climbed  from  13%  to  75%.  Perhaps 
even  more  insidious  than  spam  are  participants  who 
join  a  group  expressly  to  disrupt  the  conversation 
(Doostdar,  2004) . 

Identity  and  Voice 

Perhaps  the  most  studied  aspect  of  CMC  is  how  it 
differs  from  face-to-face  conversation.  Communica¬ 
tion  patterns  are  also  affected  by  whether  participants’ 
identities  are  known,  anonymous,  or  pseudonymous. 
These  variables  have  dramatic  impacts  on  how  people 
communicate  but  are  sometimes  not  explicitly  consid¬ 
ered  in  research  designs  or  their  evaluation. 

Subrahmanyam  et  al.  (2004)  were  interested  in 
studying  developmental  issues  of  sexuality  and  iden¬ 
tity  and  found  that  anonymity  allowed  girls  in  an 
adult -monitored  teen  chat  room  to  be  sexually  aggres¬ 
sive  in  ways  that  are  uncommon  in  environments  that 
are  not  anonymous.  Panyametheekul  and  Herring 
(2003)  reported  similar  findings  regarding  female 
empowerment  in  a  Thai  chat  room.  In  both  cases,  the 
social  mediator’s  anonymity,  rather  than  the  technical 
affordances  of  the  medium,  is  what  most  affects  the 
discussion.  Group  conversations  in  large  classes 
where  not  all  students  know  each  other  seem  to  main¬ 
tain  some  of  these  characteristics  of  freer  expression 
(Collins,  1996). 

Studying  how  relationships  are  formed  face  to  face 
vs.  in  CMC,  Tidwell  and  Walther  (2002)  assigned 
mixed-gender  pairs  using  either  CMC  or  face-to-face 
discussion  to  either  ( 1 )  get  to  know  each  other,  or  (2) 
work  on  a  decision-making  problem.  Results  showed 
that  those  in  the  CMC  group  were  more  likely  to  elicit 
and  disclose  personal  information.  Not  explicitly  dis¬ 
cussed  in  the  reporting  is  that,  because  they  used  e-mail 
accounts  set  up  especially  for  the  study,  the  CMC 
participants  maintained  their  anonymity  while  those  in 
the  face-to-face  condition  did  not.  This  additional  vari¬ 
able  likely  not  only  contributed  to  the  participants’ 
sharing  more  personal  information  but  also  increased 
the  need  for  requesting  some  personal  information.  As 
O’Connor  and  Ross  (2004)  pointed  out,  a  common 
question  in  chat  rooms  is  a/s/l  (age,  sex,  location),  but 
such  a  question  is  superfluous  or  rude  in  most  face-to- 
face  contexts.  Anonymity  is  a  major  social  factor 
affecting  how  we  communicate. 

In  activities  where  anonymity  is  not  an  issue, 
online  chat  can  provide  affordances  similar  to  those 
for  face-to-face  chat;  for  example,  online  dyads  work¬ 
ing  collaboratively  have  been  found  to  perform  better 
than  their  counterparts  working  alone  (Uribe  et  al., 
2003),  just  as  is  the  case  in  face-to-face  dyads 
(Schwartz,  1995).  Jensen  et  al.  (2002)  allowed  students 
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to  chat  while  taking  online  tests.  Another  group  who 
got  random  members’  grades  rather  than  their  own  was 
significantly  more  likely  to  use  this  tool  to  help  see 
that  everyone  did  well  on  the  test. 

Group  Text  Production 

A  new  type  of  communication  that  exists  primarily 
online  is  the  concept  of  group  communication,  made 
possible  by  wikis,  which  are  Web-based  applications 
that  allow  any  user  to  edit  any  page  (although  more 
elaborate  permission  schemes  exist).  Wikipedia.org  is 
an  encyclopedia  with  over  a  million  pages  that  is  pro¬ 
duced  virtually  anonymously  from  the  perspective  of 
most  readers,  who  are  interested  primarily  in  the  infor¬ 
mation  presented,  not  in  who  produced  it.  A  problem 
with  group  work  is  that  it  is  difficult  to  know  to  whom 
to  attribute  various  contributions.  Because  wikis  can 
track  who  makes  changes  to  a  document,  wikis  might 
enable  groups  to  work  together  on  a  project  while  still 
maintaining  some  ownership  of  their  individual  con¬ 
tributions;  therefore,  wikis  are  a  tool  that  may  be  very 
useful  in  tracking  how  groups  work  together.  The 
effects  of  wikis  on  group  performance  are  a  topic  for 
further  research.  At  least  one  case  study  suggests  that 
wikis  can  support  collaborative  knowledge  creation 
and  sharing  (Raman  et  al.,  2005). 


GOALS  OF  RESEARCH 

The  goals  of  research  in  computer-mediated  communi¬ 
cation  are  varied.  Arguably  all  educational  media 
research  is  concerned  with  learning,  but  some  research 
has  focused  more  explicitly  on  measures  of  learning. 
Another  goal  is  to  study  the  effects  of  using  the  tech¬ 
nology  (e.g.,  a  test  or  artifact)  and  analyze  ways  in 
which  student  communications  change  over  time.  Some 
research  focuses  on  how  to  encourage  participants  to 
embrace  a  new  technology— for  example,  a  case  study 
of  how  to  increase  the  use  of  a  learning  management 
system  (Bell  and  Bell,  2005).  Another  body  of  research 
covered  here  is  concerned  with  teacher  professional 
development.  Because  these  communication  tools  are 
often  used  outside  of  the  schools,  they  are  sometimes 
used,  especially  by  social  scientists,  as  a  source  of  data; 
chat  rooms,  for  example,  have  been  used  to  study  the 
communities  of  practice  of  various  intact  groups. 

Learning 

Research  discussed  in  this  section  is  focused  explicitly 
on  connecting  some  type  of  communication  with  evi¬ 
dence  of  learning— via  changes  in  the  nature  of  dis¬ 


cussion  itself  or  with  regard  to  some  other  factor,  such 
as  a  class  grade  or  other  measure  of  learning.  Wang  et 
al.  (2001)  looked  at  synchronous  discussion  in  a  sta¬ 
tistical  methods  of  psychology  course  and  found  that 
the  number  of  comments  correlated  strongly  with 
learning  as  measured  by  students’  grades.  Discourse 
analysis  of  a  week  of  discussion  indicated  that  the 
number  of  responses  to  questions  about  problems  or 
examples  given  in  lecture  was  also  a  predictor  of  the 
course  grade. 

Krentler  and  Willis-Flurry  (2005)  used  asynchro¬ 
nous  discussion  with  six  sections  of  an  undergraduate 
course  in  marketing.  The  number  of  posts  that  con¬ 
tained  at  least  five  sentences  expressing  a  coherent 
thought  based  on  theory  rather  than  opinion  was  used 
as  a  measure  of  technology  use,  and  the  students’ 
grades  on  three  exams  and  a  project  were  used  as  a 
measure  of  learning.  Increased  use  of  the  discussion 
board  was  a  reliable  predictor  of  students’  grades. 

Weinberger  et  al.  (2005)  asked  students  to  analyze 
a  case  involving  attribution  theory.  Learning  was  mea¬ 
sured  by  expert  evaluation  of  students’  case  analyses. 
This  experimental  design  crossed  two  independent 
variables:  use  of  (1)  an  epistemic  script  and  (2)  a  social 
script.  Epistemic  scripts  are  designed  to  prestructure 
the  learning  task  to  encourage  knowledge  construction 
activities;  note  starters  are  an  example  of  epistemic 
scripts.  Social  scripts  are  those  that  prescribe  a  struc¬ 
ture  for  interaction,  perhaps  by  assigning  roles  for  par¬ 
ticipation;  a  well-known  example  is  reciprocal  teaching 
(Brown  and  Palincsar,  1989).  Learning  was  measured 
before  and  after  the  intervention  by  having  students 
evaluate  a  case.  Results  showed  that  in  the  context  of 
this  threaded  discussion  social  scripts  resulted  in  better 
learning  results  and  epistemic  scripts  did  not. 

Although  millions  of  people  blog  regularly  outside 
of  classrooms,  making  a  discussion  tool  available  to  a 
class  of  students  is  usually  not  enough  to  get  them  to 
use  it.  Sometimes  online  discussion  must  be  a  course 
requirement  before  students  will  take  to  it  (Khan, 
2005).  Requiring  participation  is  often  not  enough; 
instead,  it  is  often  more  effective  to  model  or  prescribe 
how  to  participate  (Bryant,  2005)  and  integrate  the 
discussion  into  the  pedagogical  approach  (Massimo, 
2003).  Using  course  grades  as  a  measure  of  learning, 
Ellis  and  Calvo  (2004)  found  that,  regardless  of 
medium,  students  interested  in  deep,  as  opposed  to 
superficial,  learning  learned  more.  Cluster  analysis  of 
surveys  of  Australian  engineering  students  identified 
two  groups,  one  that  used  discussion  to  understand 
the  subject  more  deeply  (about  a  third  of  the  partic¬ 
ipants)  and  another  that  engaged  only  superficially 
(the  remaining  two  thirds).  The  deep-learning  group 
also  reported  treating  face-to-face  discussion  more 
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meaningfully.  Other  techniques  to  better  connect  CMC 
with  learning  include  an  understanding  of  language 
and  literacy  (Mitchell  and  Erickson,  2004),  mentoring 
and  taxonomy  (Brescia,  2003),  and  scaffolding  cogni¬ 
tive  engagement  (Oriogun  et  al.,  2005). 

The  Nature  of  Communication 

Many  who  study  computer-mediated  communication 
are  interested  in  the  nature  of  communication,  so  often 
they  study  the  communication  itself  or  are  concerned 
with  ways  to  make  certain  kinds  of  communication 
patterns  more  frequent.  As  an  example,  they  might  use 
CMC  to  improve  relationships  and  foster  closeness 
(O’Sullivan  et  al.,  2004)  or  to  encourage  reflection 
(Greene,  2005;  Maor,  2003;  Nicholson  and  Bond, 
2003) .  Others  have  studied  the  various  patterns  of  com¬ 
munication  that  occur  in  CMC  (Jeong,  2003). 

Guiding  Discussion  with  CMC 

An  unstated  assumption  in  some  CMC  research  is  that 
discussion  is  an  indicator  of  learning;  consequently,  a 
common  theme  in  this  research  is  how  to  engender 
discussion,  and  how  to  encourage  particular  types  of 
communication.  Gilbert  and  Dabbagh  (2005)  found 
that  providing  exemplars  for  discussion  to  facilitators, 
posting  discussion  protocols,  and  offering  evaluation 
rubrics  all  increased  meaningful  discourse  with  grad¬ 
uate  students  in  education.  Working  with  undergradu¬ 
ates  in  psychology,  Nussbaum  et  al.  (2004)  found  that 
conversation  starters  increased  the  frequency  of  dis¬ 
agreements  (the  authors  assumed  that  disagreement  is 
a  positive  indicator  of  deeper  thinking)  and  encouraged 
students  to  consider  alternative  points  of  view.  Corre¬ 
lating  these  data  with  a  personality  survey  suggests 
that  this  technique  is  most  effective  with  students  who 
are  less  open  to  new  ideas  and  are  not  highly  anxious . 
Those  interested  in  note  starters  should  refer  to  Com¬ 
puter-Supported  Intentional  Learning  Environment 
(CSILE)  (Scardamalia  and  Bereiter,  1994),  which  later 
became  the  Knowledge  Forum.  When  Moore  and 
Marra  (2005)  investigated  students’  co-construction  of 
knowledge,  they  set  up  two  sections  of  instructional 
design  graduate  students  who  used  asynchronous  dis¬ 
cussion.  One  group  was  instructed  to  use  a  constructive 
argumentation  approach;  the  other  was  not.  Content 
analysis  showed  that  both  groups  co-constructed 
knowledge,  but  the  group  without  the  constraints 
attained  a  higher  level  of  knowledge  building. 

Mazzolini  and  Maddison  (2003)  investigated  how 
instructor  participation  affected  the  frequency  of  stu¬ 
dents’  posting,  lengths  of  threads,  as  well  as  self-report 
of  satisfaction  and  other  affective  variables  in  an 


undergraduate  course  in  astronomy.  Results  showed 
that  more  frequent  instructor  postings  did  not  affect 
the  frequency  of  student  posts  but  did  reduce  the  length 
of  threads.  Professors  who  posted  more  often  were 
judged  to  be  more  enthusiastic  and  knowledgeable 
than  those  who  posted  less  frequently. 

CMC  as  Change  Agent 

Some  researchers  use  technology  as  a  fulcrum  to  push 
reform.  Sometimes  merely  adding  online  discussion  to 
a  face-to-face  class  is  enough  to  redefine  authority  and 
voice  (Merry field,  2006),  change  classroom  practices 
relating  to  honoring  diverse  cultural  perspectives 
(Maher  and  Jacob,  2006),  or  change  classroom  dis¬ 
course  patterns  (Wickstrom,  2003). 

Investigating  Other  Domains  Using  CMC 

Increasingly,  CMC  is  used  by  those  who  have  not 
traditionally  used  or  studied  these  tools.  Woodruff  et 
al.  (2001)  took  advantage  of  the  fact  that  many  teen¬ 
agers  have  access  to  computers  and  found  that  a  vir¬ 
tual-reality  chat  room  was  a  viable  means  to  counsel 
a  group  of  high-risk  teenagers  in  smoking  cessation. 
Others  have  investigated  the  psychological  effects  of 
using  CMC .  Morgan  and  Cotten  (2003)  surveyed  500 
college  freshmen  and  found  that  using  communication 
tools  such  as  e-mail  and  instant  messaging  was  asso¬ 
ciated  with  decreased  depression;  however,  students 
who  spent  more  time  using  the  Internet  for  other  activ¬ 
ities  such  as  shopping,  gaming,  or  research  were  more 
likely  to  exhibit  depressive  symptoms.  Beebe  et  al. 
(2004),  on  the  other  hand,  found  a  link  between  the 
chat-room  use  of  high-school  students  and  psycholog¬ 
ical  distress  and  risky  behaviors.  Internet-based  CMC 
tools  can  also  provide  public  datasets  that  can  provide 
insights  regarding  various  groups  of  people.  Mazur 
(2005)  used  blogs  intended  for  teenagers  in  an  under¬ 
graduate  course  on  adolescence  as  a  window  into  the 
lives  of  teenagers  to  help  college  students  better  under¬ 
stand  adolescents.  Surveys  of  students  in  the  course 
suggested  that  using  blogs  in  this  way  increased  stu¬ 
dents’  understanding  of  adolescent  issues  and  was  an 
effective  pedagogical  technique. 

Teacher  Training  and 
Professional  Development 

A  frequent  topic  of  educational  communications 
research  is  teacher  training.  Asynchronous  tools  enable 
students  to  participate  regardless  of  location,  which 
makes  them  good  for  enabling  teachers  to  have  con¬ 
versations  about  their  practice  (Geer  and  Hamill,  2003; 
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Jetton,  2003;  Reynolds,  2002;  Singer  and  Zeni,  2004). 
Devlin-Scherer  and  Daly  (2001)  asked  student  teachers 
to  use  asynchronous  discussion  to  share  their  intern¬ 
ship  experiences  outside  the  limited  class  time.  Gold¬ 
stein  and  Freedman  (2003)  used  student  journals  to 
identify  and  resolve  problems  with  how  their  student 
teachers  were  interacting  with  their  students  and  their 
parents. 

SUMMARY 

Computer-mediated  communication  (CMC)  is  becom¬ 
ing  increasingly  commonplace  in  education.  This 
chapter  began  by  pointing  out  which  features  a  partic¬ 
ular  educational  intervention  is  using  to  provide  a 
means  to  better  align  the  tool  chosen  with  the  activities 
that  it  must  support.  CMC  environments  are  used  for 
a  variety  of  purposes  in  educational  environments, 
from  duplicating  or  supplementing  classroom  dis¬ 
course  to  providing  a  broader  audience  for  student 
work.  CMC  is  a  current  subject  of  research  that  is 
looking  into  how  CMC  affects  learning  and  evaluating 
whether  or  not  discussion  is  evidence  of  learning.  It  is 
also  being  applied  as  a  means  for  research  when  it  can 
provide  access  to  populations  otherwise  difficult  to 
locate  or  can  serve  as  a  convenient  means  of  collecting 
data. 
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ABSTRACT 

Since  the  early  20th  century,  public  school  library 
media  centers  in  the  United  States  have  housed  non¬ 
print  educational  materials.  Since  the  middle  of  that 
century,  K-12  library  media  specialists  have  designed 
learning  experiences  integrating  these  materials  into 
classroom  practice.  For  today’s  library  media  special¬ 
ist,  technology  (both  as  materials  and  as  process)  is 
essential.  Current  American  standards  highlight 
instructional  leadership,  collaboration  with  teachers, 
and  technology  as  the  three  integrating  issues  essential 
to  the  library  media  program  (AASL  and  AECT,  1998). 
Much  early  research  focused  on  developing  and 
explaining  the  multifaceted  role  of  the  library  media 
specialist.  Until  the  1990s,  few  studies  looked  at  the 


relationship  of  K-12  library  media  programs  to  student 
learning.  Since  then,  about  two  dozen  studies  have 
investigated  that  link.  The  political  focus  of  much  of 
this  work  is  undeniable;  concerns  about  the  absence  of 
library  media  from  key  national  legislation  related  to 
American  schooling  have  led  to  research  designed  to 
establish  links  between  library  media  programs  and 
student  achievement.  Similar  pressures  in  the  United 
Kingdom  and  Australia  have  led  to  comparable  efforts 
there.  Quantitative  and  qualitative  approaches  have 
been  used  to  (1)  demonstrate  widespread  correlations 
between  high-quality  library  media  programs  and  stu¬ 
dent  learning,  and  (2)  identify  specific  contributions  of 
such  programs.  Various  recent  initiatives  suggest  an 
expansion  of  K-12  school  library  media  research  and 
a  wider  dissemination  of  its  findings. 
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KEYWORD 

Library  media  center:  A  physical  repository  of  instruc¬ 
tional  materials  and  technology  resources  found  in 
most  American  public  schools. 


INTRODUCTION 

For  almost  100  years,  the  library  media  center  has  been 
home  to  K-12  schools’  emerging  technology— from 
lantern  slides  to  video  editing  equipment.  For  almost 
50  years,  the  K-12  library  media  specialist  has  been 
expected  to  play  a  major  role  in  helping  students  and 
teachers  understand  how  to  use  the  technology  of  the 
day  to  enhance  learning.  Today’s  library  media  spe¬ 
cialist-through  formally  specified  roles  as  teacher,  as 
other  teachers’  partner  in  instructional  design  and 
delivery,  as  information  specialist,  and  as  administra¬ 
tor— is  charged  with  "cnsur[  ing  |  that  students  and  staff 
are  effective  users  of  ideas  and  information”  (AASL 
and  AECT,  1998,  p.  6).  The  library  media  specialist 
achieves  this  mission  not  only  through  fulfilling  these 
roles  but  also  through  promoting  instructional  and 
informational  technologies  as  agents  of  learning. 

In  the  United  States,  the  library  media  specialist’s 
dual  focus  on  technology  and  instruction  has  been 
chronicled  through  successive  editions  of  national 
standards  and  guidelines  for  the  field  — especially 
since  the  publication  of  Standards  for  School  Library 
Programs  (AASL,  1960),  which  first  introduced  both 
the  term  “audiovisual  materials”  and  the  library  media 
specialist’s  role  as  an  instructor  in  “library  skills.” 
Standards  for  School  Media  Programs  (AASL  and 
NEA,  1969),  the  first  set  of  standards  that  saw  the 
imprint  of  AECT  (which  was  then  DAVI),  and  Media 
Programs:  District  and  School  (AASL  and  AECT, 
1975)  promoted  the  role  of  the  library  media  specialist 
as  an  instructional  designer.  Information  Power: 
Guidelines  for  School  Library  Media  Programs 
(AASL  and  AECT,  1988)  brought  the  evolving 
responsibilities  of  the  library  media  specialist  into  the 
electronic  age,  while  the  current  national  stan¬ 
dards — Information  Power:  Building  Partnerships  for 
Learning  (AASL  and  AECT,  1998)— highlight  “lead¬ 
ership,  [instructional]  collaboration  [with  teachers], 
and  technology”  as  the  three  integrating  issues  that 
“are  integral  to  every  aspect  of  the  library  media  pro¬ 
gram  and  every  component  of  the  library  media  spe¬ 
cialist’s  role”  (p.  49).  School  Library  Guidelines 
(IFLA/UNESCO ,  2002)  notes  that  “[t]eachers  and 
librarians  [should]  work  together  in  order  to  ... 
develop,  instruct,  and  evaluate  pupils’  learning  across 
the  curriculum”  (p.  12)  by  using  the  resources  of  a 


library  that  “must  provide  access  to  all  necessary  elec¬ 
tronic,  computer,  and  audiovisual  equipment”  (p.  8). 

The  K-12  library  media  field  has  thus  tried  to 
position  itself  at  the  nexus  of  technology,  information, 
and  instructional  design  in  the  schools  for  almost  50 
years.  Small  (1998)  provided  a  brief  overview  of  this 
history  in  the  United  States,  and  Neuman  (2004)  pro¬ 
vided  an  extensive  one— including  an  examination  of 
the  instructional  design  models  developed  specifically 
for  library  media  specialists  to  use  in  collaborating 
with  teachers.  As  society  has  increased  its  demands 
for  accountability  from  the  schools,  the  library  media 
field  has  increasingly  stressed  its  contributions  to  stu¬ 
dent  learning  and  has  sought  to  demonstrate  that  strong 
K-12  library  media  programs  are  essential  not  only  to 
student  learning  but  also  to  student  achievement  in  the 
information  age. 

THE  RESEARCH  CONTEXT 

Much  of  the  early  research  in  the  field  of  school  library 
media  focused  on  developing  and  delineating  the  role 
of  the  library  media  specialist.  Before  the  1990s,  com¬ 
paratively  few  studies  looked  at  the  contributions  of 
library  media  programs  to  student  learning.  One  land¬ 
mark  study  published  in  1963,  however,  addressed 
questions  that  resonate  even  today:  Mary  Gaver,  who 
is  still  revered  by  library  media  researchers,  investi¬ 
gated  the  effectiveness  of  centralized  library  services 
in  271  elementary  schools  in  13  states  (Lance,  2006): 

She  compared  the  test  scores  of  students  in  schools  with 
classroom  libraries  only,  those  with  centralized  libraries 
run  by  non-librarians,  and  those  with  centralized  libraries 
run  by  librarians,  with  predictable  results.  Students  in 
schools  with  centralized  libraries  managed  by  qualified 
librarians  tended  to  score  higher  [on  the  Iowa  Test  of 
Basic  Skills]  than  their  counterparts  in  schools  without 
centralized  libraries  or  qualified  librarians. 

Although  Gaver’s  findings  are  confounded  by  several 
methodological  difficulties— for  example,  the  students 
who  scored  higher  “had  [also]  received  sessions  in 
learning  about  library  use-skills  similar  to  those  on  the 
[library-skills  section]  of  the  exam”  (Callison,  in 
Lance,  2006)— Gaver  was  clearly  ahead  of  her  time  in 
her  attempt  to  link  strong  library  programs  to  student 
achievement.  Some  20  years  later,  Didier  reviewed  the 
accumulated  studies  that  had  looked  at  the  impact  of 
K-12  library  media  programs  on  student  learning— all 
38  of  them— and  concluded  that  “overall,  the  findings 
show  much  evidence  that  school  library  media  pro¬ 
grams  can  be  positively  related  to  student  achieve¬ 
ment”  (Didier,  1985,  p.  33). 
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Throughout  the  1990s  and  into  the  early  part  of  this 
century,  school  library  media  researchers  in  North 
America  investigated  a  number  of  aspects  related  to 
library  media  programs  and  learning:  resource-based 
learning  (Eisenberg  and  Small,  1995),  learning  as  pro¬ 
cess  (Kuhlthau,  1993;  McGregor,  1994;  Pitts,  1994; 
Todd,  1995),  and  learning  in  electronic  environments 
(Bilal,  2001;  Fidel  et  al.,  1999;  Large  et  al.,  1998;  Neu¬ 
man,  1997;  Solomon,  1994).  Although  the  Library 
Power  Project  (1998-1991),  the  largest  project  to  date 
in  the  history  of  the  K-12  library  media  field,  focused 
on  the  development  and  demonstration  of  effective 
library  media  programs  rather  than  specifically  on  the 
learning  such  programs  engendered,  subsequent  analy¬ 
sis  of  its  data  revealed  some  limited  learning  effects 
(Kuhlthau,  1999).  The  chapter  on  the  library  media  cen¬ 
ter  in  the  previous  edition  of  this  Handbook  provides  an 
extensive  overview  of  the  research  and  scholarly  activ¬ 
ities  undertaken  during  these  years  (Neuman,  2004). 
Callison  (2002)  also  provides  a  comprehensive  review 
of  the  research  conducted  during  this  and  earlier  periods . 
Williams  and  Wavell  (2001)  and  Williams  et  al.  (2002) 
provide  similar  reviews  from  a  British  perspective,  and 
Lonsdale  (2003)  offers  insights  from  an  Australian  one. 


RECENT  RESEARCH 

The  school  library  media  field  encompasses  many  fac¬ 
ets,  and  its  researchers  continue  to  focus  on  all  of  them: 
the  evolving  role  of  the  library  media  specialist,  the 
nature  and  practice  of  collaboration,  information  liter¬ 
acy,  information  searching  in  electronic  environments, 
policy  issues,  principals’  perspectives  on  library  media 
specialists  and  their  programs,  etc.  A  review  of  the 
contents  of  School  Library  Media  Research — the 
research  journal  of  the  American  Association  of  School 
Librarians  (AASL)— for  the  past  5  years  reveals  a 
potpourri  of  topics  and  treatments.  A  similar  review  of 
the  offerings  of  School  Libraries  Worldwide  —  the 
research  journal  of  the  International  Association  of 
School  Librarians  (IASL)— reveals  a  similarly  eclectic 
mix.  Within  this  broad  array,  however,  the  relationship 
of  library  media  programs  to  learning  has  emerged  in 
recent  years  as  the  single  most  important  research  topic 
within  the  field.  As  Neuman  (2003,  p.  517)  noted  in  a 
special  issue  of  a  key  journal  devoted  to  research  ques¬ 
tions  for  the  21st  century:  “The  question  of  student 
learning  is  at  the  crux  of  all  the  most  significant 
research  to  be  done  in  school  library  media  in  the  next 
decade.”  Driven  by  such  factors  as  “the  emergence  of 
electronic  information  resources  that  highlight  the 
relationship  between  learning  and  information  use  as 
never  before  and  the  publication  of  statements  of  learn¬ 


ing  outcomes  [in  information  literacy]  related 
[directly]  to  information  use”  (Neuman,  2003,  p.  503), 
today’s  most  visible  school  library  media  researchers 
both  in  the  United  States  and  abroad  concentrate  on 
documenting  the  importance  of  strong  K-12  library 
media  programs  and  their  impact  on  student  learning 
and  achievement.  Since  the  1990s,  over  two  dozen 
studies  have  looked  at  this  link. 

The  Phenomenon  of  the  Lance  Studies 

Many  of  these  investigations  are  known  as  the  “Lance 
Studies”  because  they  represent  efforts  begun  by  Keith 
Curry  Lance  and  his  colleagues  at  the  Colorado  State 
Library  and  replicated  by  Lance’s  group  and  others. 
The  first  in  the  series,  published  in  1993,  established 
the  methodological  parameters  for  all  the  subsequent 
research.  Lance  and  his  colleagues  used  statistical 
modeling  to  assess  the  impact  of  library  factors  on  test 
scores  while  controlling  for  various  school  and  com¬ 
munity  conditions  (Lance,  2006).  The  researchers 
applied  correlational  analysis,  followed  by  factor  anal¬ 
ysis,  followed  by  regression  analysis  to  determine  the 
relationships  of  23  independent  variables  to  students’ 
scores  on  either  the  Iowa  Tests  of  Basic  Skills  or  the 
Tests  of  Achievement  and  Proficiency.  The  scores 
came  from  a  nonrandom  but  probably  representative 
sample  of  221  public  elementary  and  secondary 
schools  in  Colorado  that  had  responded  to  a  1989 
survey  of  library  media  centers;  Lance  et  al.  used  1980 
U.S.  Census  data  about  the  schools’  communities  and 
building-level  files  from  the  Colorado  Department  of 
Education  (where  Lance  directs  the  Library  Research 
Service)  to  round  out  the  dataset.  The  study  produced 
a  series  of  results  (Lance,  1994)— including  the  dra¬ 
matic  finding  that  "the  size  of  a  library  media  program, 
as  indicated  by  the  size  of  its  staff  and  collection,  is 
the  best  school  predictor  of  academic  achievement”  (p. 
172).  In  other  words,  “The  size  of  the  school  library 
staff  and  collection  explained  21%  of  the  variation  in 
seventh-grade  Iowa  Tests  of  Basic  Skills  reading 
scores,  while  controlling  for  socioeconomic  condi¬ 
tions”  (Scholastic  Library  Publishing,  2006,  p.  8). 

Not  surprisingly,  the  first  of  the  Lance  Studies, 
known  as  the  “Colorado  Study,”  dominated  research 
discussions  in  the  library  media  community.  After  a 
period  of  initial  excitement,  however,  questions  about 
its  methodology— the  small  sample  size,  nonrandom¬ 
ness  of  the  sample,  nonexperimental  design,  and  reli¬ 
ance  on  existing  (rather  than  original)  data— seemed 
to  limit  its  influence  for  several  years.  By  the  end  of 
the  decade,  however,  political  factors  had  intervened. 
No  Child  Left  Behind,  the  landmark  national  educa¬ 
tional  legislation  enacted  in  the  United  States  in  2002, 
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failed  to  mention  school  library  media  programs; 
many  states  failed  to  include  information  literacy  in 
their  standardized  tests .  Alarmed  by  this  marginaliza¬ 
tion  of  key  library  media  content  and  witnessing  a 
concomitant  drop  in  the  number  of  library  media 
programs  in  schools  across  the  country,  the  field  saw 
a  pressing  need  to  demonstrate  the  relationship  of 
library  media  programs  to  student  achievement.  In  at 
least  17  states,  organizations  of  library  media  spe¬ 
cialists  and  other  interested  stakeholders  commis¬ 
sioned  Lance’s  group  and  others  to  demonstrate  that 
relationship.  The  earliest  of  the  state  reports  was  pub¬ 
lished  in  1999.  Interest  crested  in  2002  and  again  in 
2003  with  reports  from  four  more  states  each  year 
(Scholastic  Library  Publishing,  2006). 

The  sheer  number  of  these  reports  as  well  as  the 
widespread  geographic  distribution  of  the  study  sites 
captured  the  field’s  attention  once  again.  Taken  as  a 
whole,  these  reports  paint  a  compelling  national  picture 
of  the  relationship  of  strong  K-12  school  library  media 
programs  and  student  learning.  Although  some  of  the 
work  is  still  in  progress,  the  findings  of  the  14  reports 
released  by  March  2005  show  that  the  relationship 
between  school  library  programs  and  test  scores  cannot 
be  explained  away  by  non-library  factors  or  conditions 
(Lance,  2006).  Refinements  of  the  original  methodol¬ 
ogy  by  Lance’s  group  and  by  various  other  researchers 
(e.g.,  greater  numbers  of  schools,  original  as  well  as 
available  data,  more  extensive  and  detailed  questions, 
and  even  visits  to  school  library  media  centers)  provide 
even  stronger  evidence  of  the  relationship.  Lance 
(2006)  argued  strenuously  against  those  who  would 
dismiss  all  this  evidence  as  merely  correlational: 

The  repeated  identification  of  links  between  libraries  and 
test  scores  that  cannot  be  explained  away  by  other  school 
and  community  conditions  seems  to  me  to  offer  consid¬ 
erable  evidence  for  a  cause-and-effect  argument.  If  the 
correlation  is  spurious,  why  has  it  been  found  in  multiple 
states  over  time?  If  there  are  other  school  or  community 
conditions  that  consistently  explain  away  library-test 
score  links,  I  am  unaware  of  them. 

Lance  had  become  a  celebrity  in  the  school  library 
media  field,  keynoting  many  conferences  and  present¬ 
ing  an  overview  of  the  findings  at  the  White  House 
Conference  on  School  Libraries  in  2002.  By  May 
2005,  a  Web-based  survey  he  had  designed  to  deter¬ 
mine  the  impact  of  the  studies  gathered  information 
from  501  respondents  from  36  states,  who  reported 
using  the  results  to  advocate  for  library  media  pro¬ 
grams  by  sharing  the  research  with  principals  (81%), 
teachers  (66%),  and  others.  Perhaps  more  importantly, 
respondents  reported  spending  “more  time  collaborat¬ 
ing  with  teachers,  teaching  information  literacy  skills 


to  students  (62%) ...  identifying  materials  for  teachers 
(60%)  ...  and  spending  more  time  teaching  collabora- 
tively  with  teachers  (48%)”  (Lance,  2006).  Clearly,  the 
phenomenon  of  the  Lance  research  stream  has  had  a 
major  impact  not  only  on  documenting  the  value  of 
strong  K-12  library  media  programs  but  also  on  devel¬ 
oping  such  programs. 

International  Insights 

The  influence  of  the  Lance  Studies  has  extended  well 
beyond  the  borders  of  North  America  to  influence 
research  in  other  parts  of  the  world.  For  example, 
research  reviews  conducted  for  the  Scottish  Council  for 
Museums,  Archives,  and  Libraries  on  the  impact  of  the 
school  library  resource  center  (SLRC)  in  both  primary 
and  secondary  schools  in  England  cited  Lance’s  work 
(Williams  and  Wavell,  2001)  and  suggested  that  it  can 
be  applied  to  the  English  context  (Williams  et  al.,  2002). 
Similarly,  a  critical  desktop  review  of  research  con¬ 
ducted  in  Australia  noted  that  "the  most  influential  body 
of  research  into  the  impact  of  school  libraries  on  student 
achievement  is  that  of  Keith  Curry  Lance  and  his  col¬ 
leagues  in  the  United  States”  (Lonsdale,  2003,  p.  12). 

Although  these  reviews  cited  the  importance  of 
Lance  and  his  colleagues,  they  also  staked  out  terri¬ 
tory  that  is  especially  important  in  their  own  contexts. 
Williams  and  Wavell  (2001)  sought  to  gain  an  under¬ 
standing  of  “the  broader  learning  experience”  (p.  4) 
in  Scottish  secondary  schools  by  conducting  focus 
groups  and  case  studies  designed  to  assess  “teachers’ , 
students’,  and  librarians’  perceptions  of  how  the 
[learning  resource  center]  can  contribute  to  learning,” 
to  identify  “learning  experiences  related  to  the 
[resource  center],”  and  to  investigate  “indicators  to 
be  used  as  evidence  of  learning  and  examine  [d]  the 
process  of  assessing  the  of  the  impact  of  the  [learning 
resource  center]  on  learning”  (p.  i).  Through  these 
focus  groups,  Williams  and  Wavell  (2001)  identified 
a  number  of  perceptions  of  impact  shared  by  all  three 
groups  — from  the  acquisition  of  information  and 
information  skills  to  the  development  of  the  ability 
to  work  both  independently  and  interpersonally;  from 
the  case  studies ,  they  drew  “learning  themes  of  moti¬ 
vation,  progression,  independence,  and  interaction” 
(p.  ii).  Suggestions  for  moving  ahead  on  the  basis  of 
their  findings  include  a  much  more  intense  focus  on 
the  broader  context  they  sought  to  address:  “the  need 
to  look  beyond  the  immediate  SLRC  for  impact  . . . 
across  learning  contexts,  beyond  the  immediate  situ¬ 
ation,  and  over  appropriate  timescales”  (p.  4). 

The  review  by  Williams  et  al.  (2002)  of  research 
into  the  SLRC’s  role  in  primary  schools  built  on  ideas 
and  methods  used  a  decade  earlier  by  Streatfield  and 
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Markless  (1994).  The  title  of  that  early  report  to  the 
British  Library  Research  and  Development  Depart¬ 
ment  set  the  context  for  the  more  recent  study  as  well: 
Invisible  Learning?  The  Contribution  of  School 
Libraries  to  Teaching  and  Learning.  Both  pieces  argue 
that  SLRCs  do  in  fact  contribute  to  student  learning 
but  in  ways  that  are  difficult  to  measure.  They  also 
suggest  research  into  the  impact  of  various  models  of 
library  media  service  provision— for  example,  flexible 
or  fixed  scheduling,  various  levels  and  activities  of 
staff,  kinds  of  instructional  and  collaborative  activi¬ 
ties— on  student  learning.  The  question  of  scheduling 
has  been  an  issue  in  the  American  K-12  library  media 
field  for  years:  Should  library  media  programs  be 
delivered  through  a  fixed  schedule,  in  which  classes 
come  to  the  facility  for  predetermined  periods  each 
week,  or  through  a  flexible  schedule,  in  which  indi¬ 
viduals  and  groups  come  to  the  facility  as  needed  for 
specific  research  projects  collaboratively  planned  by 
the  teacher  and  the  library  media  specialist?  By  linking 
research  on  student  achievement  to  the  issue  of  sched¬ 
uling,  both  reports  suggest  an  important  question  for 
future  research. 

The  Australian  review,  completed  over  a  4-week 
period  for  the  Australian  Library  Association,  included 
both  Australian  and  international  research  conducted 
since  1990  and  concluded  that  “existing  research 
shows  that  school  libraries  can  have  a  positive  impact, 
whether  measured  in  terms  of  reading  scores,  literacy 
or  learning  more  generally,  on  student  achievement” 
(Lonsdale,  2003,  p.  30).  Specific  details  related  to  that 
general  conclusion  echo  insights  provided  by  Didier 
(1985)  and  expanded  more  recently  by  Lance  (Scho¬ 
lastic  Library  Publishing,  2005/2006).  Lonsdale  (2003, 
p.  2)  suggested,  further,  that  action  research  and  evi¬ 
dence-based  practice  are  critical  to  school  libraries  in 
the  future: 

In  general,  the  literature  confirms  the  need  for  local, 
evidence-based  practice  if  the  roles  of  the  school  library 
and  teacher  librarian  in  student  learning  are  to  be  valued 
in  the  way  that  the  research  suggests  they  should  be 
valued.  Such  research  is  an  important  strategic  tool  for 
raising  the  profile  and  prestige  of  library  professionals 
and  for  reinforcing  in  the  minds  of  policy-makers  and 
school  communities  the  crucial  contribution  that  school 
libraries  can  make  to  student  achievement. 

The  Ohio  Study 

Despite  the  influence  of  the  Lance  Studies,  Lance 
himself  noted  that  the  Colorado  model  has  been 
exhausted  and  that  it  is  time  to  develop  new  directions 
for  school  library  media  impact  research  (Lance, 
2006).  Lance’s  personal  concern  is  to  devise  research 


methods  that  will  align  school  library  media  research 
with  the  experimental  designs  favored  by  the  U.S. 
Department  of  Education ,  but  the  suggestions  for  con¬ 
textual  research  noted  above  as  well  as  the  most  prom¬ 
inent  study  in  the  United  States  today  involve  a  qual¬ 
itative  rather  than  a  quantitative  approach.  The  Ohio 
Study,  conducted  under  the  auspices  of  Rutgers  Uni¬ 
versity’s  Center  for  International  Scholarship  in 
School  Libraries  (CISSL;  pronounced  “sizzle”), 
involved  “39  effective  school  libraries  across  Ohio  . . . 
13,123  students  in  grades  3  to  12  and  879  faculty.  The 
research  study  sought  to  understand  how  students  ben¬ 
efit  from  school  libraries”  (Todd,  2004,  p.  1). 

The  focus,  then,  was  to  identify  effective  libraries 
and  to  ask  those  who  use  them  to  rate  the  dimensions 
of  their  effectiveness.  Researchers  received  nomina¬ 
tions  for  the  schools  to  be  included,  and  their  choices 
were  validated  by  an  expert  panel.  They  collected  data 
through  two  Web-based  surveys,  one  for  students  and 
one  for  faculty.  Each  survey  included  the  same  48 
items  (adjusted  to  reflect  the  nature  of  the  respondent) 
in  seven  blocks:  (1)  finding  and  getting  information; 
(2)  using  information  to  complete  school  work;  (3) 
supporting  school  work  in  general;  (4)  using  comput¬ 
ers  in  the  library,  elsewhere  in  the  school,  and  at  home; 
(5)  supporting  general  reading  interests;  (6)  helping 
with  information  tasks  outside  the  school;  and  (7)  sup¬ 
porting  general  school  achievement.  Respondents  used 
a  five-part  Likert  scale  to  rate  how  helpful  the  library 
was  (most,  quite,  some,  a  little,  and  does  not  apply) 
with  regard  to  each  statement.  Statements  were  a  mix 
of  the  cognitive  (e.g.,  “The  school  library  has  helped 
me  find  different  opinions  about  my  topics”)  and  the 
affective  (e.g.,  “The  school  library  has  helped  me  feel 
better  about  finding  information”).  Each  survey  also 
included  an  open-ended  question  designed  to  contex¬ 
tualize  the  responses  provided  through  the  surveys. 

Not  surprisingly,  the  results  were  overwhelmingly 
positive  (Todd,  2004,  p.  2): 

Statistically  99.44%  of  the  sample  (13,050  students)  indi¬ 
cated  that  the  school  library  and  its  services,  including 
roles  of  school  librarians,  have  helped  them  in  some  way, 
regardless  of  how  much,  with  their  learning  as  it  relates 
to  the  48  statements. 

Faculty  were  even  more  positive  than  students:  Their 
mean  responses  were  higher  than  the  student  means  in 
all  seven  blocks.  Overall,  the  helps  were  most  helpful 
at  the  elementary  level,  where  students  encounter 
information-literacy  concepts  for  the  first  time  and 
decreased  in  ranking  as  students  moved  through  their 
school  years.  The  helps  that  were  “ranked  highest  by 
students”  involved  information  technology  and 
achieved  high  scores  in  whatever  blocks  they  appeared 
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(Todd,  2004,  p.  5).  Two  of  them  received  higher  per¬ 
centages  of  “most  helpful”  ratings  than  any  other  items 
on  the  instrument:  “Computers  have  helped  me  find 
information  inside  and  outside  of  the  school  library” 
(49.0%)  and  “Computers  in  the  school  library  have 
helped  me  do  my  school  work  better”  (41.6%). 

The  Ohio  study  was  responsive  to  the  K-l 2  library 
media  field’s  perceived  need  to  demonstrate  its  rele¬ 
vance  to  student  achievement,  and  it  has  correspond¬ 
ing  limitations.  It  involved  a  dataset  that  was  “very 
suburban  with  over  90%  white  population”  and 
accepted  “only  positive  testimony  from  teachers  and 
students  ...  so  that  conclusions  would  emphasize  the 
constructive  role  the  library  could  play  in  providing 
services  with  a  staff  of  qualified  librarians”  (Callison, 
in  Lance,  2006).  In  addition,  it  relied  exclusively  on 
self-reported  data  collected  through  a  website— a 
methodology  that  might  well  present  an  irresistible 
temptation  for  mischief  to  middle-  and  high-school 
students,  especially  as  they  get  to  the  end  of  a  49-item 
instrument.  Nevertheless,  the  results  of  the  study  are 
informative  about  the  specific  ways  in  which  high- 
quality  library  media  programs  can  support  student 
learning.  Its  widely  publicized  major  conclusion  “that 
an  effective  school  library,  led  by  a  credentialed 
library  media  specialist  who  has  a  clearly  defined  role 
in  information-centered  pedagogy,  plays  a  critical  role 
in  facilitating  student  learning  for  building  knowl¬ 
edge”  (Scholastic  Library  Publishing,  2006,  p.  17)  has 
evidentiary  backing  as  well  as  political  force. 


CURRENT  ISSUES 

Despite  all  the  research  suggesting  that  a  comprehen¬ 
sive,  integrated  school  library  media  program  is 
closely  associated  with  increased  learning,  many  edu¬ 
cators  still  think  of  school  library  media  programs  as 
a  luxury  — worthwhile  but  not  integral  to  the  curricu¬ 
lum  or  to  student  achievement— especially  in  the  areas 
highlighted  in  the  United  States  by  federal  legislation 
and  covered  by  most  states’  standardized  tests.  Indeed, 
many  jurisdictions  classify  school  library  media  as  a 
specialty  area  like  music  and  physical  education,  mak¬ 
ing  school  library  media  programs  especially  vulner¬ 
able  in  an  era  of  tightening  budgets  and  increasing 
attention  to  standardized  test  scores.  Anecdotal  evi¬ 
dence  abounds  about  library  media  centers  being 
closed  and  positions  either  abolished  or  effectively  cut 
in  half  as  individual  library  media  specialists  are 
deployed  to  serve  more  than  one  school.  The  pressures 
in  other  parts  of  the  world  may  be  less  immediate  and 
less  visible,  but  British  and  Australian  researchers  also 
cite  the  need  for  “demonstrating  the  value  of,  and 


attaining  appropriate  recognition  for,  the  work  of  the 
SLRC,  an  issue  of  concern  to  the  Information  and 
Library  Science  (ILS)  profession  world-wide”  (Will¬ 
iams  and  Wavell,  2002,  p.  4). 

Facing  a  continuing  threat  of  marginalization  or 
elimination,  the  K-l 2  library  media  field  continues  to 
turn  to  research  as  the  master  key  that  will  unlock  the 
barriers  to  a  full  appreciation  of  the  importance  of 
library  media  programs  and  open  the  doors  to  fully 
realized,  high-quality  programs  throughout  the  educa¬ 
tional  system.  Scholastic  Library  Publishing  has 
released  two  editions  of  a  20-page  Research  Foundation 
Paper  entitled  School  Libraries  Work!  (2005/2006), 
drawing  together  findings  from  recent  United  States 
studies  into  a  glossy  public-relations  brochure  that  has 
been  widely  circulated.  A  2005  summit  convened  by 
School  Library  Journal ,  the  major  North  American 
magazine  for  practitioners  in  the  field,  “ended  with  a 
loud  call  for  more  scholarly  research  into  the  impact 
of  school  libraries  on  learning”  (Glick,  2005,  p.  11)  and 
expressed  a  yearning  “for  a  way  to  demonstrate  the 
complex  ways  in  which  school  libraries  contribute  to 
student  learning  and  growth,  something  beyond  the 
grasp  of  standardized  tests”  (p.  12).  Summiteers  iden¬ 
tified  research  into  teacher-librarian  collaboration  and 
action  research  undertaken  by  practitioners  as  promis¬ 
ing  avenues  for  demonstrating  school  library  media 
programs’  contributions  to  student  learning. 

Attendees  also  recognized  the  importance  of  com¬ 
municating  their  work  to  the  broader  educational 
world,  noting  that  “a  large  body  of  research  . . .  already 
exists”  (Glick,  2005,  p.  12)  but  that  it  has  not  been 
available  to  or  accepted  by  those  outside  the  school 
library  media  field.  This  isolation  has  been  a  cause  of 
concern  for  years,  leading  to  repeated  calls  for  school 
library  media  researchers  to  publish  and  present  in 
allied  fields.  As  Callison  (2002,  p.  362)  noted,  “Until 
research  strands  ...  move  into  a  broader  research 
framework,  it  is  likely  that  findings,  no  matter  how 
dramatic  or  significant,  will  remain  dormant  without 
causing  change.”  Neuman  (2004)  discussed  the  situa¬ 
tion  in  depth  and  suggested  that  a  number  of  current 
educational  trends  — particularly  the  acceptance  of  the 
constructivist  philosophy,  the  ubiquity  of  information 
and  information  technology,  and  the  necessity  for  chil¬ 
dren  to  learn  how  to  use  information  as  a  tool  for 
learning  in  all  the  content  areas— converge  into  a  fruit¬ 
ful  venue  for  advancing  interest  in  and  communication 
of  school  library  media  research. 

Two  current  developments  suggest  that  the  field  is 
indeed  moving  ahead,  both  in  its  research  agenda  and 
in  its  efforts  to  broaden  its  audience.  CISSL,  founded 
by  prominent  school  library  media  researchers  Carol 
Kuhlthau  and  Ross  Todd  in  2003,  is  committed  to 
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establishing  a  firm  foundation  for  national  and  inter¬ 
national  research  in  the  field.  In  2005,  CISSL  hosted 
a  research  symposium  (held  at  the  headquarters  of 
Scholastic  Publishing)  that  brought  together  many  of 
the  field’s  premier  researchers.  In  the  spring  of  2006, 
CISSL  announced  the  formation  of  the  CISSL  Virtual 
Research  Community,  an  open-source  website  at 
which  five  of  those  researchers  lead  online  discussions 
around  some  of  the  major  themes  that  emerged  during 
the  symposium  held  in  2005:  collaboration,  learning 
theory,  the  nature  of  research,  organizational  change, 
leadership,  inquiry  learning,  and  translating  research 
into  practice.  CISSL  plans  to  open  the  site  to  the  K-12 
school  library  media  community  at  large  and,  eventu¬ 
ally,  to  anyone  who  wants  to  participate. 

Also  in  the  spring  of  2006,  the  American  Educa¬ 
tional  Research  Association  voted  to  accept  a  new  Spe¬ 
cial  Interest  Group  (SIG)  for  school  library  media 
researchers.  The  new  Research,  Education,  Informa¬ 
tion,  and  School  Libraries  SIG  plans  to  bring  school 
library  media  research  to  the  programs  and  publications 
of  the  largest  educational  research  association  in  the 
world.  The  effort  is  in  its  early  stages,  but  it  suggests 
a  promising  development  in  the  expansion  of  the 
research  and  issues  of  the  K-12  library  media  field  into 
the  broader  realm  of  educational  research  and  practice. 


CONCLUDING  REMARKS 

The  political  focus  of  much  of  the  recent  work  in 
school  library  media— both  in  the  United  States  and 
elsewhere— is  undeniable:  Concerns  about  the  dwin¬ 
dling  profile  of  the  field  have  created  a  research  agenda 
that  often  seems  more  about  proving  a  foregone  con¬ 
clusion  than  about  discovering  new  knowledge.  Nev¬ 
ertheless,  recent  research  has  in  fact  produced  compel¬ 
ling  evidence  that  library  media  programs  are  related 
to  various  kinds  of  student  achievement.  In  addition, 
a  number  of  recent  studies  have  resulted  in  a  deeper 
understanding  of  the  specific  contributions  of  strong 
library  media  programs  to  student  learning.  Over  the 
past  decade,  both  quantitative  and  qualitative 
approaches  have  been  used  to  gather  this  evidence. 

The  focus  of  this  research  reflects  the  culmination 
of  the  school  library  media  field’s  evolution:  No  longer 
content  to  be  seen  as  overseers  of  a  collection  and 
distribution  point  for  materials— tucked  away  at  the 
end  of  the  hall— today’s  K-12  library  media  profes¬ 
sionals  have  staked  a  claim  to  centrality  in  the  schools’ 
most  important  mission.  Although  the  field  still  places 
a  high  priority  on  its  traditional  mission  of  promoting 
reading  and  a  love  of  literature,  years  of  research  atten¬ 
tion  to  its  role  in  promoting  learning  through  the  use 


of  technology  and  instructional  design  have  led  to  an 
understanding  that  library  media  programs  play  an 
essential  role  in  student  learning  in  the  information 
age.  In  the  past  decade,  as  the  United  States  and  other 
nations  have  increased  their  demands  for  accountabil¬ 
ity  from  schools— evidence  that  learning  has  actually 
occurred— the  school  library  media  field  has  increas¬ 
ingly  recognized  the  importance  of  demonstrating  to 
other  educators,  to  parents,  and  to  policymakers  that 
strong  library  media  programs  are  demonstrably  essen¬ 
tial  to  student  achievement  as  well  as  to  "the  authentic 
[information-based]  learning  that  modern  education 
seeks  to  promote”  (AASL  and  AECT,  1998,  p.  2).  Now 
looking  at  research  as  a  political  as  well  as  an  intel¬ 
lectual  endeavor,  the  modern  K-12  school  library 
media  field  has  indeed  come  of  age. 
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ABSTRACT 

Knowledge  is  collected  and  organized  into  technol¬ 
ogy-based  systems  in  various  ways  to  support  learning 
and  performance.  Four  main  types  of  systems  are  iden¬ 
tified  under  the  general  term  of  knowledge  systems : 
(1)  expert  or  knowledge -based,  (2)  knowledge  man¬ 
agement,  (3)  knowledge  communities,  and  (4)  hybrid 
systems  that  combine  elements  of  the  other  types. 
Expert  systems  are  computer  programs  that  use  knowl¬ 
edge  derived  from  human  experts  to  provide  guidance 
to  novices.  A  number  of  these  systems  have  been 
adapted  to  assist  in  learning.  Expert  systems  use  cap¬ 
tured  knowledge  to  help  a  computer  program  provide 
assistance  to  a  user.  In  knowledge  management  sys¬ 


tems,  the  knowledge  is  captured  and  organized  for 
direct  access  by  the  user.  The  third  type  of  system  is 
aimed  at  facilitating  direct  human-to-human  commu¬ 
nication  of  knowledge  and  includes  research  that 
comes  under  the  categories  of  communities  of  practice 
or  knowledge  communities.  This  area  also  includes 
informal  knowledge  sharing  networks  that  emerge 
through  such  technologies  as  instant  messaging.  This 
chapter  highlights  common  research  issues  that  cut 
across  the  three  approaches,  including  how  knowledge 
is  derived  from  human  sources;  how  knowledge  is 
encoded  in  a  machine  readable  form;  how  collections 
of  digital  knowledge  are  organized  using  taxonomies, 
metadata,  and  ontology;  and,  finally,  knowledge  qual¬ 
ity  assurance. 
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KEYWORDS 

Knowledge-based  system :  A  computer  program  that 
can  reason,  based  on  a  database  of  knowledge 
acquired  from  a  human  expert  (also  known  as  an 
expert  system). 

Knowledge  communities:  Technology  focused  on  bring¬ 
ing  communities  of  people  together  to  both  generate 
and  share  knowledge  about  a  common  interest. 

Knowledge  management:  A  collection  of  technologies 
and  organizational  processes  aimed  at  capturing, 
disseminating,  and  archiving  performance-related 
knowledge  within  an  organization. 


INTRODUCTION 

Before  we  begin  to  examine  the  area  of  knowledge 
systems,  we  need  to  establish  what  we  mean  by  knowl¬ 
edge ,  and  in  particular  how  it  is  distinguished  from 
data  or  information.  A  number  of  philosophical  dis¬ 
cussions  on  this  distinction  have  been  published— see, 
for  example,  Ackoff  (1989),  Davenport  and  Prusak 
(1998),  Quigley  and  Debons  (1999)— but  they  are 
beyond  the  scope  of  this  chapter.  As  a  basic  working 
model,  we  can  say  that  data  are  the  basic  elements  of 
memory  represented  in  symbols.  Information  is  data 
that  have  been  given  meaning  by  way  of  a  relational 
connection  to  other  data.  Knowledge  arises  from  infor¬ 
mation  that  has  been  collected  in  such  a  way  as  to 
usefully  inform  action. 

Knowledge  is  often  broken  down  into  further  cat¬ 
egories,  such  as:  (1)  declarative  (knowledge  about 
things)  and  procedural  (knowledge  of  how  to  do  some¬ 
thing)  (Kogut  and  Zander,  1992);  (2)  explicit  (formally 
codified  in  documents  and  other  formal  communica¬ 
tions)  and  tacit  (personal  intuitions  and  beliefs  that  are 
communicated  informally  by  casual  communications) 
(Nonaka,  1994);  and  (3)  general  (knowledge  that  is 
held  by  large  numbers  of  individuals  and  easily  trans¬ 
ferred)  and  specific  (knowledge  that  is  possessed  by  a 
limited  number  of  individuals  and  is  difficult  to  trans¬ 
fer)  (Jensen  and  Meckling,  1996).  Specific  knowledge 
is  related  to  the  concept  of  expertise,  where  someone 
has  a  recognized  depth  of  knowledge  in  a  specific  area 
(Ericsson  and  Smith,  1991).  Related  to  expert  knowl¬ 
edge  is  heuristic  knowledge,  where  decisions  are  made 
using  general  rules  derived  from  experience.  Experts 
tend  to  have  a  more  developed  set  of  heuristics  than 
novices. 

Much  of  the  history  of  computing  has  involved 
developing  more  efficient  data  and  information  pro¬ 
cessing,  leaving  humans  to  derive  the  knowledge  from 
the  information  and  make  decisions  based  on  this 


knowledge  (Campbell-Kelly  and  Aspray,  1996).  What 
distinguishes  knowledge  systems  is  the  desire  to  have 
software  more  actively  assist  in  deriving  knowledge 
and  more  directly  facilitate  decision  making,  or  in 
some  cases  have  the  software  make  decisions  (e.g., 
expert  and  agent -based  systems). 

Historically,  three  predominant  areas  of  interest 
related  to  knowledge  systems  can  be  identified.  The  first 
of  these  was  research  directed  toward  computer  systems 
that  used  expert-derived  knowledge  to  guide  novice 
users  to  solve  a  problem  (Giarratano  and  Riley,  2004; 
Jackson,  1998).  This  research  first  became  prevalent  in 
the  1970s  and  1980s.  In  the  1990s,  interest  developed 
more  toward  knowledge  management  systems  (Awad 
and  Ghaziri ,  2003 ;  Becerra-Fernandez  et  al . ,  2004) .  The 
emphasis  shifted  away  from  capturing  and  structuring 
knowledge  for  an  intelligent  program  toward  capturing 
and  structuring  knowledge  for  direct  access  by  decision 
makers  (a.k.a.  knowledge  workers).  Another  distinction 
of  knowledge  management  is  that  it  is  interdisciplinary 
and  not  just  about  the  technology.  Knowledge  manage¬ 
ment  has  gained  impetus  from  the  business  and  man¬ 
agement  fields,  and  it  is  very  much  concerned  with  the 
human  and  communication  issues  in  the  processing, 
collecting,  and  dissemination  of  knowledge.  More 
recently,  there  has  been  a  move  toward  supporting  more 
dynamic  knowledge  creation  and  sharing  communities 
through  the  use  of  collaborative  technologies  (Fuks  et 
al.,  2005).  This  chapter  reviews  technologies  and 
research  issues  in  these  three  areas,  all  of  which  share 
a  number  of  underlying  problems.  In  particular,  how 
can  we  obtain  and  organize  knowledge  in  such  a  way 
that  it  facilitates  just-in-time  learning  and  effective  per¬ 
formance  by  individuals  and  organizations? 

KNOWLEDGE-BASED/ 

EXPERT  SYSTEMS 

Knowledge-based  systems  first  emerged  as  an  area  of 
interest  during  the  early  development  of  the  field  of 
artificial  intelligence  (Al)  (Russell  and  Norvig,  2002). 
In  artificial  intelligence,  computer  scientists  attempted 
to  come  up  with  computer  models  for  intelligence. 
They  sought  to  move  computer  technology  beyond 
number  and  data  processing  to  knowledge  processing, 
in  which  computers  could  make  decisions  similar  to 
those  made  by  humans.  Initially,  the  interest  was  in 
language  processing  and  general  problem-solving 
algorithms,  as  reflected  in  the  work  of  Newell  and 
Simon  (1963).  The  early  work  in  Al  hit  on  two  of  the 
enduring  research  questions  in  knowledge  systems. 
First,  how  can  we  best  represent  knowledge  within  a 
computer  system?  Second,  how  can  we  efficiently 
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search  through  represented  knowledge  to  find  the 
knowledge  relevant  to  solving  a  particular  problem? 
One  way  to  confront  these  problems  is  to  narrow  the 
domain  of  interest  in  which  we  are  trying  to  demon¬ 
strate  some  intelligence.  This  notion  led  toward  an 
emergence  in  the  1970s  of  what  were  called  knowl¬ 
edge-based  systems  or  expert  systems.  Expert  systems 
were  constrained  to  solving  problems  in  a  narrow 
domain  of  interest,  such  as  a  particular  area  of  medical 
diagnosis.  Expert  systems  introduced  another  enduring 
research  question  for  knowledge  systems:  How  can  we 
elicit  from  humans  the  knowledge  they  make  use  of 
to  solve  problem? 

An  expert  system  generally  consists  of  a  knowl¬ 
edge  base  and  inference  engine.  A  knowledge  base  is 
a  store  of  recorded  knowledge  derived  from  interview¬ 
ing  experts.  An  inference  engine  is  a  computer  pro¬ 
gram  that,  when  given  a  particular  problem,  can  iden¬ 
tify  which  knowledge  in  the  knowledge  base  is  relevant 
and  when  it  must  ask  questions  of  the  user  to  obtain 
information  relevant  to  the  problem.  Through  interact¬ 
ing  with  the  users,  an  inference  engine  can  eventually 
propose  a  solution  to  the  problem  at  hand.  One  of  the 
best  known  of  the  early  systems  was  MYCIN,  which 
performed  medical  diagnosis  (Shortliffe  and  Davis, 
1975)  and  specified  treatments  for  blood  disorders. 
The  software  architecture  of  MYCIN  became  the  basis 
for  many  subsequent  expert  systems.  It  used  what  was 
referred  to  as  a  rule-based  approach  to  knowledge 
representation,  which  represented  an  expert’s  heuristic 
knowledge  .  The  knowledge  base  consisted  of  a  number 
of  what  were  called  production  rules.  These  had  the 
general  form  of  “If  A  is  true  and  B  is  true,  then  the 
diagnosis  X  is  highly  likely,”  where  A  and  B  repre¬ 
sented  a  particular  observation  about  a  patient.  Rules 
were  derived  from  interviewing  experts  and  were 
chained  together.  A  and  B  in  the  previous  example 
might  be  established  by  other  rules,  such  as:  “If  the 
blood  pressure  is  high  and  the  patient  is  male,  then  A 
is  true.”  Eventually  the  chaining  of  the  rules  would 
result  in  the  generation  of  questions  for  the  user— for 
example,  “Is  the  patient’s  blood  pressure  140/90  or 
higher?”  If  the  system  had  enough  rules  for  the  domain 
and  the  user  could  provide  the  required  information, 
the  performance  of  the  system  could  be  tested  against 
the  real  expert’s  performance. 

It  was  recognized  early  that  expert  systems  and  the 
knowledge  they  contained  not  only  had  a  role  to  play 
in  assisting  performance,  such  as  supporting  diag¬ 
noses,  but  could  also  have  a  role  to  play  in  education. 
In  fact,  GUIDON  (Clancey,  1982),  the  first  expert  sys¬ 
tem  to  be  built  for  educational  purposes,  was  con¬ 
structed  from  the  MYCIN  knowledge  base.  In  addition 
to  the  knowledge  base  on  the  domain,  it  added  a  knowl¬ 


edge  base  of  rules  about  how  to  teach.  In  GUIDON, 
users  performed  a  diagnosis  and  asked  questions  (i.e., 
played  the  role  of  the  expert  system).  The  teaching  rule 
determined  how  to  answer  requests  for  help  or  when 
to  intervene  and  correct  less  than  optimal  performance 
by  the  user.  A  number  of  other  systems  and  models 
for  what  became  known  as  intelligent  tutoring  systems 
would  later  emerge  (Beck  et  al.,  1996;  Wenger,  1987). 

In  addition  to  domain  and  instructional  knowledge, 
another  key  defining  feature  of  such  systems  is  student 
models  (Self,  1999).  Representing  knowledge  of  the 
student  would  allow  the  tailoring  of  instructional  con¬ 
tent  and  strategies  to  suit  the  needs  of  the  individual 
student. 

Despite  a  significant  investment  in  expert  systems 
technology  throughout  the  1970s  and  1980s,  interest 
had  waned  by  the  1990s.  Although  the  term  expert 
system  is  now  seldom  used,  research  work  related  to 
it  has  not  died  out.  It  has  instead  formed  a  foundation 
for  newer  avenues  of  research.  Expert  systems  tech¬ 
nology  is  a  key  component  of  modem  computer  games 
(Laird,  2002),  where  it  assists  in  creating  more  chal¬ 
lenging  computer-controlled  opponents  for  human 
players.  It  is  also  an  integral  part  of  the  currently 
popular  area  of  research  into  intelligent  agents  (Agar- 
wal  et  al.,  2004;  d’lnverno  and  Luck,  2003).  Agents 
are  computer  programs  that  incorporate  problem-solv¬ 
ing  abilities  and  act  to  assist  users  on  some  computer- 
based  task.  The  work  on  agents  builds  upon  the  tradi¬ 
tion  of  expert  systems  research  but  also  integrates 
newer  technology;  for  example,  Baylor  and  Ryu 
(2003)  investigated  the  use  of  human-like  graphical 
interfaces  (personas)  on  agents  designed  to  assist 
learners  and  found  them  to  have  a  positive  effect  on  a 
learner’s  interactive  experience. 


KNOWLEDGE  MANAGEMENT 

The  area  of  knowledge  management  has  arisen  as  an 
important  area  of  study  in  the  last  15  years.  It  has  its 
roots  in  management  thinking  and  the  rising  importance 
of  what  has  been  referred  to  as  the  knowledge  worker 
(Drucker,  1959).  Historically,  organizations  have  con¬ 
sisted  of  large  numbers  of  menial  workers  and  relatively 
few  decisions  makers.  Much  of  the  decision  making  was 
done  by  the  people  at  the  top,  and  many  people  were 
employed  to  provide  the  information  to  those  at  the 
top;  for  example  ,  in  warehouses  people  were  employed 
to  perform  manual  inventories,  and  in  production 
plants  people  were  employed  to  walk  around  and  check 
equipment  gauges  and  operations.  At  the  same  time  as 
much  of  this  information  gathering  became  automated 
through  technologies  such  as  bar-code  readers  and 
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remote  sensors,  many  businesses  and  markets  under¬ 
went  dramatic  change.  Decision  making  became  more 
decentralized.  Many  more  workers  at  all  levels  became 
required  to  deal  with  knowledge  and  make  decisions. 
Organizations  began  to  rely  on  knowledge  and  timely 
decision  making  to  be  flexible  enough  to  survive  in 
rapidly  changing  environments.  New  knowledge- 
dependent  professions  arose  (e.g.,  marketing,  software 
engineers),  and  traditionally  knowledge-intensive  pro¬ 
fessions  (e.g.,  lawyers,  researchers,  and  teachers) 
became  more  in  demand. 

On  an  organizational  level,  businesses  became 
interested  in  how  they  could  better  support  knowledge 
workers  by  providing  them  with  processes  and  technol¬ 
ogy  to  efficiently  develop,  share,  and  use  information 
and  knowledge  .  From  this  change  in  business  and  asso¬ 
ciated  developments  in  technology  has  emerged  the 
need  for  what  is  now  called  knowledge  management. 

These  changes  do  not  just  apply  to  businesses;  they 
apply  to  all  organizations,  including  political  and  mil¬ 
itary  ones .  Among  the  U.S .  military  is  much  discussion 
regarding  force  transformation.  Rather  than  training 
based  on  preestablished  doctrine  in  relation  to  known 
threats,  it  is  now  necessary  to  deal  with  what  is  called 
asymmetric  warfare.  In  asymmetric  warfare,  threats 
come  from  adaptive,  often  unknown  enemies  who 
attack  from  many  directions  (Lovelace  and  Votel, 
2004).  Rigid  centralized  command  and  control  are 
seen  as  too  slow  to  respond  to  such  threats.  Another 
driver  here  is  the  fact  that,  as  military  technology  is 
becoming  more  complex,  the  manpower  and  time 
available  for  training  have  been  reduced;  thus,  tradi¬ 
tional  training  practices  (getting  knowledge  inside  the 
head)  must  be  complemented  by  systems  that  provide 
knowledge  to  decision  makers  on  an  as-needed  basis. 

Technology  obviously  plays  a  key  role  in  support¬ 
ing  the  knowledge  worker;  therefore,  much  of  the 
emphasis  in  knowledge  management  is  in  the  use  of 
technology.  Some  would  argue,  however,  this  has 
tended  to  be  overemphasized  relative  to  the  organiza¬ 
tional  and  social  factors  involved,  such  as  creating  a 
knowledge  sharing  culture. 

The  Internet — in  particular ,  the  World  Wide  Web  — 
is  an  important  knowledge-management  tool.  Knowl¬ 
edge  workers  once  had  to  spend  much  time  in  libraries 
sorting  through  catalogs,  finding  books  and  articles, 
and  scanning  through  indexes.  The  Internet  has  pro¬ 
vided  an  alternative  source  of  information  and  knowl¬ 
edge  that,  due  to  the  increasing  sophistication  of  search 
engines,  has  allowed  users  to  locate  resources  in  min¬ 
utes  rather  than  hours.  Despite  the  usefulness  of  the 
Web,  there  are  deficiencies,  and  research  is  continually 
underway  to  make  organizing,  tagging,  and  searching 
for  information  and  knowledge  easier. 


The  problem  is  that  a  search  engine  cannot  easily 
distinguish  meaning.  If  you  type  the  word  “tank”  into 
Google,  you  will  get  over  200  million  hits,  the  top  ones 
being  for  a  space  in  a  New  York  gallery,  a  children’s 
cartoon  character,  and  a  military  tank  museum. 
Research  by  Bronander  and  colleagues  (2004)  revealed 
that  many  users  take  the  path  of  least  resistance  rather 
than  using  effective  search  and  retrieval  methods.  They 
will  use  few  keywords  and  ignore  the  advanced  func¬ 
tionality  available. 

The  limitation  of  search  technology  stems  from 
the  fact  that  the  language  used  to  encode  Web  pages 
(HTML)  is  primarily  designed  to  help  a  browser  dis¬ 
play  the  content.  It  is  not  designed  to  organize  content 
according  to  its  meaning.  Much  research  is  being  car¬ 
ried  out  to  move  to  what  is  called  the  Semantic  Web , 
where  the  meaning  of  Web-based  content  is  more 
readily  apparent  to  automated  systems  and  search 
engines.  The  vision  of  the  Semantic  Web  is  being 
promoted  heavily  by  Tim  Berners-Lee,  the  original 
inventor  of  the  Web,  who  particularly  emphasizes  the 
potential  of  the  Web  as  a  knowledge-management 
environment  (Berners-Lee  et  al.,  2001).  The  develop¬ 
ment  of  the  Semantic  Web  requires  the  use  of  com¬ 
puter-readable  codes  that  designate  the  meaning  of 
Web  content.  This  includes  embedded  tags  in  text  and 
metadata  in  digital  resources  (such  as  text,  graphic 
video,  and  programs).  Metadata  are  simply  data  that 
tell  us  something  about  other  data;  for  example,  the 
Dublin  core  (see  http://dublincore.org/)  is  metadata 
that  designate  such  things  as  the  author  and  date  of 
creation  of  a  resource  on  the  Internet.  Metadata  are  an 
important  part  of  the  development  of  learning  object 
technology  (see  Chapter  29  in  this  volume).  Another 
important  technology  is  Extended  Markup  Language 
(XML),  which  is  a  meta-language  for  creating  systems 
of  tags  that  add  meaning  to  Web  content. 


KNOWLEDGE  COMMUNITIES 

Some  commentators  have  witnessed  a  move  toward  a 
second  generation  of  knowledge-management  systems 
(McElroy,  2003).  In  the  first  generation,  the  emphasis 
was  mainly  on  using  technology  to  capture,  organize, 
and  archive  existing  and  primarily  explicit  knowledge. 
Traditional  knowledge-management  initiatives  have 
often  failed,  and  the  reason  most  often  cited  for  these 
failures  is  an  overemphasis  on  technology  and  under¬ 
emphasis  on  human  and  organizational  issues,  such  as 
providing  leadership  and  incentives  for  knowledge 
sharing  (Ambrosio,  2000;  Lesser  et  al.,  2000;  Ribiere 
and  Sitar,  2003).  Second-generation  knowledge  man¬ 
agement  is  more  focused  on  connecting  people  to  share 
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their  tacit  knowledge  and  on  the  knowledge-generation 
process  rather  than  the  knowledge- archiving  process. 
McElroy  (2003)  sees  the  work  in  knowledge  manage¬ 
ment  as  converging  with  the  concept  of  organizational 
learning  developed  by  Senge  (1990) ,  in  which  learning 
and  innovation  are  seen  as  social  rather  than  adminis¬ 
trative  processes. 

Other  influences  in  this  direction  are  a  develop¬ 
ment  of  interest  in  virtual  communities  (Ellis  et  al., 
2004)  and  communities  of  practice  (Wenger  and  Sny¬ 
der,  2000)  and  technologies  that  facilitative  group 
work  (Fuks  et  al.,  2005).  Virtual  communities  research 
is  concerned  with  the  development  of  relationships  that 
occur  exclusively  through  online  interaction.  The 
issues  that  researchers  are  considering  include  the 
nature  of  online  identities  (Boyd  and  Heer,  2006),  their 
relationships  to  real  communities  (Wellman  and  Gulia, 
1999),  and  the  relative  importance  of  social  support  to 
information  support  (Preece,  1999).  Communities  of 
practice  relate  to  collections  of  people  with  the  same 
goals,  coming  together  to  share  knowledge  about  how 
best  to  achieve  them.  Learning  and  information  sharing 
is  situated  in  the  context  of  work  practices.  There  is 
particular  interest  in  how  communities  of  practice  are 
facilitated  through  virtual  communities  (Davenport 
and  Hall,  2002).  Research  into  online  communities  of 
practice  is  often  focused  on  their  voluntary  and  self- 
sustaining  nature  (Wenger  and  Snyder,  2000). 

Although  most  work  in  knowledge  systems  is  built 
around  knowledge  within  the  organization,  several  tech¬ 
nologies  have  had  a  particular  effect  on  the  way  that 
people  form  networks,  communicate,  and  share  knowl¬ 
edge  in  everyday  life.  In  particular,  social  networking 
websites  such  as  myspace.com  (Boyd  and  Heer,  2006) 
and  instant  messaging  communications  (Grinter  and 
Palen,  2002)  are  having  an  increasing  impact. 

Expert  systems,  knowledge  management,  and 
knowledge  communities  are  not  necessarily  exclusive 
approaches.  It  is  possible  to  create  hybrid  systems. 
Communities  can  share  tacit  knowledge,  commentary, 
and  experience  within  the  framework  of  more  formal 
structures.  Jubert  (1999)  described  such  a  system  in  the 
context  of  business  process  innovation  at  Siemens  Busi¬ 
ness  Services  (SBS)  in  France  and  highlighted  the  inter¬ 
play  of  virtual  communities,  communities  of  practice, 
and  traditional  knowledge  management.  Weber  and 
Kaplan  (2003)  described  the  interrelationships  between 
knowledge-based  systems  and  knowledge  management. 
Douglas  (2004)  described  how  configurable  software 
architecture  enables  hybrid  knowledge  systems.  The 
software  architecture  was  used  to  create  a  hybrid  system 
that  incorporated  knowledge  management  and  online 
communities  using  an  organizational  framework  of  per¬ 
formance  roles  and  goals  (Sasson  and  Douglas,  2006). 


CROSS-CUTTING  ISSUES 

Cross-cutting  issues  for  knowledge  systems  include 
how  knowledge  is  derived  from  human  sources;  how 
knowledge  is  encoded  in  a  machine-readable  form; 
how  knowledge  is  organized  using  taxonomies,  meta¬ 
data,  and  ontology;  and,  finally,  knowledge  quality 
assurance. 

Populating  a  System  with  Knowledge 

A  knowledge  system  is  only  as  useful  as  the  knowledge 
it  contains,  so  an  important  first  step  in  creating  any 
system  is  obtaining  the  knowledge  from  existing 
sources.  The  process  of  doing  this  is  known  as  knowl¬ 
edge  elicitation,  and  it  has  often  been  viewed  as  a  bot¬ 
tleneck  for  the  creation  of  usable  systems.  The  issues 
involved  will  be  familiar  to  instructional  designers  who 
also  have  to  derive  knowledge  from  subject-matter 
experts.  The  study  of  the  effectiveness  of  different  tech¬ 
niques  is  relevant.  In  addition  to  traditional  techniques 
such  as  interviews  and  surveys,  more  sophisticated  tech¬ 
niques  are  often  required  for  knowledge-based  systems 
given  that  they  are  focused  on  more  depth  of  analysis 
and  are  more  often  required  to  extract  tacit  rather  than 
explicit  knowledge.  Among  the  relevant  techniques 
studied  are  cognitive  task  analysis  (see  Chapter  43  in 
this  volume),  protocol  analysis  (Ericsson  and  Simon, 
1993),  diagrammatic  techniques  (Cheng,  1996;  see  also 
Chapter  28  in  this  volume),  and  automated  techniques 
(Shute  et  al.,  1999;  see  also  Chapter  24  in  this  volume). 

Ethnographic  studies  are  another  way  to  learn 
about  how  knowledge  workers  both  apply  their  knowl¬ 
edge  and  organize  information  to  support  their  work. 
An  example  of  this  approach  is  the  work  of  Gabbay 
and  le  May  (2004),  who  spent  2  years  observing  pri¬ 
mary-care  clinicians  in  a  medical  practice  and  con¬ 
cluded  that  the  clinicians  rarely  used  explicit  evidence 
from  research  or  other  sources  but  instead  relied  on 
collectively  reinforced  tacit  guidelines  which  the 
authors  termed  mindlines.  Reading  played  some  part 
in  the  formation  of  mindlines,  but  they  were  mainly 
formed  by  interactions  with  patients,  colleagues,  and 
pharmaceutical  representatives.  Once  formed,  they 
were  shared  and  negotiated  into  socially  constructed 
knowledge  within  the  practice. 

In  addition  to  studying  humans  and  focusing  on 
tacit  knowledge,  studies  can  be  made  of  recorded 
online  interactions  and  existing  formal  representations 
of  knowledge  such  as  documents.  This  can  involve 
human  study  and  coding  of  items  such  as  e-mail  mes¬ 
sages,  discussion  boards,  or  instant  messaging  tran¬ 
scripts.  It  can  also  involve  automated  systems  of  anal¬ 
ysis.  One  important  technique  in  this  regard  is  the  use 
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of  latent  semantic  analysis  (LSA)  (Landauer  et  al., 
1998),  which  extracts  the  contextual  meaning  of  words 
by  statistically  analyzing  large  bodies  of  text  within  a 
particular  domain. 

In  collaborative  knowledge-sharing  systems,  a 
major  issue  is  what  motivates  contributors  to  share 
when  there  is  seldom  a  financial  reward  for  contribut¬ 
ing  their  knowledge  .  The  development  of  online  com¬ 
munities  is  seen  as  a  mix  of  self-interest  and  altruism 
(Kollick,  1999),  and  various  mechanisms  have  been 
adopted  to  encourage  participation  such  as  community 
recognition  (Wellman  and  Gulia,  1999).  Jeong  (2006) 
performed  an  analysis  of  the  language  used  in  threaded 
discussions  in  an  online  class  and  found  that  the  use 
of  conversational  language  was  an  important  factor  in 
increasing  responses  within  the  dialog. 

Encoding  Knowledge 

Once  knowledge  is  derived  from  a  source,  it  must  be 
encoded  in  the  system  in  such  a  way  as  to  be  accessible 
by  software  such  as  inference  engines,  search  engines, 
and  knowledge-management  classification  schemes.  In 
talking  about  expert  systems,  we  introduced  one  form 
of  knowledge  representation:  the  production  rule.  This 
is  a  popular  method  of  representing  procedural  knowl¬ 
edge.  In  addition  to  the  procedural  knowledge,  we  must 
also  represent  declarative  knowledge  or  factual  infor¬ 
mation  about  entities  in  the  world.  This  is  often  repre¬ 
sented  in  hierarchical  or  networked  collections  of  nodes 
and  links.  In  artificial  intelligence,  this  knowledge  is 
often  encoded  as  semantic  networks  (Sowa,  1991), 
which  have  a  close  relationship  with  the  idea  of  concept 
maps  (see  Chapter  29).  Some  knowledge  encoding 
schemes  can  combine  declarative  and  procedural 
knowledge.  The  idea  behind  object-oriented  program¬ 
ming  (Weisfeld,  2003)  embodies  this  concept.  An  object 
embodies  attributes  (facts  about  the  object)  and  methods 
(procedural  code)  that  provide  the  behavior  of  the  object 
within  the  system.  Another  important  aspect  of  object- 
oriented  knowledge  systems  is  the  hierarchical  classifi¬ 
cation  of  objects.  Knowledge  classification  schemes  or 
knowledge  ontologies  have  become  a  very  important 
area  of  research,  particularly  for  those  interested  in 
working  toward  the  Semantic  Web  (Gomez-Perez  et  al., 
2004).  Ontology  is  a  shared  conceptualization  of  mean¬ 
ing,  and  several  technologies  exist  for  the  construction 
of  ontologies  to  assist  in  identifying  the  meaning  of 
content  on  the  Web  (Warren,  2006).  As  many  of  these 
technologies  are  difficult  to  work  with,  higher  level 
tools  have  also  emerged  to  assist  researchers  in  creating 
ontologies  for  different  domains.  A  prominent  example 
of  this  is  Protege  (see  http://protege.stanford.edu/),  a 
free,  open-source  tool  developed  at  Stanford  University. 


As  the  Internet  and  the  World  Wide  Web  are  seen 
as  the  key  platforms  for  knowledge  management  in  the 
future,  it  is  import  that  good  tools  and  standards  are 
developed  to  facilitate  this.  The  main  standards  body 
behind  the  Web  has  been  active  in  promoting  knowl¬ 
edge  representation  and  classification  technologies 
(see  http://www.w3.org/2001/sw). 

Ensuring  Knowledge  Quality 

There  has  always  been  concern  in  ensuring  the  quality 
of  the  knowledge  provided  by  knowledge  systems.  In 
the  early  expert  systems,  it  was  recognized  that  the 
user  might  not  trust  the  conclusion  provided  by  the 
system.  Expert  systems  evolved  systems  of  explana¬ 
tion  by  which  they  could  show  the  reasoning  behind 
their  decision  (Richards,  2003).  The  explanation  was 
seen  as  providing  confidence  for  the  users  that  a  con¬ 
clusion  was  based  on  good  reasoning.  In  early  expert 
systems,  explanation  was  merely  a  rule  trace,  but  later 
systems  evolved  to  provide  more  naturalistic  explana¬ 
tions  of  decision  making. 

A  pillar  of  conventional  quality  assurance  of 
knowledge  is  peer  review.  This  model  features  recog¬ 
nized  outlets  for  the  publication  of  knowledge,  but 
before  a  contribution  can  be  accepted  for  publication 
a  number  of  peers  must  attest  to  the  proposed  contri¬ 
bution’s  correctness  and  worthiness.  For  logistical  rea¬ 
sons,  the  number  of  reviewers  is  often  limited,  and  the 
process  of  editing  and  review  prior  to  publication  can 
be  time  consuming.  The  Internet  has  made  it  possible 
for  other  models  to  emerge,  particularly  the  move 
toward  publication  and  review  becoming  an  open,  con¬ 
tinuous,  and  collaborative  process. 

An  interesting  controversy  has  arisen  recently  over 
the  development  of  Wikipedia  (see  http://en.wikipe- 
dia.org/wiki/Wikipedia),  an  online  encyclopedia  devel¬ 
oped  by  a  voluntary  and  self-regulated  community. 
Some  have  argued  that,  without  traditional  expert-led 
and  peer-review  mechanisms,  Wikipedia  has  the  poten¬ 
tial  for  disseminating  false  knowledge  (Denning  et  al., 
2005).  Others  have  argued  that  such  a  system  leads  to 
more  efficient  and  refined  knowledge  sources  due  to 
the  openness  of  the  process  and  the  greater  numbers  of 
people  involved.  Surowiecki  (2003)  suggested  that, 
under  certain  conditions,  the  collective  wisdom  of  large 
groups  can  lead  to  more  effective  decision  making  than 
reliance  on  small  groups  of  experts. 

A  number  of  mechanisms  have  emerged  to  assist 
people  in  gauging  the  quality  of  resources.  Collectively, 
these  have  been  termed  recommender  systems  (Resnick 
and  Varian,  1997).  Any  mechanism  that  allows  a  user 
to  distinguish  the  quality  or  usefulness  of  a  resource 
can  be  considered  a  recommender  system.  The  most 
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prominent  type  of  recommender  system  is  sometimes 
referred  to  as  collaborative  filtering,  where  users  of 
resources  review  and  rate  them  for  the  benefit  of  oth¬ 
ers.  Automated  approaches  to  recommending  use 
knowledge  of  users’  stated  preferences  or  online 
behavior  to  construct  recommendations  for  other 
resources  to  view.  Burke  (2000)  provided  a  review  of 
attempts  to  integrate  knowledge-based  system 
approaches  into  recommender  systems.  One  example 
of  such  a  system  is  Pandora  (see  www.pandora.com), 
a  development  of  the  music  genome  project.  Pandora 
uses  a  knowledge  base  that  classifies  thousands  of 
pieces  of  music  based  on  hundreds  of  musical 
attributes.  When  a  user  enters  the  name  of  an  artist  or 
song.  Pandora  generates  a  radio  station  that  plays  a 
range  of  artists  and  songs  that  share  similar  attributes, 
thus  allowing  users  to  discover  new  music  they  might 
not  otherwise  have  encountered. 


DISCUSSION 

Many  new  technologies  generate  enthusiasm  and  inter¬ 
est  when  they  first  emerge  and  then  seem  to  fade  into 
the  mainstream.  Multimedia  is  a  good  example  of  this, 
and  one  of  the  reasons  for  highlighting  expert  systems 
in  this  chapter  is  that  it  also  followed  a  similar  trend. 
Since  the  initial  enthusiasm  and  interest  arose  in  the 
1980s,  work  in  this  area  has  now  merged  into  other 
interests  in  computing,  such  as  agent  technology.  It  is 
likely  that  a  number  of  the  technologies  noted  in  this 
chapter  will  adapt  and  merge  into  hybrid  systems  that 
are  part  of  the  more  general  phenomenon  of  networked 
computer  systems. 

In  addition  to  the  enduring  themes  we  have  cov¬ 
ered-how  we  can  extract,  codify,  classify,  and  quality 
check  knowledge  within  knowledge  systems  — some 
other  trends  are  apparent.  The  first  is  a  move  from  the 
document  as  the  main  unit  of  knowledge  transfer  to 
more  granular  digital  units.  Many  readers  will  be 
familiar  with  learning  objects.  In  other  areas,  the  object 
concept  is  also  being  adapted.  The  technical  data  doc¬ 
umentation  community,  which  creates  content  that  is 
often  used  by  instructional  designers,  has  adopted  an 
object  standard  for  making  technical  data  documenta¬ 
tion  more  granular  (S1000D;  see  http://  www.slOOOd. 
org /). 

Object  standards  are  the  first  step  before  creating 
digital  repositories  through  which  people  can  access 
and  share  such  knowledge  resources.  The  integration 
of  collaboration  and  recommender  systems  with  these 
repositories  is  the  next  step  toward  achieving  knowl¬ 
edge  systems  that  both  serve  and  harness  the  efforts 
of  entire  communities  rather  than  single  organizations. 


Network-centric  ways  of  operating  will  have  the 
most  effect  on  human  activities  that  currently  use  non¬ 
digital,  document-based,  or  linear  processes.  One  par¬ 
ticularly  prominent  example  of  this  is  the  recent  U.S. 
National  Science  Foundation  initiative  to  develop 
research  into  a  cyber  infrastructure  for  science  and 
engineering  (National  Science  Foundation,  2005).  This 
program  is  intended  to  stimulate  research  in  the  use  of 
the  Internet  to  facilitate  scientists  and  engineers  in 
better  sharing  resources  and  better  managing  the  devel¬ 
opment  and  archiving  of  knowledge.  Central  standards 
and  repositories  of  scientific  knowledge  with  data- 
sharing  standards,  open  discussion,  and  mass  peer 
review  are  seen  as  the  future  in  this  domain. 


CONCLUDING  REMARKS 

Individual  knowledge  systems  in  their  various  forms 
have  experienced  varying  degrees  of  success  and 
impact;  however,  the  development  of  the  ultimate 
knowledge  system,  the  World  Wide  Web,  has  had  an 
impact  on  the  way  people  create,  obtain,  and  share 
knowledge  .  The  development  of  specific  technologies 
for  knowledge  systems  is  likely  to  continue  at  a  searing 
pace,  despite  understandable  anxieties  about  the  effect 
of  these  technologies  on  society.  Underlying  issues 
that  researchers  will  have  to  address  include:  (1)  elic¬ 
iting,  representing,  structuring,  accessing,  and  verify¬ 
ing  the  quality  of  knowledge,  and  (2)  evaluating  the 
effects  of  technology  on  individuals,  organizations, 
and  society.  These  issues  overlap  the  traditional  con¬ 
cerns  of  those  in  the  field  of  instructional  design  who 
have  a  great  deal  to  contribute  to  research  in  knowl¬ 
edge  systems.  Indeed,  it  is  possible  to  argue  that  learn¬ 
ing  systems  and  knowledge  systems  should  not  be  seen 
separately  but  as  integral  components  of  systems  for 
supporting  human  performance  and  the  development 
of  human  society. 
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ABSTRACT 

The  point  of  departure  for  this  chapter  is  the  idea  that 
learning  activity  is  becoming  less  constrained  by  time, 
space,  and  the  organizational  requirements  of  educa¬ 
tional  providers.  As  people  take  more  control  over 
their  learning  activity,  there  is  a  multiplication  of  the 
possible  influences  of  time  and  space  on  their  learning. 
If  learning  can  take  place  anywhere,  then  we  need  to 
know  more  about  the  pedagogical  affordances  of  the 
various  wheres  in  which  it  is  situated.  This  becomes 
a  priority  for  the  effective  learner— needing  to  know 
how  to  select  and  configure  appropriate  learn- 
places— as  well  as  for  researchers  trying  to  understand 
the  sometimes  subtle  connections  between  place  and 
learning.  The  chapter  offers  a  summary  description  of 


flexible  learning  and  then  moves  on  to  consider  the 
relations  between  learning  and  place.  The  account  is 
structured,  in  part,  by  a  distinction  between  weaker 
and  stronger  interpretations  of  what  is  meant  by  situ¬ 
ated  learning.  The  function  of  the  weaker  interpreta¬ 
tion  is  to  highlight  the  importance  of  the  learning 
context,  such  that  we  can  focus  on  the  affordances  of 
the  learnplace  and  consider  the  ergonomics  of  support¬ 
ive  learning  environments.  The  stronger  interpretation 
of  situated  learning  causes  us  to  take  seriously  the  idea 
that  being  a  learner  is  first  and  foremost  engagement 
in  a  cultural  practice,  that  people  have  to  learn  to 
engage  in  such  cultural  practices,  and  that  place  has  a 
distinctive  role  in  cultural  practice.  Paradoxically, 
ideas  about  flexible  learning  and  mobile  learning  help 
us  put  learning  in  its  place. 
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KEYWORDS 

Ergonomics  of  learning  environments:  The  applied  sci¬ 
ence  that  helps  illuminate  the  relationships 
between  a  learner  and  a  learnplace. 

Flexible  learning:  Learning  that  is  relatively  free  of 
logistical  and  educational  constraints. 

Learning  places  (or  learnplaces):  The  immediate 
physical  setting  for  someone’s  learning  activity, 
including  the  tools  and  artifacts,  digital  and  mate¬ 
rial,  that  come  to  hand. 

Situated  learning:  A  perspective  on  learning  that 
emphasizes  its  social  and  physical  context. 


INTRODUCTION 

Categorizations  of  instructional  approaches  and  of  the 
experience  of  learning  are  rarely  symmetrical.  A 
worldview  that  makes  perfect  sense  to  the  teacher  or 
instructional  systems  designer  may  well  prove  unrec¬ 
ognizable  to  the  learner.  This  issue  is  particularly 
thorny  when  we  talk  about  flexible  learning,  distance 
education,  or  blended  learning.  Who  bends  what  in 
flexible  learning?  Where  is  the  distance  in  distance 
education?  What  gets  blended  in  blended  learning? 

This  chapter  is  aimed  at  doing  two  things.  At  a 
substantive  level,  it  introduces  some  key  ideas  and 
literature  about  flexible  learning,  the  situated  nature  of 
learning,  and  the  relations  between  learning  and  learn¬ 
places.  At  a  meta-level,  the  chapter  tracks  and  com¬ 
bines  some  arguments  for  a  richer  conception  of  what 
is  involved  in  being  a  learner.  This  is  not  an  area  with 
a  rich  base  of  empirical  literature  (Van  Note  Chism, 
2002);  consequently,  many  of  the  references  are 
intended  to  help  the  reader  explore  better  ways  of 
conceptualizing  the  field.  Some  suggestions  for  prom¬ 
ising  lines  of  research  can  be  found  at  the  end  of  the 
chapter. 

The  term  learnplace  is  used  interchangeably  with 
learning  place  in  this  chapter,  depending  on  the  flow 
of  the  text.  The  two  mean  the  same.  Learnplace,  by 
analogy  with  workplace,  is  a  place  at  which  serious 
learning  is  done.  Ford  and  colleagues  (1996)  and  Slack 
and  colleagues  (1996)  provided  the  earliest  recorded 
usages  of  the  term.  Place  is  used  deliberately  as  a  way 
of  indicating  something  more  specific  and  concrete 
than  space  — something  imbued  with  meaning  and 
value  (Auburn  and  Barnes,  2006;  Jamieson  et  al., 
2000;  Sime,  1986). 

Flexible  learning  connotes  those  learning  situa¬ 
tions  in  which  the  learner  has  substantial  control  over 
the  logistics  of  learning,  notably  the  location  and  tim¬ 
ing  of  the  learner’s  learning  activity.  It  can  also  connote 


situations  in  which  learners  have  substantial  control 
over  what  they  set  out  to  learn,  how  they  go  about 
learning  it,  and  how  their  learning  is  assessed.  In  short, 
flexible  learning  involves  the  loosening  of  logistical  or 
educational  constraints  (Boot  and  Hodgson,  1987). 
The  focus  on  learning  places  reflects  a  growing 
acknowledgment  of  the  importance  of  the  physical 
environment  in  influencing  how  people  learn  and  what 
they  learn  (Bliss  et  al.,  1999;  Poysa  et  al.,  2005;  Sin¬ 
gleton,  1998).  This  is,  in  part,  connected  with  trends 
toward  more  flexible  arrangements  for  learning 
(Jamieson  et  al.,  2000;  Monahan,  2002).  If  one  con¬ 
ceives  of  learning  as  that  which  takes  place  in  a  time- 
delimited  lesson  in  a  teacher’s  classroom,  then  the 
relations  between  learning  and  place  seem  bounded 
and  obvious— part  of  what  we  take  for  granted  in  edu¬ 
cational  practice  even  though  the  physical  layout  of  a 
classroom  may  impose  strong  constraints  on  pedagogy 
(Comber  and  Wall,  2001;  Graetz  and  Goliber,  2002; 
Van  Note  Chism,  2002).  Shifts  toward  anytime,  any¬ 
where,  anyhow  learning  can  be  accompanied  by  two 
competing  views  about  place:  that  place  becomes  irrel¬ 
evant  (the  mobile  fantasy)  or  that  the  qualities  of  place 
must  be  understood  in  a  more  nuanced  way.  The  sec¬ 
ond  main  section  of  the  chapter  examines  this  through 
a  consideration  of  ideas  in  the  literature  about  weaker 
and  stronger  interpretations  of  what  it  means  for  learn¬ 
ing  to  be  situated.  The  implications  of  these  two  inter¬ 
pretations  are  drawn  out  in  the  third  and  fourth  sec¬ 
tions,  which  focus  on  physical  contexts  for  learning 
and  learning  conceived  as  participation  in  cultural 
practice,  where  the  culture  provides  physical  and  dig¬ 
ital  resources  that  influence  the  very  nature  of  learning 
activity. 


FLEXIBLE  LEARNING 

...space  and  time  are  not  just  material  constraints  to  be 
overcome  or  resources  to  be  used;  rather,  they  have  dif¬ 
ferent  qualities  depending  on  the  historically  specific  dis¬ 
course  through  which  they  are  understood.  Thus,  new 
interpretive  frames  by  which  space  and  time  are  under¬ 
stood  are  integral  to  the  creation  of  institutions  which 
achieve  new  forms  of  control  over  the  spacing  and  timing 
of  human  activity.  (Friedland  and  Boden,  1994,  p.  29) 

The  literature  on  flexible  learning  is  of  two  kinds.  One 
kind  is  essentially  functional.  In  broad  terms,  it  takes 
flexibility  for  granted  as  a  good  thing  and  aims  to 
provide  practical  knowledge  and  advice  about  how  to 
promote  and  support  more  flexible  forms  of  learning 
activity  (de  Boer  and  Collis,  2005;  Khan,  2006;  Lock- 
wood  and  Latchem,  1997;  Mason,  1994;  Van  den 
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Brande,  1993;  Wade  et  al„  1994).  The  other  kind  takes 
a  more  critical  perspective.  Much  of  it  has  the  goal  of 
demonstrating  that  intended  benefits  are  not  always 
achieved  or  that  the  liberationist  discourse  of  flexibility 
conceals  some  hidden  or  unanticipated  costs  to  stu¬ 
dents  (Edwards,  1997;  Edwards  and  Clark,  2002; 
Paechter  et  al„  2001;  Willems,  2005).  Both  perspec¬ 
tives  are  necessary.  Indeed,  some  of  the  practical  guid¬ 
ance  provided  in  the  literature  would  be  more  useful 
if  it  took  a  less  romantic  or  naive  view  of  the  charms 
of  flexibility.  Lieve  Van  den  Brande’s  (1993,  p.  2) 
definition  of  flexible  learning  still  serves  us  well: 
"Flexible  learning  is  enabling  learners  to  learn  when 
they  want  (frequency,  timing,  duration),  how  they  want 
(modes  of  learning),  and  what  they  want  (that  is,  learn¬ 
ers  can  define  what  constitutes  learning  to  them).” 

Flexible  Use  of  Time  and  Space 

Temporal  flexibility  implies  that  learning  is  scheduled 
at  times  to  suit  the  learner.  This  might  include  control 
over  the  deadline  for  achieving  a  learning  objective, 
the  duration  and  frequency  of  learning  activities,  and 
so  on.  At  a  macro-level  (time  spans  measured  in  years), 
temporal  flexibility  includes  such  considerations  as 
lifelong  learning  (i.e.,  that  learning  is  not  just  restricted 
to  the  years  of  formal  schooling).  It  refers  to  the  free¬ 
dom  to  learn  when  the  need  arises  in  the  course  of 
one’s  life,  not  just  at  the  stages  predetermined  for  the 
convenience  of  educational  providers.  Meso-level  tem¬ 
poral  flexibility  operates  on  time  spans  of  weeks  and 
months.  The  ability  to  start  a  course  of  study  at  any 
time  in  the  year,  rather  than  (say)  just  at  the  beginning 
of  a  semester,  would  be  an  example  of  meso-level 
flexibility.  Micro-level  flexibility  operates  on  a  time 
scale  of  days  down  to  seconds.  Flexibility  on  this  scale 
allows  the  learner  detailed  control  over  the  scheduling 
and  pace  of  their  learning  activity,  allowing  them,  for 
example,  to  "harvest  fragments  of  time”  for  learning 
in  an  otherwise  hectic  schedule  (Roberts  et  al.,  2003). 

The  spatial  dimension  of  flexible  learning  refers 
to  learning  that  takes  place  at  locations  that  suit  the 
learner.  Flexible  learning  should  minimize  the  disrup¬ 
tion  to  the  learner’s  other  activities  (at  work,  at  home, 
etc.).  Fearning  that  involves  the  learner’s  absence 
from  work  or  home  may  have  the  advantage  of  allow¬ 
ing  the  learner  to  concentrate  fully  on  their  learning 
activity,  but  it  can  also  disadvantage  those  who  are 
tied  to  the  home  or  workplace  and  can  be  expensive 
for  both  the  learner  and  their  employer.  Equally, 
enabling  learners  to  study  from  their  workplaces  can 
help  contextualize  learning  and  dissolve  unhelpful 
boundaries  between  work  and  learning  (Goodyear, 
2006;  Van  den  Brande,  1993). 


Flexibility  over  Goals, 

Methods,  and  Assessment 

Flexibility  over  learning  goals  means  that  learning 
activity  can  have  its  roots  in  the  needs  and  interests  of 
the  learner  and  emerge  from  (or  be  inseparable  from) 
their  ongoing  work  and  life.  Much  conventional  learn¬ 
ing  activity  is  strongly  shaped  by  the  existence  and 
nature  of  assessment  systems  or  by  the  goals  of  teach¬ 
ers  and  educational  institutions.  Flexible  learning  may 
be  relatively  free  from  such  influences. 

The  learner  should  take  a  lead  in  defining  what  it 
is  they  need  to  know.  They  may  be  helped  in  this  by 
a  teacher  (or  by  other  learners)  who  may  have  knowl¬ 
edge  about  good  ways  of  refining  and  reaching  the 
learner’s  objectives.  Definition  of  needs  can  occur  at 
a  number  of  levels;  the  learner  may  define  a  high-level 
need  and  pass  over  to  a  teacher  the  lower  level  deci¬ 
sions  about  how  this  need  can  be  met.  Conversely,  the 
higher  level  decisions  may  be  made  by  a  teacher  or  a 
competence-certifying  institution,  while  control  over 
the  details  of  what  is  learned  stays  with  the  learner. 

In  a  flexible  learning  regime,  learners  should  exert 
a  high  level  of  control  over  how  they  study:  whether 
reading  texts  or  listening  to  lectures  or  through  prac¬ 
tical  work.  There  is  a  strong  impetus  to  recognize  the 
learner’s  need  for  autonomy,  but  this  is  not  to  say  that 
the  unsupported  learner  is  always  able  to  make  good 
decisions  about  their  learning  or  that  they  should  be 
dissuaded  from  asking  for  advice. 

Flexible  learning  should  make  learning  opportuni¬ 
ties  available  to  all  those  who  wish  to  take  advantage 
of  them.  This  means  doing  away  with  entry  qualifica¬ 
tions.  Fess  strongly,  it  means  removing  barriers  that 
have  no  clearly  demonstrable  causal  link  with  the 
learner’s  benefiting  from  the  learning  experience. 

Use  of  self-study  materials,  such  as  books,  articles, 
audio  and  video  tapes,  computer  programs,  and  so  on, 
is  frequently  associated  with  flexible  learning.  This  is 
largely  a  byproduct  of  the  economics  and  social  orga¬ 
nization  of  education— few  people  can  afford  on- 
demand  access  to  teachers  and  other  subject-matter 
experts.  Nevertheless,  there  is  no  principled  reason 
why  flexible  learning  should  involve  the  use  of  self- 
study  materials. 

Flexible  learning  is  also  often  thought  of  as  a  sol¬ 
itary  activity.  In  so  far  as  this  is  true,  it  is  a  byproduct 
of  distance-taught  correspondence  courses,  models  for 
flexible  learning,  and  the  costs  (financial  and  other)  of 
communication  between  dispersed  learners.  Flexible 
learning  need  not  be  a  solitary  activity.  It  often  is  so, 
because  of  the  methods  that  institutions  have  adopted 
for  supporting  flexible  learning  or  because  the  learner 
is  learning  without  reference  to  an  institution.  A  tension 
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also  exists  between  individual  flexibility  and  the  cir¬ 
cumstances  most  conducive  to  collaborative  learning. 
Where  individual  learners  choose  to  have  a  high  level 
of  control  over  the  timing  of  their  study,  for  example, 
it  can  be  very  difficult  for  them  to  engage  in  collabo¬ 
rative  activities  such  as  seminars  (which  may  benefit 
from  participants  being  at  a  common  point  in  their 
learning).  Resolving  this  tension  between  individual 
and  group  interests  can  be  a  difficult  challenge. 

WEAK  AND  STRONG 
INTERPRETATIONS 
OF  SITUATEDNESS 

We  now  need  to  turn  to  place.  Jean  Lave  and  Etienne 
Wenger  are  to  be  credited  for  reminding  us  that  learn¬ 
ing  is  socially  and  physically  situated  (Lave,  1988; 
Lave  and  Wenger,  1991;  Rogoff  and  Lave,  1984), 
although  a  number  of  other  authors  must  also  be 
acknowledged  in  considering  the  renaissance  of  this 
idea,  such  as  Hutchins  (1995),  Scribner  (1985),  and 
Suchman  (1987).  The  force  of  situatedness  varies 
somewhat  in  different  accounts,  but  Engestrom  (1999) 
neatly  arranges  these  into  weaker  and  stronger  inter¬ 
pretations.  The  weak  version  asserts  that  learning  is 
situated  in  physical  and  social  contexts  so  context  must 
be  taken  into  account  when  one  studies  learning.  Lor 
examples  of  this  position,  see  Greeno  (1989)  or  Barab 
and  Plucker  (2002).  The  strong  interpretation— exem¬ 
plified  in  Lave  and  Wenger’s  work— is  that  learning  is 
a  byproduct  of  participation  in  a  social  practice,  that 
such  social  practices  do  not  have  to  be  explicitly 
defined  as  practices  of  learning,  and  that  one  should 
focus  on  the  practice  rather  than  the  context.  Thinking 
about  weak  and  strong  interpretations  of  physical  sit¬ 
uatedness  helps  us  partition  the  relevant  literature  on 
learning  places.  The  weak  (contextual)  interpretation 
invokes  notions  of  ergonomics  and  affordance.  The 
stronger  (social  practice)  interpretation  raises  deeper 
questions  about  being  and  becoming  a  learner  and  the 
role  of  place  in  knowledge  practices.  These  two  sub- 
themes  are  reviewed  in  the  next  two  sections. 


THE  ERGONOMICS  OF 
LEARNING  ENVIRONMENTS 

The  work  of  John  Sweller  (this  volume),  Richard  Mayer 
(2001),  and  others  on  the  implications  of  cognitive  load 
theory  for  learning  from  multimedia  is  a  useful  starting 
point  for  thinking  about  the  design  of  resources  that 
support  efficient  learning.  This  body  of  work  represents 
the  micro-level  pole  of  a  continuum  for  which  the 


macro-level  pole  is  the  global  environment  within  which 
a  learner’s  activity  is  situated.  A  focus  on  leamplaces 
suggests  the  meso- level,  but  one  must  acknowledge 
straight  away  that  leamplaces  are  constituted  by  artifacts 
of  the  micro-level  and  situated  within  influential,  spa¬ 
tially  more  extensive  contexts.  The  ways  in  which  arti¬ 
facts  and  tools  (objects  at  the  micro-level)  exert  influ¬ 
ence  upon  learning  is  commonly  conceptualized  using 
notions  of  affordance.  This  construct  originated  in  the 
ecological  psychology  of  Gibson  (1977)  and  was  trans¬ 
planted  to  educational  technology  via  human-computer 
interaction  (Norman,  1990,  1999),  suffering  some 
sleight  of  hand;  it  is  now  widely  used  to  connote  pro¬ 
cesses  of  influence  rather  than  causation— as  when  one 
implies  that  the  features  of  a  tool  or  other  artifact  suggest 
one  usage,  or  one  course  of  action,  rather  than  another 
(see  Chapter  22  in  this  volume). 

At  the  meso-level,  where  we  can  deal  appropriately 
with  the  qualities  of  a  leamplace,  it  may  be  more  pro¬ 
ductive  to  focus  on  ergonomics  rather  than  the  logic  of 
affordance.  The  case  for  an  ergonomics  of  learning 
environments  can  be  found  in  Goodyear  (1997,  2000). 
The  essence  of  this  perspective  is  that  the  design  of 
educational  technologies  is  best  informed  by  an  under¬ 
standing  of  the  actuality  of  learners’  work.  One  reason 
why  educational  technologies  are  ignored  by  learners 
is  that  they  do  not  fit  with  the  learners’  actual  work. 
They  reflect  teachers’  views  of  the  prescribed  tasks 
rather  than  learners’  real-world  activity.  An  understand¬ 
ing  of  leamplace  qualities  can  then  be  conceived  as  a 
kind  of  cognitive  anthropology  of  situated  learning. 

BEING  A  LEARNER:  PUTTING 
LEARNING  IN  ITS  PLACE 

Llexible  use  of  space  (mobile  learning)  means  that 
place  cannot  be  used  to  delimit  the  social  practices  of 
learning:  If  one  can  leam  anywhere,  what  does  it  mean 
to  be  a  learner?  Probing  at  this  question  reveals  some 
interesting  answers— not  least,  that  most  of  the  social 
practices  we  associate  with  learning  are  subtly  influ¬ 
enced  by  place.  As  an  example,  take  the  work  of 
Charles  Crook  and  Paul  Light  on  campus-based  stu¬ 
dents’  engagement  in  learning  with  information  and 
communication  technology  (ICT)  (Crook,  2002; 
Crook  and  Barrowcliff,  2001;  Crook  and  Light,  1999, 
2002).  This  program  of  research  looked  at  differences 
and  similarities  in  the  study  practices  of  undergraduate 
students  from  the  same  university,  half  of  whom  had 
computers  with  high-speed  network  connections  in 
their  study  bedrooms  and  half  of  whom  did  not.  Sim¬ 
ilarities  in  the  study  practices  of  the  two  groups  help 
us  identify  some  of  the  subtle  influences  of  the  material 
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world  and  of  the  patterns  of  interaction  and  commu¬ 
nication  lodged  in  that  material  world  (Crook  and 
Light,  2002,  p.  174): 

Deliberate  learning  involves  engaging  with  exposition, 
orchestrated  discussion,  research,  systematic  annotation, 
the  focused  reading  of  text,  and  a  variety  of  other  directed 
activities  that  many  students  may  not  always  find  easy  to 
mobilize  and  manage  independently.  Sites  of  formal  edu¬ 
cation  have  evolved  structures  that  sustain  and  coordinate 
such  activities  with  a  scaffold  of  cultural  resources:  time¬ 
tables,  curricula,  designed  spaces,  discourse  rituals  and 
so  on  ...  making  progress  within  this  infrastructure 
amounts  to  a  process  of  enculturation.  Students  are  con¬ 
fronted  with  the  various  arenas  of  study  as  formalized 
versions  of  activities  well  rehearsed  in  their  informal  lives. 

A  good  example  is  private  study.  An  expectation 
of  higher  education  is  that  engagement  in  private  study 
will  be  focused  and  sustained.  It  is  easy  to  miss  the 
way  in  which  the  achievement  of  such  study  depends 
on  the  design  and  cultivation  of  various  institutional 
spaces  and  practices  (such  as  study  bedrooms  and  quiet 
library  spaces) .  ICT  can  break  down  some  of  the  help¬ 
ful  insulation  between  the  protecting  spaces  and 
orderly  practices  conducive  to  difficult  study  and  the 
personal  spaces  and  informal  practices  familiar  from 
everyday  life  (Crook  and  Light,  2002). 

Place  matters  in  other  ways.  Places  are  home  to 
artifacts,  to  a  greater  or  lesser  degree.  From  a  socio¬ 
cultural  perspective,  artifacts  are  embodiments  of 
ideas,  concepts,  and  methods.  They  are  the  “manufac¬ 
tured  objects  that  silently  impregnate  the  furniture  of 
the  world  with  human  intelligence”  (White,  1996,  p. 
xiii).  Not  just  books  and  computers  but  also  rulers, 
watches,  notepads,  and  Post-it  notes  enable  some 
forms  of  knowledge  work  by  “being  to  hand.”  The 
ability  to  offload  memory  to  artifacts  in  the  leamplace 
is  a  strong  influence  on  what  is  cognitively  possible, 
as  well  as  a  skill  to  be  mastered  (Perkins,  1993).  To 
assert  that  one’s  PDA  can  do  all  these  things  is  to 
trample  roughshod  over  the  subtle  qualities  of  the 
leamplace  (Crook  and  Barrowcliff,  2001). 


FURTHER  RESEARCH 

Three  areas  of  research  look  particularly  promising. 
First,  we  need  more  of  the  painstaking,  anthropological 
studies  that  can  help  us  understand  the  subtle  influ¬ 
ences  of  place  on  study  practices  and  outcomes,  along 
the  lines  of  the  work  by  Crook  and  colleagues  (Crook, 
2002;  Crook  and  Barrowcliff,  2001;  Crook  and  Light, 
2002).  A  satisfactory  program  of  study  would  embrace 
both  familiar  and  emerging  leamplaces  and  would 


include  study  practices  associated  with  new  mobile 
technologies  and  ubiquitous  intelligence  (Benford, 
2005;  Sharpies,  2000).  Second,  we  need  ways  to  cap¬ 
ture,  formalize,  and  share  knowledge  about  recurring 
patterns  in  the  physically  situated  activities  associated 
with  study  and  learning,  such  that  we  can  help  design¬ 
ers,  teachers,  and  learners  develop  a  better  practical 
understanding  of  how  to  improve  leamplaces.  The  peo¬ 
ple  most  closely  involved  in  such  matters  do  not  have 
shared  language  or  constructs  to  deal  with  phenomena 
that  are  of  central  importance  to  them,  such  as  learning, 
study,  the  ergonomics  and  affordances  of  physical 
space,  and  digital  tools.  One  promising  line  of  work 
here  is  represented  by  a  resurgence  of  interest  in  the 
analytic  methods  and  writing  of  the  architect  Christo¬ 
pher  Alexander  on  design  patterns  and  pattern  lan¬ 
guages  (see,  for  example,  Alexander  et  al.,  1977;  Fri- 
zell  and  Hubscher,  2002;  Goodyear,  2005).  Third,  we 
need  to  understand  students’  experiences  of  learning 
in  situations  where  they  are  taking  greater  control  over 
the  time  and  location  of  their  learning  activity.  Phe- 
nomenographical  research  methods  are  highly  appro¬ 
priate  here  (Marton,  1981;  Prosser  and  Trigwell, 
1999),  as  this  research  tradition  has  probably  given  us 
deeper  insights  into  the  nature  and  variation  in  stu¬ 
dents’  experiences  of  learning  than  any  comparable 
approach.  Phenomeno graphic  research  is,  however, 
remarkably  quiet  about  the  effects  of  place  on  student 
experience. 

CONCLUDING  REMARKS 

Technological  change  allows  a  shift  in  the  expecta¬ 
tions,  practices  and  discourse  around  the  location  of 
activity  in  time  and  space.  Although  flexible  learning, 
mobile  learning,  and  their  analogs  are  sometimes  held 
to  have  conquered  time  and  space,  paradoxically  they 
render  time  and  space  more  important.  This  chapter 
has  provided  an  entry  point  into  some  of  the  ideas  and 
literature  connecting  learning  to  place  and  has  sug¬ 
gested  some  points  of  departure  for  new  research. 
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ABSTRACT 

This  chapter  examines  technologies  that  enable  time  and 
place  independence.  Its  particular  focus  is  on  the  affor- 
dances  of  these  technologies:  their  ability  to  support 
self-paced  learning ,  offline  or  online,  and  group-based 
learning,  asynchronously  or  synchronously.  Self-paced 
learning  offline  is  a  mode  of  learning  that  enables  indi¬ 
viduals  to  study  with  the  help  of  portable  technologies 
in  their  own  time,  at  their  own  pace,  and  in  their  own 
place.  Technologies  that  support  this  mode  of  learning 
include  printed  books  and  a  whole  range  of  portable 
non-print  media.  Self-paced  learning  online  is  a  mode 


of  learning  that  enables  individuals  to  study  online  and 
in  their  own  time,  at  their  own  pace,  and  possibly  from 
their  own  place.  The  most  prominent  technologies  that 
support  this  model  of  learning  include  the  Internet  and 
various  other  computer-mediated  communications  tech¬ 
nologies.  Group-based  learning  asynchronously  is  a 
mode  of  learning  that  enables  individuals  to  learn  in 
groups  with  online  technologies  but  in  their  own  time, 
at  their  own  pace,  and  from  their  own  place.  Technolo¬ 
gies  that  support  this  mode  of  learning  include  online 
learning  management  systems,  mailing  lists,  bulletin 
boards,  Web  logs,  and  wikis.  Group-based  learning  syn¬ 
chronously  is  a  mode  of  learning  that  enables  individuals 
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to  learn  in  groups  with  online  technologies  at  the  same 
time  and  at  the  same  pace  as  that  of  the  group,  but  from 
different  places.  Commonly  known  technologies  that 
allow  this  kind  of  flexibility  are  audio  and  video  con¬ 
ferencing,  broadcast  radio  and  television,  and  newer 
technologies  such  as  Internet  telephony  (VoIP),  inter- 
relay  chat,  and  online  games  and  simulations. 


KEYWORDS 

Group-based  learning  asynchronously:  A  mode  of 
learning  that  enables  individuals  to  learn  in  groups 
with  online  technologies  in  their  own  time,  at  their 
own  pace,  and  from  their  own  place. 

Group-based  learning  synchronously:  A  mode  of 
learning  that  enables  individuals  to  learn  in  groups 
with  online  technologies  at  the  same  time  and  at 
the  same  pace  as  that  of  the  group  but  from  their 
own  place. 

Self-paced  learning  offline:  A  mode  of  learning  that 
enables  individuals  to  study  with  portable  technol¬ 
ogies  in  their  own  time,  at  their  own  pace,  and  from 
their  own  place. 

Self-paced  learning  online:  A  mode  of  learning  that 
enables  individuals  to  study  online  in  their  own 
time,  at  their  own  pace,  and  from  their  own  place. 


INTRODUCTION 

The  time  and  place  of  any  educational  activity  are  of 
significant  importance  to  learners  and  teachers  as  well 
as  the  educational  organization  that  is  offering  the 
activity.  They  are  of  interest  to  learners  because  of  their 
implications  for  when  and  where  they  need  to  be  to 
leam.  Teachers  are  similarly  affected  by  the  implica¬ 
tions  of  time  and  place  in  terms  of  when  they  must 
teach  and  where  they  need  to  be  to  teach.  The  time 
and  place  of  learning  and  teaching  activities  also  affect 
educational  organizations  in  terms  of  what  infrastruc¬ 
ture  and  resources  they  must  have  and  how  to  organize 
them  to  meet  the  requirements  of  where  and  when 
learning  and  teaching  must  take  place. 

Learning  and  teaching  activities  in  campus-based 
educational  settings  have  conventionally  been  regu¬ 
lated  by  time  and  place.  Learners  and  teachers  in  these 
educational  settings  are  expected  to  be  present  at  des¬ 
ignated  places  and  times  to  engage  in  the  educational 
activities.  In  so  doing,  this  mode  of  learning  and  teach¬ 
ing  imposes  constraints  on  both  learners  and  teachers 
who,  for  various  reasons,  are  unable  to  be  present  in 
a  required  place  and  at  the  appointed  time.  This  pre¬ 
vents  a  large  number  of  learners  from  participating  in 


their  educational  advancement  because  of  their  inabil¬ 
ity  to  be  present  at  a  particular  place  and  time.  The 
time  and  place  of  learning  impose  additional  con¬ 
straints  on  learners  who  are  able  to  access  campus- 
based  educational  provision  but  for  various  reasons  are 
unable  to  complete  their  learning  activities  within  a 
certain  time  frame. 


STRATEGIES 

The  constraints  of  the  time  and  place  of  learning  in 
campus-based  educational  settings  potentially  disad¬ 
vantage  a  wide  range  of  learners,  such  as  those  who 
are  in  regular  employment  or  committed  to  other  fam¬ 
ily  care  responsibilities,  who  are  physically  located  too 
far  away  from  the  educational  organization  or  source 
of  the  service,  who  are  too  poor  to  afford  the  various 
costs  of  campus-based  education,  or  who  lack  the  for¬ 
mal  qualifications  necessary  to  gain  entry  to  this  form 
of  education. 

Correspondence  Education 

In  most  educational  settings,  the  foregoing  situation  was 
found  to  be  unacceptable,  and  something  needed  to  be 
done  about  it.  The  growth  of  correspondence  education 
was  a  direct  result  of  an  effort  to  address  this  problem. 
By  capitalizing  on  two  technological  developments  of 
the  time— namely,  the  printing  press  and  the  postal  ser¬ 
vices— correspondence  education  was  able  to  offer  edu¬ 
cation  to  those  who  were  unable  to  access  it  in  campus- 
based  educational  settings.  Early  initiatives  with  corre¬ 
spondence  education  involved  individual  teachers  who 
were  trying  to  reach  small  numbers  of  learners  wherever 
they  were.  Notable  instances  included  the  teaching  of 
shorthand,  typing,  and  the  English  language.  The  suc¬ 
cesses  of  these  early  and  solitary  efforts  with  correspon¬ 
dence  education  led  to  established  educational  organi¬ 
zations  rapidly  adopting  this  approach  alongside  their 
campus-based  educational  programs. 

As  an  alternative  mode  of  learning  and  teaching, 
correspondence  education  developed  rapidly  and 
steadily  in  Europe,  Canada,  Australia,  New  Zealand, 
and  South  Africa  and  a  little  later  in  Asia  and  Africa. 
In  the  United  Kingdom,  for  example,  the  growth  of 
correspondence  education  was  spearheaded  by  politi¬ 
cal  will  and  with  the  establishment  of  the  United  King¬ 
dom  Open  University  by  the  Labor  Government.  In 
South  Africa,  the  need  for  correspondence  education 
was  driven  by  the  apartheid  government’s  divisive  pol¬ 
icies  of  racial  segregation  that  restricted  certain  racial 
groups  from  participating  in  mainstream  educational 
provision  leading  to  the  establishment  of  the  University 
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of  South  Africa  (UNIS A).  In  Asia  and  Africa,  the  driv¬ 
ers  of  correspondence  education  were  the  very  large 
numbers  of  people  (both  children  and  adults)  who 
needed  education  and  training,  as  well  as  the  inability 
of  many  of  them  to  afford  the  comparably  high  costs 
of  campus-based  education.  Currently,  numerous  edu¬ 
cational  organizations  all  over  the  world  use  corre¬ 
spondence  education  to  provide  educational  opportu¬ 
nities  to  many  students.  Many  of  these  efforts,  despite 
having  access  to  other  delivery  technologies,  continue 
to  rely  on  little  more  than  printed  study  materials  to 
offer  formal  education  to  very  large  numbers  of  learn¬ 
ers  throughout  the  world  via  postal  services  (Daniel, 
1996;  Keegan  and  Rumble,  1982). 

The  essence  of  correspondence  education  was  the 
asynchronous  nature  of  the  communication  between 
the  learners  and  their  teachers  or  the  educational  orga¬ 
nization  (Holmberg,  1995, 2001;  Moore,  1989;  Peters, 
1971).  This  model  of  learning  allowed  learners  to 
study  at  a  time,  pace,  and  place  that  most  suited  them 
or  their  situation,  regardless  of  where  their  teachers  or 
the  teaching  organization  may  have  been  located.  The 
acts  of  teaching  included  preparation  of  the  study 
materials  and  assignments  for  the  students  and  com¬ 
munication  with  them  through  feedback  and  comments 
on  the  assignments  that  they  submitted. 

Although  correspondence  education  offered  a  via¬ 
ble  solution  for  the  constraints  imposed  by  the  time, 
place,  and  pace  dependencies  of  campus-based  educa¬ 
tion,  it  had  many  problems.  The  absence  in  such  set¬ 
tings  of  various  kinds  of  learning  supports  and  services 
that  are  available  to  learners  in  campus-based  educa¬ 
tional  settings  caused  many  problems  for  learners  as 
well  as  teachers.  Some  of  these  obvious  learning  sup¬ 
ports  included  facilities  for  laboratory,  tutorial,  and 
small-group  work  and  various  forms  of  guidance  and 
counseling  services  that  students  often  need.  The 
inability  of  correspondence  education  to  provide  a 
comparable  level  of  such  learner  supports  caused  many 
learners  to  experience  serious  problems  with  their 
studies,  often  leading  them  to  abandon  their  studies 
altogether  (Amundsen  and  Bernard,  1989;  Simpson, 
2003;  Sweet,  1986;  Woodley  and  Parlett,  1983). 

Distance  Education 

To  remain  a  viable  educational  alternative,  correspon¬ 
dence  education  began  to  gradually  incorporate  in  its 
armory  increasingly  more  learner  support  strategies, 
ranging  from  local  study  center  support  to  residential 
study  sessions,  usually  during  the  summer  breaks  (Ber- 
nath  and  Szucs;  2004;  Brindley  et  al.,  2004;  Sewart, 
1993;  Tait  and  Mills,  2003).  It  also  included  the  use 
of  a  growing  range  of  technologies  to  supplement  the 


printed  study  materials  to  support  the  interaction 
between  the  learners  and  the  teachers. 

This  shift  in  correspondence  education  from  an 
exclusive  reliance  on  the  printed  word  and  the  postal 
services  to  the  incorporation  of  a  wider  range  of  tech¬ 
nologies  for  communication  between  learners  and 
teachers  made  the  term  correspondence  education 
increasingly  unsuitable  .  With  the  growing  use  of  non¬ 
print  media  in  this  mode  of  learning  and  teaching, 
much  more  than  simply  correspondence  was  taking 
place  between  the  learners  and  the  teachers  or  the 
teaching  organization. 

This  led  to  a  growing  push  for  the  adoption  of  the 
term  distance  education  for  this  mode  of  learning  and 
teaching.  This  new  term  was  favored  because  it  directed 
attention  away  from  the  mode  of  communication  (i.e., 
print  and  postal  services).  The  concept  of  distance  in 
this  mode  of  education  focused  attention  on  the  nature 
of  the  separation  of  the  learners  from  their  teachers  and 
the  teaching  organization  and  on  the  noncontiguous 
nature  of  the  learning  and  teaching  transaction  (Keegan, 
1990;  Perraton,  1987;  Rumble,  1989).  Use  of  the  term 
distance  education  for  correspondence  education  grew 
due  to  the  proliferation  of  newer  technologies  that  were 
becoming  available  to  support  time,  place,  and  pace 
independence.  Although  this  shift  has  been  a  growing 
trend  in  the  more  developed  and  resource-rich  econo¬ 
mies,  print  and  the  postal  services  continue  to  be  widely 
used  for  distance  education  in  the  less  developed  and 
resource-poor  economies. 


TECHNOLOGIES 

A  meaningful  way  to  cluster  technologies  that  enable 
time,  place,  and  pace  independence  is  presented  in 
Table  22.1  (Naidu,  2006;  Romiszowski,  2004).  The 
approach  in  this  table  helps  to  focus  our  attention  on 
the  key  affordances  of  these  technologies— -that  is,  the 
opportunities  that  these  technologies  offer  for  individ¬ 
uals  as  well  as  groups  of  learners  to  work  in  their  own 
time,  at  their  own  pace,  and  place,  asynchronously  or 
synchronously  (see  also  Chapters  21  and  23  in  this 
volume) . 


TABLE  22.1 

Clusters  of  Technology  Affordances 
for  Learning 

Self-paced  learning  offline  Self-paced  learning  online 
Group-based  learning  Group-based  learning 

asynchronously  synchronously 
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The  concept  of  ajfordance ,  which  was  first  devel¬ 
oped  by  James  Gibson  in  relation  to  his  work  on  per¬ 
ception,  refers  to  what  an  environment  has,  offers,  or 
provides  as  clues  or  stimuli,  either  positive  or  negative, 
for  perception  or  cognition  to  take  place  (Gibson, 
1977, 1979).  In  relation  to  the  use  of  educational  tech¬ 
nologies,  the  term  ajfordance  is  being  commonly  used 
to  refer  to  the  opportunities  that  various  features  or 
attributes  of  technologies  offer  for  various  types  of 
learning  activities  (Bames,  2000;  Gaver,  1991,  1992). 

The  review  of  research  on  the  affordances  of  tech¬ 
nology  for  learning  adopts  a  consistent  format.  It  starts 
off  with  a  definition  and  description  of  the  concept  and 
mode  of  learning.  This  is  followed  by  a  description  of 
attributes  and  affordances  of  the  technologies  that 
enable  particular  modes  of  learning.  A  review  of  key 
research  directions  on  the  mode  of  learning  is  then 
presented.  This  also  includes  key  unanswered  ques¬ 
tions  and  some  directions  for  further  research. 

Self-Paced  Learning  Offline 

The  concept  of  self-paced  learning  implies  freedom 
from  the  constraints  of  time  and  pace.  In  this  mode  of 
learning,  individuals  are  able  to  carry  out  their  learning 
activities  within  a  time  frame  and  at  a  pace  that  suits 
them,  although  some  or  all  of  these  activities  may  have 
to  be  carried  out  at  specific  locations  such  as  a  library 
or  a  laboratory.  Self-paced  learning  suits  learners  for 
different  reasons.  It  is  ideally  suited  for  the  indepen¬ 
dent  learner  who  is  pursing  a  hobby  or  who  is  learning 
something  for  very  personal  reasons  and  not  necessar¬ 
ily  for  a  formal  credit  (Brookfield,  1982).  It  is  also 
suited  to  a  learner  who  is  studying  for  formal  credit 
but  who  might  need  more  or  less  time  as  well  as  a 
different  pace  from  that  of  others. 

A  strong  argument  in  the  educational  literature 
favors  allowing  the  time  and  the  pace  that  a  learner 
needs  to  complete  the  required  learning  activities  or 
to  achieve  his  or  her  full  potential  (Carroll,  1963).  Two 
models  of  learning  that  have  been  developed  around 
the  concepts  of  time  and  pace  flexibility  include  Mas¬ 
tery  Learning  and  the  Personalized  System  of  Instruc¬ 
tion  (Block  and  Anderson,  1975;  Bloom,  1968;  Keller, 
1968).  These  models  of  learning  are  based  on  the 
premise  that  any  learner  is  capable  of  achieving  mas¬ 
tery  if  he  or  she  has  been  allowed  the  time  and  the 
pace  that  he  or  she  needs. 

A  technology  that  affords  the  greatest  amount  of 
flexibility  in  terms  of  the  time,  pace,  and  place  of  study 
is  clearly  the  printed  textbook.  This  can  come  in  the 
form  of  commercially  produced  reference  books,  cus¬ 
tomized  readers,  and  study  guides.  Carefully  designed 
textual  material  makes  judicious  use  of  a  range  of 


design  strategies  to  capture  and  communicate  its  mes¬ 
sage  to  the  readers.  These  strategies  include  anything 
from  introductions,  in-text  questions,  and  summaries 
to  pictures  and  graphic  illustrations.  The  printed  text¬ 
book  is  a  widely  portable  and  relatively  durable  item 
that  can  be  used  by  anyone  who  is  able  to  read.  It  gives 
the  reader  the  flexibility  to  read  it  when  and  where  he 
or  she  needs  or  wishes  to  read  it.  Also,  readers  can 
vary  the  pace  at  which  they  might  read  different  kinds 
of  textual  material;  for  example,  students  might  read 
much  of  the  daily  newspaper  a  lot  more  quickly  than 
they  might  read  a  journal  article  for  their  studies. 

A  wide  range  of  non-print  media  also  affords  a 
considerable  amount  of  flexibility  in  terms  of  time, 
pace,  and  place  of  study.  The  more  conventional  of  the 
non-print  technologies  include  the  audiocassette,  vid¬ 
eocassette,  CD-ROMs,  and  DVDs.  These  technologies 
are  capable  of  capturing  sound,  animation,  and  the 
moving  image  which  is  crucial  for  representing  various 
kinds  of  content.  They  are  very  durable  and  portable, 
and  they  offer  users  a  good  deal  of  flexibility  in  terms 
of  the  time,  pace,  and  place  of  its  use. 

Contemporary  non-print  technologies  include  a 
growing  list  of  portable  devices,  including  mobile 
phones,  a  variety  of  personal  digital  assistants 
(PDAs),  iPODs,  laptop  computers,  and  tablet  PCs. 
Although  the  form  and  function  of  these  portable 
technologies  continue  to  change  incessantly,  their 
unique  attribute  is  their  ability  to  support  time,  place, 
and  in  many  cases  pace  independence.  Mobile 
phones,  for  example,  in  addition  to  serving  as  com¬ 
munication  devices  are  able  to  support  a  range  of 
other  functions  such  as  organizing  and  scheduling. 
Personal  digital  assistants  that  are  now  coming  onto 
the  market  have  similarly  progressed  from  serving  as 
simple  calendars  or  schedulers  to  also  supporting 
communication.  The  laptop  computer  is  now  the  main 
working  machine  for  many,  as  its  power  and  speed 
have  expanded  to  match  those  of  the  desktop  machine. 
The  increasing  power  and  potential  of  these  technol¬ 
ogies  afford  substantially  improved  opportunities  for 
individuals  as  well  as  groups  to  work  and  study  at  a 
time,  pace,  and  place  that  is  convenient  for  them. 

There  has  been  extensive  research  on  the  educa¬ 
tional  uses  of  the  more  conventional  mobile  technol¬ 
ogies,  such  as  printed  study  materials  and  other  non¬ 
print  mobile  technologies  (Bernard  and  Naidu,  1990, 
1992;  Bernard  et  al.,  1991;  Hackbarth,  1996;  Heinich 
et  al.,  1993;  Lockwood,  1998;  Naidu,  1994;  Naidu, 
and  Bernard,  1992;  Rigney,  1978).  Research  on  the 
use  of  the  more  contemporary  mobile  technologies  in 
learning  and  teaching,  such  as  mobile  phones  and  per¬ 
sonal  digital  assistants,  is  only  just  beginning  (Kuku- 
lska-Hulme  and  Traxler,  2005). 
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A  growing  body  of  research  on  the  use  of  hand¬ 
held  devices  such  as  mobile  phones  in  the  classroom 
(Prensky,  2005)  indicates  that  hand-held  devices,  and 
especially  mobile  phones,  are  becoming  increasingly 
more  affordable  in  both  developed  and  developing 
countries.  These  are  powerful  tools  that  offer  voice- 
based  communication,  text  messaging,  graphic  dis¬ 
plays,  and  Web  browsing;  however,  their  ubiquity  has 
yet  to  be  fully  explored  for  learning  and  teaching  pur¬ 
poses  (Prensky,  2005). 

The  PDAs  currently  appearing  on  the  market  are 
able  to  serve  several  useful  educational  functions, 
including  content  delivery,  organization,  communica¬ 
tion,  and  access  to  various  types  of  educational  guid¬ 
ance  and  support  services  (Kukulska-Hulme  and  Trax- 
ler,  2005).  Preliminary  reports  are  beginning  to  emerge 
on  the  use  of  various  mobile  technologies  for  a  variety 
of  educational  purposes.  These  include  the  use  of 
mobile  phones  in  language  learning  (Levy  and 
Kennedy,  2005)  and  the  use  of  PDAs  by  medical  staff 
and  physicians  for  accessing  critical  information  while 
they  are  away  from  their  offices  (Kneebone  and  Bren- 
ton,  2005;  Smordell  and  Gregory,  2005).  There  are  also 
reports  on  the  use  of  handheld  devices  and  wireless 
computers  to  improve  assessment  of  learning  and 
instruction  (Moallem  et  al.,  2005). 

As  the  demand  for  greater  flexibility  in  learning 
and  teaching  and  just-in-time  learning  opportunities 
(i.e.,  learning  at  the  time  of  need)  increases,  it  is  likely 
that  greater  use  will  be  made  of  mobile  technologies. 
No  doubt,  the  mobile  technologies  of  the  future  will 
have  a  lot  more  capacity  than  today’s  personal  com¬ 
puters;  however,  their  attraction  will  lie  in  their  tar¬ 
geted  use  to  support  specific  learning  and  teaching 
activities.  Reports  on  their  use  should  focus  on  the 
following  types  of  questions: 

•  What  are  the  ways  in  which  mobile  technol¬ 
ogies  can  be  used  to  support  learning  in  both 
individualized  and  group-based  educational 
settings?  What  are  the  impacts  of  such  use 
of  mobile  technologies  on  various  aspects  of 
learning? 

•  What  are  the  ways  in  which  mobile  technol¬ 
ogies  can  be  used  to  leverage  various  learn¬ 
ing  and  teaching  activities?  What  are  the 
impacts  of  such  use  of  mobile  technologies 
on  various  aspects  of  learning? 

To  adequately  answer  these  types  of  questions,  great 
care  should  be  taken  to  focus  attention  on  the  foregoing 
affordances  of  these  technologies  and  especially  on  the 
ways  in  which  they  can  support  specific  learning  and 
teaching  functions.  These  studies  will  have  to  use  the 


full  range  of  approaches  on  research  and  evaluation 
and  draw  upon  the  collective  wisdom  on  research  and 
evaluation  of  technology-enhanced  learning  (Abrami 
and  Bernard,  2006;  Taylor,  2003). 

Self-Paced  Learning  Online 

Self-paced  learning  online  is  a  mode  of  learning  in 
which  an  individual  is  able  to  study  at  his  or  her  own 
pace  with  a  range  of  online  technologies.  A  growing 
list  of  technologies  is  becoming  widely  available  and 
also  affordable  to  make  time  and  pace  independence 
online  a  considerably  more  pleasant  experience.  Per¬ 
haps  the  most  prominent  among  the  technologies  sup¬ 
porting  self-paced  learning  online  are  the  Internet  and 
the  World  Wide  Web.  The  Internet  refers  to  the  net¬ 
work  of  computers  that  are  connected  to  one  another, 
thus  enabling  the  sharing  of  data,  information,  com¬ 
munication,  and  other  types  of  subject  matter  among 
its  users  via  file-sharing  protocols.  Another  set  of 
technologies  that  is  able  to  support  self-paced  learn¬ 
ing  is  the  variety  of  computer-mediated  communica¬ 
tions  technologies,  such  as  e-mail,  mailing  lists,  and 
discussion  forums  (Naidu,  1989).  This  suite  of  tech¬ 
nologies  uses  the  electronic  text  to  enable  users  to 
communicate  with,  and  share  information  with  indi¬ 
viduals  and  groups  in  their  own  time  and  at  their  own 
pace,  and  from  a  place  that  is  convenient  to  them 
(Naidu,  2006). 

Also  growing  in  popularity  for  supporting  self- 
paced  learning  online  in  both  campus-based  and  dis¬ 
tance  education  systems  are  online  learning  manage¬ 
ment  systems.  These  are  software  applications  com¬ 
prised  of  a  collection  of  tools  that  can  support  a  variety 
of  learning  and  teaching  activities,  such  as  self-paced 
independent  study  as  well  as  group-based  learning 
activities;  they  provide  the  opportunity  to  access  and 
work  on  a  particular  subject  matter  at  the  learner’s  own 
pace  and  time  and  from  a  place  of  his  or  her  choice. 
They  also  allow  communication  with  peers  and  teach¬ 
ers  at  the  learner’s  own  convenience  (Naidu,  2006). 

A  critical  enabler  of  self-paced  online  learning  is 
access  to  electronic  resources,  which  are  becoming 
increasingly  available  to  users  online.  These  include 
various  types  of  electronic  databases,  learning  object 
repositories  and  archives,  journals,  and  books  (Mc- 
Greal,  2004;  Richards  et  al.,  2004).  They  allow  learn¬ 
ers  to  search  for  and  retrieve  data  and  information  from 
rich  repositories  at  a  time,  pace,  and  from  a  place 
convenient  to  them.  Publishers  and  promoters  of  these 
electronic  resources  are  suggesting  that  these  resources 
are  the  “building  blocks  of  e-learning  ...  [and]  the 
libraries  of  the  e-learning  era”  (Richards  et  al.,  2004, 
pp.  236,  242). 
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Despite  its  obvious  advantages,  self-paced  learn¬ 
ing  online  does  have  some  limitations,  particularly 
with  relation  to  the  loneliness  and  boredom  that  may 
set  in  due  to  the  lack  of  social  presence ,  which  refers 
to  the  degree  to  which  participants  seem  to  be  real  in 
noncontiguous  educational  settings.  Social  learning 
theorists  would  argue  that  learning  could  be  con¬ 
strained  by  the  lack  of  social  presence  (Bandura, 
1977;  Lave,  1991;  Vygotsky,  1978;  Wenger,  1998; 
Wertsch,  1991).  This  lack  of  social  presence  in  self- 
paced  online  learning  has  been  its  major  criticism  and 
the  subject  of  a  great  deal  of  research.  The  focus  of 
that  research  is  on  the  impacts  and  implications  of 
the  lack  of  social  presence  on  learning  (Gunawardena, 
1995;  Gunawardena  and  Zittle,  1997;  Richardson  and 
Swan,  2003;  Swan  and  Shih,  2005)  and  on  strategies 
that  can  be  employed  to  reintegrate  social  presence 
in  learning  online  (Conrad,  2005;  Kreijns  et  al.,  2002, 
2003;  Swan,  2002).  A  key  premise  of  this  line  of 
research  is  that  building  social  cohesiveness  and  com¬ 
munity  is  essential  to  our  learning  and  our  learning 
capability  (Kreijns  et  al.,  2002,  2003). 

Group-Based  Learning  Asynchronously 

Group-based  learning  is  grounded  in  the  principles  of 
cooperative  and  collaborative  learning  and  on  the 
belief  that  the  development  of  knowledge  is  a  social 
process.  It  involves  groups  of  people  engaged  in  the 
negotiation  of  meaning  and  understanding  (Slavin, 
1990,  1994;  Wenger,  1998).  Although  individuals  can 
learn  by  themselves,  proponents  of  group-based  learn¬ 
ing  argue  that  group-based  learning  is  a  more  powerful 
means  for  developing  knowledge  and  understanding 
(Pea,  1993;  Resnick  et  al.,  1991). 

Group-based  learning  asynchronously  involves 
groups  of  learners  working  together  without  the  need 
to  be  studying  in  the  same  place  or  at  the  same  time. 
This  is  becoming  increasingly  possible  with  the  avail¬ 
ability  of  a  range  of  technologies  that  can  support 
asynchronous  communication  between  individuals 
and  groups  who  are  not  in  the  same  location.  Prom¬ 
inent  among  these  is  the  suite  of  computer-mediated 
communications  technologies ,  such  as  online  learning 
management  systems,  mailing  lists.  Weblogs  (blogs), 
wikis,  and  podcasts.  Online  learning  management 
systems  are  software  applications  that  allow  learners 
to  work  together  without  the  need  to  be  at  the  same 
place  or  time.  Students  can  log  onto  these  systems  at 
a  time  and  from  a  place  that  suits  them  to  carry  out 
tasks  that  they  have  been  assigned,  and  they  are  able 
to  see  what  others  have  done.  They  can  continue  to 
work  on  these  tasks  for  as  long  as  they  like  or  need 
to  and  as  often  as  necessary  until  the  group  is  satisfied 


that  the  work  is  done.  In  this  environment,  teachers 
are  able  to  monitor  the  contributions  of  individuals 
as  well  as  the  group. 

Considerable  research  has  been  conducted  on 
asynchronous  group-based  learning  and  the  technolo¬ 
gies  that  support  it  (Beldarrain,  2006;  Mason,  1993; 
Mason  and  Kaye,  1989;  Naidu,  1989;  Rapaport,  1991). 
A  predominant  focus  of  much  of  this  research  is  on 
the  affordances  of  these  technologies  for  collaborative 
learning  (Koschmann,  1996;  McConnell,  2000;  Stahl, 
2002),  and  building  learning  communities  (Bernard 
and  Lundgren-Cayrol,  2001;  Hathorn  and  Ingram, 
2002;  Kanuka  and  Anderson,  1998;  Paulus,  2004, 
2005;  Salmon,  2000;  2003). 

Evidence  and  experience  from  this  body  of 
research  suggest  that,  to  achieve  the  best  outcomes, 
asynchronous  group-based  learning  requires  the  same 
level  of  rigor  as  any  other  mode  of  learning,  including 
paying  attention  to  structuring,  managing,  and  moder¬ 
ating  such  activities.  Although  few  would  argue  the 
benefits  of  asynchronous  group-based  learning,  we 
must  learn  a  great  deal  more  about  how  to  best  assess 
learning  outcomes  within  such  educational  settings. 
The  affordances  of  the  technology  in  this  regard  are 
still  very  primitive  and  underutilized.  Future  research 
on  asynchronous  group-based  learning  must  focus  on 
how  learning  achievement  can  be  reliably  and  validly 
assessed  in  such  educational  settings. 

Group-Based  Learning  Synchronously 

Group-based  learning  synchronously  also  enables 
groups  of  learners  to  work  together;  however,  in  this 
mode  of  learning,  although  the  learners  need  not  be  in 
the  same  place,  they  must  be  present  at  the  same  time 
and  progress  pretty  much  at  the  same  pace  as  the 
group.  Commonly  known  technologies  that  offer  this 
kind  of  flexibility  include  audio  and  video  conferenc¬ 
ing  and  broadcast  radio  and  television.  These  technol¬ 
ogies  give  learners  who  primarily  study  independently 
the  rare  opportunity  to  communicate  and  work  with 
their  peers  for  brief  periods  of  intensive  synchronous 
activities.  These  activities  may  include  guest  lectures, 
tutorials,  and  demonstrations.  The  use  of  these  tech¬ 
nologies  is  very  popular  in  distance  education  settings, 
where  they  serve  a  critical  role  in  complementing  indi¬ 
vidualized  self-paced  study. 

Audio  and  video  conferencing,  as  well  as  radio 
and  television,  are  widely  known  technologies  that 
have  demonstrated  a  great  deal  of  opportunities  for 
group-based  synchronous  learning  (Hutton,  1984; 
Michel,  1987;  Schramm,  1977;  Thomas,  1987;  Zuber- 
Skerritt,  1984).  In  distance  education  settings,  wide¬ 
spread  use  of  radio  and  television  broadcasts  enhances 
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and  supplements  classroom  and  home  schooling  activ¬ 
ities  (Green,  2006;  Potter  and  Naidoo,  2006). 

Newer  technologies  that  offer  opportunities  for 
synchronous  activity  include  voice  over  IP  (VoIP), 
inter-relay  chat  (IRC),  and  various  types  of  games  that 
can  be  played  with  MOOs  and  MUDs  by  multiple 
users  on  the  Internet  (MOO  is  an  acronym  for  MUD, 
object-oriented,  and  MUD  stands  for  multi-user 
domain). 

The  growth  of  online  distance  learning  seems  to 
be  spearheading  a  growing  interest  in  the  newer  col¬ 
laborative  learning  technologies  that  promise  group- 
based  synchronous  learning  opportunities.  Noteworthy 
work  in  this  regard  has  attempted  to  integrate  proven 
learning  strategies  such  as  role  playing  and  problem 
solving  with  technologies  to  promote  the  concept  that 
school  learning  should  be  more  like  life  itself  (Chil¬ 
dress  and  Braswell,  2006;  Doering,  2006;  Naidu  et  al., 
2000).  This  is  encouraging  and  is  the  direction  that 
research  and  development  activity  in  synchronous 
group-based  learning  should  be  taking. 


DISCUSSION 

Frequently  asked  questions  about  technology  and  edu¬ 
cation  include  (Clark  and  Solomon,  1986;  Kozma, 
1991):  How  does  technology  influence  learning?  Is  this 
influence  an  improvement  over  face-to-face  instruc¬ 
tion?  This  line  of  inquiry  is  problematic ,  as  it  suggests 
that  face-to-face  education  is  an  ideal  form  of  learning 
and  teaching  and  that  it  serves  as  a  benchmark  that 
must  be  met  to  establish  success.  Furthermore,  it  fails 
to  define  the  meaning  of  the  term  influence,  nor  does 
it  address  how  such  influence  is  being  ascertained  and 
in  relation  to  what  specific  attributes  of  face-to-face 
education. 

Although  the  search  for  the  influence  of  technol¬ 
ogy  on  learning  is  justified,  much  of  the  research  in 
this  regard  is  misguided  (Clark,  1983,  1994;  Kulik, 
1985).  Too  many  of  these  studies  focus  attention  on 
the  unique  impacts  of  technology,  which  are  almost 
impossible  to  delineate  from  how  the  technology  is 
being  used  and  for  what  purpose.  Researchers  have 
argued  that  with  that  kind  of  focus  these  studies  run 
the  risk  of  reporting  results  that  cannot  be  attributed 
to  the  technology  alone,  as  they  are  very  likely  to  be 
due  to  the  combined  effects  of  both  the  technology  and 
the  teaching  method  (Clark,  1994;  Kozma,  1991). 

There  is  growing  consensus  around  the  view  that 
research  on  the  influence  of  technology  on  education 
should  focus  on  the  affordances  of  these  technologies 
and  not  the  technology  itself  (Clark,  1983,  1994). 
These  affordances  include  the  capabilities  of  tech¬ 


nologies  to  capture  and  represent  different  types  of 
content  and  messages,  activate  learning,  provide 
opportunities  for  socialization,  assess  learning  out¬ 
comes,  and  provide  feedback  and  remediation  to 
learners  (Naidu,  2003).  Although  these  are  common 
educational  activities ,  research  should  focus  on  how 
these  activities  are  enhanced  and  supported  by  vari¬ 
ous  technologies;  for  example,  do  time-  and  place- 
independent  technologies  have  particular  advantages 
for  these  learning  and  teaching  activities?  What  are 
these  advantages?  How  can  these  advantages  be  opti¬ 
mized  with  the  use  of  various  technologies?  Research 
should  focus  on  how  technologies  can  be  used  to 
capture  data  and  other  types  of  information  to  make 
them  accessible  to  users  when  and  where  they  are 
needed.  e-Books,  for  example,  might  be  a  great  idea, 
but  are  they  best  delivered  on  a  PDA  or  an  iPOD? 
What  types  of  information  are  best  delivered  on 
mobile  technologies  and  to  support  what  kind  of 
learning  and  use? 


CONCLUDING  REMARKS 

Technologies  that  enable  time,  pace,  and  place  inde¬ 
pendence  are  becoming  widely  available  in  both  devel¬ 
oped  and  developing  societies.  As  these  technologies 
become  more  accessible  and  affordable,  they  are  likely 
to  permeate  all  aspects  of  our  daily  lives;  therefore,  it 
would  seem  sensible  to  deploy  them  appropriately  in 
learning  and  teaching  rather  than  limit  or  ban  them 
from  these  contexts.  Inefficient  use  of  these  technolo¬ 
gies  will  only  lead  to  blaming  the  technology  for  ensu¬ 
ing  problems,  as  we  have  done  in  the  past.  Technolo¬ 
gies  that  enable  time,  pace,  and  place  independence 
have  particular  advantages  for  various  types  of  learn¬ 
ing  activities  and  different  groups  of  learners.  They 
allow  learners  and  teachers  to  do  things  that  are  not 
possible  within  the  parameters  of  campus-based  edu¬ 
cational  settings.  Many  of  these  technologies  can  put 
a  great  deal  of  resources  within  easy  reach  of  learners 
and  in  so  doing  can  empower  them  in  various  ways 
and  open  up  new  opportunities  for  learning  and  teach¬ 
ing.  These  types  of  learning  opportunities  will  require 
teachers  as  well  as  educational  organizations  to  rethink 
their  learning  and  teaching  processes  and  how  they 
may  have  been  conducting  their  business  in  conven¬ 
tional  campus-based  educational  settings  (Herrington 
et  al.,  2006).  This  kind  of  reorientation  to  learning  and 
teaching  will  have  numerous  implications  for  the 
design  and  development  of  such  learning  environ¬ 
ments,  and  this  is  where  research  and  development 
activities  in  the  field  ought  to  place  much  of  their 
emphasis. 
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ABSTRACT 

In  recent  decades,  rapid  technological  innovation  has 
facilitated  a  convergence  between  traditional  face-to- 
face  and  distributed  (or  technology-mediated)  learning 


environments.  These  blended  learning  environments  try 
to  take  advantage  of  the  strengths  of  both  archetypal 
learning  environments  (Graham,  2006).  The  emergence 
of  blended  learning  is  highlighted  in  higher  education 
and  in  industry  training  literature.  The  Chronicle  of 
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Higher  Education  reports  that  the  President  of  Penn¬ 
sylvania  State  University  regards  the  convergence 
between  online  and  residential  instruction  as  the  “sin¬ 
gle  greatest  unrecognized  trend  in  higher  education 
today”  (Young,  2002,  p.  A33).  Similarly,  the  American 
Society  for  Training  and  Development  identifies 
blended  learning  as  one  of  the  top  ten  emergent  trends 
in  the  knowledge  delivery  industry  (Finn,  2002).  Yet, 
surprisingly,  we  understand  little  about  the  nature  of 
blended  learning  systems.  This  chapter  identifies  core 
issues  and  research  about  blended  learning  using  the 
Sloan  Consortium’s  five  pillars  (learning  effectiveness, 
student  satisfaction,  faculty  satisfaction,  cost  effective¬ 
ness,  and  access)  as  an  organizing  framework  (Lorenzo 
and  Moore,  2002).  The  authors  also  discuss  future 
directions  in  blended  learning  research. 


KEYWORDS 

Blended  learning  environment :  A  learning  environ¬ 
ment  that  combines  face-to-face  instruction  with 
technology-mediated  instruction. 

Hybrid  learning  environment :  Alternative  term  for 
blended  learning  environment. 


INTRODUCTION 

The  term  blended  learning  is  relatively  new  in  higher 
education  and  in  corporate  settings  (the  terms  hybrid 
and  blended  can  be  used  interchangeably) .  An  ongoing 
discussion  has  ensued  on  the  precise  meaning  of  the 
term  (Driscoll,  2002;  Graham  et  al.,  2003;  Jones,  2006; 
Laster,  2004;  Masie,  2006;  Oliver  and  Trigwell,  2005; 
Osguthorpe  and  Graham,  2003),  however,  the  most 
common  position  is  that  blended  learning  environ¬ 
ments  combine  face-to-face  instruction  with  technol¬ 
ogy-mediated  instruction  (Graham,  2006;  Graham  et 
al.,  2003).  Traditional  face-to-face  instruction  involves 
interactions  between  instructors  and  learners  who  are 
in  the  same  place,  whereas  technology-mediated 
instruction  uses  information  and  communication  tech¬ 
nologies  (ICT)  to  mediate  the  learning  experience  and 
interactions  without  requiring  that  learners  and  instruc¬ 
tors  be  located  together. 

Research  suggests  three  primary  reasons  for  adopt¬ 
ing  a  blended  approach  to  instruction:  (1)  improved 
learning  effectiveness,  (2)  increased  access  and  con¬ 
venience,  and  (3)  greater  cost  effectiveness  (Graham, 
2006).  Most  often,  educators  adopt  blended  learning 
approaches  to  explore  gains  and  tradeoffs  in  compar¬ 
ison  with  strictly  traditional  settings  or  entirely  distrib¬ 
uted  environments. 


This  chapter  identifies  core  issues  and  research  in 
the  blended  learning  format  using  the  Sloan  Consor¬ 
tium’s  five  pillars  (learning  effectiveness,  student  sat¬ 
isfaction,  faculty  satisfaction,  cost  effectiveness,  and 
access)  as  an  organizing  framework  (Lorenzo  and 
Moore,  2002).  We  also  provide  some  directions  for 
future  research.  We  have  chosen  to  focus  on  blended 
learning  environments  in  a  higher  education  context, 
while  acknowledging  that  significant  innovations  also 
occur  in  informal,  military,  and  corporate  contexts 
(Collis  et  al.,  2005;  Harris,  2005;  Kirkley  and  Kirkley, 
2005,2006;  Lewis  and  Orton,  2006;  Newton  and  Ellis, 
2005;  Wenger  and  Ferguson,  2006;  Wisher,  2006). 

LEARNING  EFFECTIVENESS 

Key  questions  with  regard  to  the  effectiveness  of 
blended  environments  include: 

•  What  are  the  affordances  of  face-to-face  and 
technology-mediated  contexts  and  how  can 
the  strengths  of  each  be  used  to  improve 
teaching  and  learning? 

•  How  do  short-term  student  learning  out¬ 
comes  interact  with  more  systemic  longer 
term  student  outcomes? 

•  What  are  the  emerging  models  for  assessing 
learning  outcomes  in  the  blended  learning 
environment? 

•  What  is  the  appropriate  role  for  students 
becoming  involved  in  the  assessment  of  their 
own  learning  effectiveness? 

Transformational  Potential 

Researchers  recognize  the  potential  for  transforming 
learning  when  combining  both  face-to-face  and  tech¬ 
nology-mediated  instruction  (Garrison  and  Kanuta, 
2004;  Graham,  2006;  Graham  and  Robison,  2007). 
Many  allude  to  this  potential  when  they  state  that 
blended  learning  capitalizes  on  the  best  of  both  worlds. 
The  simple  elegance  of  the  blended  learning  concept 
can  also  be  a  weakness,  however,  if  the  focus  is  entirely 
on  the  mode  of  instruction  rather  than  the  holistic 
nature  of  the  learning  experience.  For  example, 
instructors  commonly  state  that  their  course  is  a  blend 
that  consists  of  x%  online  and  y%  face-to-face,  which 
is  not  informative  without  knowing  the  nature  of  the 
activities  occurring  in  the  distinct  learning  environ¬ 
ments  and  how  the  course  effectively  uses  the  affor¬ 
dances  of  the  two  environments  (Cross,  2006).  For 
blended  learning  to  reach  its  full  transformational 
potential,  the  primary  goal  should  be  rethinking  and 
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redesigning  the  teaching  and  learning  relationship  (i.e., 
improved  pedagogy)  with  efficiency  and  convenience 
as  possible  secondary  benefits  (Garrison  and  Kanuta, 
2004,  p.  99).  Blended  learning  must  capitalize  on  the 
strengths  of  both  online  and  face-to-face  modalities  to 
create  a  more  active  learning  environment  (Graham 
and  Robison,  2007). 

Completion  Rates  and  Academic  Performance 

Current  studies  on  learning  effectiveness  in  blended 
courses  and  programs  concentrate  on  measures  such 
as  grades  and  withdrawal  rates  that  are  highly  sensitive 
to  factors  such  as  course  level,  college,  and  depart¬ 
ment.  This  instability  suggests  that  using  course  mode 
as  an  effect  in  comparison  studies  is  not  a  particularly 
viable  line  of  inquiry;  however,  some  studies  have  been 
directed  toward  learning  effectiveness  in  blended  envi¬ 
ronments.  Rochester  Institute  of  Technology’s  pilot 
program  reported  that  completion  rates  were  approxi¬ 
mately  95%  (Humbert  and  Vignare,  2004;  Starenko  et 
al.,  2007),  and  Reasons  and  colleagues  (Reasons, 
2004;  Reasons  et  al.,  2005)  found  that  fully  online 
students  succeed  at  rates  higher  than  those  in  face-to- 
face  or  blended  courses.  Additional  studies  have  shown 
comparable  success  in  blended  courses,  yet  others 
report  them  as  superior  regarding  learning  effective¬ 
ness  (Boyle  et  al.,  2003;  Cottrell  and  Robinson,  2003; 
Dowling  et  al.,  2003;  O’Toole  and  Absalom,  2003; 
Riff  ell  and  Sibley,  2004).  In  larger  data- mining  studies 
of  several  thousand  student  registrations,  researchers 
at  the  University  of  Central  Florida  found  that  blending 
learning  courses  produced  comparable  or  superior  suc¬ 
cess  rates  compared  to  face-to-face  or  fully  online 
modes  when  college  and  gender  contributions  are 
removed  (Dziuban  et  al.,  2006). 

Assessment 

At  present,  most  assessment  mechanisms  in  blended 
learning  remain  traditional,  as  they  are  objective,  non- 
contextual,  and  inauthentic;  however,  as  the  initiative 
matures,  student  assessment  will,  by  necessity,  become 
interpretive,  contextual,  and  authentic.  Brown  et  al. 
(2007)  identified  several  important  student  perceptions 
about  the  efficacy  of  assessment  techniques  in  the 
blended  environment.  They  found  that  novice  learners 
believe  that  traditional  measures  such  as  multiple-choice 
tests  better  reflect  their  learning  status  than  more  inter¬ 
pretive  measures.  More  experienced  learners,  however, 
report  confidence  in  assessment  activities  that  involve 
collaborative  work  and  interactive  feedback.  These  find¬ 
ings  give  credence  to  developing  theories  that  incorpo¬ 
rate  the  nexus  of  information  literacy,  technology  liter¬ 


acy,  and  critical  thinking  into  a  broader  concept  of 
information  fluency— a  foundation  for  assessment  in 
blended  learning  (University  of  Central  Florida,  2005). 

ACCESS 

Key  questions  with  regard  to  access  include: 

•  How  does  the  enhanced  accessibility 
afforded  by  blended  learning  impact  com¬ 
pletion  rates  in  higher  education? 

•  How  does  blended  learning  impact  the  edu¬ 
cational  opportunities  for  under-represented 
populations? 

•  How  does  accessibility  interact  with  quality 
of  learning? 

The  issue  of  learner  access  is  fundamental  to  blended 
learning  and  includes  access  to  institutions,  access  to 
programs,  and  access  to  courses  (Mayadas,  2001).  The 
issue  of  quality  is  critical  when  considering  the  goal 
of  access.  Shea  (2007,  pp.  19-20)  asserts:  “If  quality 
suffers,  increased  access  is  of  no  benefit.  Students 
don’t  want  access  to  low  quality  programs,  faculty  do 
not  wish  to  teach  in  such  programs,  and  alumni  do  not 
wish  to  support  such  programs.” 

Three  student  populations  appear  to  have  particular 
needs  with  regard  to  access:  those  far  from  campus,  those 
near  campus,  and  those  on  campus.  The  issue  for  these 
populations  hinges  on  determining  the  degree  to  which 
faculty  and  students  need  alternative  instructional  modal¬ 
ities  (Otte,  2005).  In  addition,  instructional  modalities 
such  as  blending  might  facilitate  access  to  educational 
opportunities  for  students  with  disabilities.  Rochester 
Institute  of  Technology  provides  an  excellent  example 
of  how  blended  courses  can  increase  educational 
opportunities  and  learning  effectiveness  for  hearing- 
impaired  students  (Starenko  et  al.,  2007).  Fundamen¬ 
tally,  economic  principles  (e.g.,  reduced  opportunity 
costs  and  comparative  advantage)  yield  advantages  for 
students  who  are  either  on  or  near  campus . 

COST  EFFECTIVENESS 

Key  questions  regarding  cost  effectiveness  include: 

•  What  are  some  effective  models  for  assess¬ 
ing  cost  effectiveness  in  blended  learning? 

•  For  which  contexts  (high  enrollment  courses , 
specific  disciplines)  is  cost  effectiveness 
accepted  and  valued  by  stakeholders? 

•  What  relationships  exist  between  cost  effec¬ 
tiveness  and  student  learning  outcomes? 
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The  Center  for  Academic  Transformation  (Twigg,  2003) 
developed  the  primary  models  for  cost  effectiveness  in 
blended  learning.  Using  these  models,  several  institu¬ 
tions  have  demonstrated  that  it  is  possible  to  improve 
quality  and  reduce  costs,  typically  through  reduced 
dependence  on  human  resources.  Robinson  (2005)  dem¬ 
onstrated  that,  by  adding  technology  to  the  instructional 
design,  quality  increases,  and  in  most  cases  costs  are 
comparable  or  somewhat  lower  than  face-to-face  offer¬ 
ings.  The  University  of  Central  Florida  has  reported  cost 
savings,  improved  facilities  utilization,  improved  learn¬ 
ing,  and  continued  program  growth  (Dziuban  et  al., 
2006);  however,  return  on  investment  cannot  be  deter¬ 
mined  via  a  simple  spreadsheet  calculation.  Many  less 
tangible  factors  contribute,  including  increased  success 
rates,  reduced  number  of  drop  outs,  and  improved  fac¬ 
ulty  and  student  skills.  Depending  on  the  institutional 
context  and  the  model  chosen,  cost  savings  may  or  may 
not  be  realized.  Blended  learning,  however,  offers  real 
potential  for  a  positive  return  on  investment. 

STUDENT  SATISFACTION 

Key  research  questions  pertaining  to  student  satisfac¬ 
tion  include: 

•  What  components  in  a  blended  learning 
environment  contribute  most  to  student  sat¬ 
isfaction? 

•  In  what  contexts  is  student  satisfaction  a  via¬ 
ble  outcome  measure  for  learning  quality? 

•  How  is  student  satisfaction  impacted  when 
students  are  given  a  range  of  options  regard¬ 
ing  the  nature  of  the  blend  in  a  course? 

Student  issues  in  blended  learning  emerge  from  the 
traditional  academy  and  from  the  burgeoning  online 
environments  in  higher  education.  Prensky  (200 1  a, b) 
suggested  that  digital  natives  (the  millennial  genera¬ 
tion),  who  expect  the  immediacy  of  technology,  col¬ 
laborative  learning  opportunities,  and  active  learning 
environments,  force  faculty  and  administrators  to 
adopt  more  effective  pedagogies.  Oblinger  and 
Oblinger  (2005)  claim  that  for  these  students  comput¬ 
ers  and  personal  technologies  are  a  way  of  life.  The 
Internet  is  more  important  to  them  than  television,  and 
they  learn  primarily  through  the  processes  of  trial  and 
error  (Bisoux,  2002;  Oblinger  and  Oblinger,  2005).  We 
should  not  be  surprised,  therefore,  that  some  tension 
exists  between  the  millennial  generation’s  preferred 
learning  styles  and  what  higher  education  currently 
offers— even  in  blended  courses  (Aviles  et  al.,  2005). 


Several  issues  mediate  student  satisfaction  with 
blended  learning.  Some  studies  report  consistently 
high  satisfaction  levels  for  blended  courses  (Dziuban 
et  al.,  2004),  while  others  indicate  somewhat  less  pos¬ 
itive  attitudes  (Utts  et  al.,  2003).  Some  studies  indicate 
that  students  with  an  intuitive  cognitive  style  experi¬ 
ence  a  lower  sense  of  community  in  their  blended 
courses  than  students  with  analytic  approaches  to 
learning  (Graff,  2003).  Conversely,  studies  such  as 
those  conducted  by  Rovai  and  Jordan  (2004)  have 
revealed  a  greater  sense  of  community  in  blended 
courses  when  compared  with  face-to-face  and  fully 
online  courses.  Even  though  investigators  report  con¬ 
flicting  results  about  student  satisfaction,  most  studies 
with  substantial  and  stable  samples  have  found  pre¬ 
dominately  positive  reactions;  the  majority  indicate 
that  convenience,  flexibility,  and  the  reduced  opportu¬ 
nity  costs  involved  in  the  learning  process  are  the  pri¬ 
mary  factors  (Vignare,  2002).  These  elements  tend  to 
be  independent  of  several  potentially  biasing  factors, 
such  as  class  size  and  discipline. 

FACULTY  SATISFACTION 

Key  research  questions  with  regard  to  faculty  satisfac¬ 
tion  with  blended  learning  include: 

•  What  factors  lead  to  faculty  satisfaction  in 
blended  learning? 

•  What  models  do  we  have  for  supporting  fac¬ 
ulty  adoption  of  blended  learning? 

•  How  do  faculty  workloads  relate  to  satisfac¬ 
tion  with  blended  learning  in  the  context  of 
tenure  and  promotion? 

Faculty  satisfaction  is  an  important  element  that  sup¬ 
ports  or  detracts  from  the  adoption  of  blended  learning. 
The  University  of  Central  Florida  reported  that  88% 
of  the  faculty  who  taught  blended  courses  were  satis¬ 
fied  with  the  course  and  would  teach  in  a  blended 
format  again  in  the  future,  but  only  41%  of  the  faculty 
in  the  Rochester  Institute  of  Technology  Blended 
Fearning  Pilot  Project  expressed  a  similar  interest 
(Dziuban  et  al.,  2004;  RIT  Online  Fearning  Depart¬ 
ment,  2005).  Many  documented  factors  influence  fac¬ 
ulty  satisfaction  (Hartman  et  al.,  2000).  Three  major 
elements  are  (1)  impact  on  learning,  (2)  impact  on 
workload,  and  (3)  recognition  that  faculty  efforts  are 
valued.  Many  faculty  members  adopt  blended  learning 
because  they  believe  it  will  improve  learning  effective¬ 
ness;  some  also  believe  that  it  will  add  convenience 
and  improve  their  efficiency. 
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Increasingly,  faculty  are  acquiring  new  technolog¬ 
ical  skills  and  assuming  new  role  expectations  associ¬ 
ated  with  those  skills  (Dziuban  et  al.,  2006;  Kaleta  et 
al.,  2007).  In  blended  learning,  the  faculty  must  master 
the  skills  of  both  the  face-to-face  instructor  and  the 
online  facilitator;  thus,  most  research  reports  that 
implementing  blended  learning  requires  additional  fac¬ 
ulty  time  and  effort  (Kaleta  et  al.,  2007;  Lee  and  Im, 
2006;  Lefoe  and  Hedberg,  2006).  The  faculty  time 
investment  can  be  reduced  through  properly  designed 
professional  development  and  instructional  support  ser¬ 
vices  such  as  training  opportunities  and  performance 
support  systems.  Many  faculty  members  incur  the  addi¬ 
tional  workload  costs  because  they  see  the  benefits  for 
student  learning  (Starenko  et  al.,  2007)  or  because  they 
view  the  extra  workload  as  a  cost  that  will  diminish  as 
they  become  more  comfortable  with  the  new  techno¬ 
logical  tools.  Although  research  cites  faculty  recogni¬ 
tion  and  compensation  as  key  elements  in  successful 
blended  programs,  only  one  fifth  of  higher  education 
institutions  report  providing  formal  recognition  and 
rewards  for  technology  integration  (Green,  2004). 


ORGANIZATIONAL  CONSIDERATIONS 

Key  research  questions  with  regard  to  organizational 
considerations  and  their  impact  on  blended  learning 
include: 

•  What  organizational  components  should  be 
in  place  for  blended  learning  to  become  a 
systemic  initiative? 

•  How  will  blended  learning  manifest  itself  in 
different  organizational  contexts  such  as 
community  colleges,  metropolitan  research 
universities,  and  liberal  arts  colleges? 

•  What  impact  is  blended  learning  having  on 
the  traditional  academy? 

Institutional  support  mechanisms  play  a  vital  role  in 
the  success  of  blended  courses  and  programs.  In  con¬ 
sidering  an  optimal  institutional  climate,  Hartman 
(2005)  suggested  that  the  organizational  foundation 
should  be  built  on  theories  of  practice  where  the  aca¬ 
demic  units  are  able  find  a  common  ground  for  instruc¬ 
tional  development.  He  specified  that  the  elements  of 
those  theories  should  address  instructional  models, 
faculty  development  issues,  course  development  struc¬ 
tures,  and  effective  assessment  designs,  both  institu¬ 
tional  and  at  the  course  level.  Further,  an  effective 
institutional  model  should  undergo  continuous  refine¬ 
ment  and  development  while  the  institution  develops 


increased  organizational  capacity.  Hartman  argued  that 
an  effective  institutional  approach  demands  up-front 
executive  buy-in  and  early  infusion  into  the  colleges 
and  departments.  Further  prerequisites  involve  faculty  - 
and  student-centered  approaches  that  make  blended 
learning  something  the  institution  is  rather  than  some¬ 
thing  it  does  (Hartman,  2005). 


RESEARCH  DIRECTIONS 

We  now  focus  on  two  important  areas  for  future 
research:  institutional  research  and  learning  effective¬ 
ness  research. 

Institutional  Research 

Faculty  Adoption 

Ultimately,  the  success  or  failure  of  blended  approaches 
hinges  on  widespread  faculty  adoption  of  effective  prac¬ 
tices.  We  know  relatively  little  about  why  faculty  adopt 
and  implement  a  specific  blended  instructional  model 
and  how  they  are  making  the  instructional  choices 
involved  in  course  redesign  (Kaleta  et  al.,  2007).  Some 
course  redesign  efforts  are  systemic  in  nature,  such  as 
the  30  projects  supported  by  the  Center  for  Academic 
Transformation  (Twigg,  2003).  The  majority  of 
change,  however,  is  occurring  through  a  process  that 
Collis  and  van  der  Wende  (2002)  refer  to  as  stretching 
the  mold.  In  a  campus-wide  survey  of  faculty,  Graham 
and  Robison  (2007)  found  that  over  one  third  of  the 
faculty  reported  having  taught  a  blended  course,  but 
many  of  the  blends  only  made  small  enhancements  to 
practice  and  did  not  change  teaching  and  learning  in 
significant  or  transformative  ways.  We  need  to  learn 
how  to  avoid  some  of  the  traps  historically  associated 
with  the  adoption  of  technology-rich  solutions,  such 
as  embracing  rigid  practices,  status  quo  adherence,  or 
a  tendency  for  educational  systems  to  preserve  them¬ 
selves  by  domesticating  new  technologies  to  support 
old  practices  (Beckwith,  1988;  Salomon,  2002). 

Models  for  Support  and  Training 

Research  supports  the  supposition  that  institutional 
support  is  necessary  for  mainstream  faculty  and  stu¬ 
dents  to  adopt  blended  learning.  Issues  that  require 
investigation  include:  (1)  how  to  minimize  increased 
demands  on  faculty  and  student  time;  (2)  how  to  pro¬ 
vide  instructors  and  learners  with  the  necessary  skills 
to  succeed,  particularly  in  the  technology-mediated 
environment;  and  (3)  how  to  change  the  organizational 
culture  into  one  willing  to  accept  innovations  such  as 


273 


Charles  R.  Graham  and  Charles  Dziuban 


blended  learning.  Other  research  might  investigate 
models  for  faculty  support  and  training  in  blended 
environments  and  what  aspects  of  the  models  are  trans¬ 
ferable  to  other  contexts. 

Learning  Effectiveness  Research 

Conceptual  Frameworks  and  Models 

Design  problems,  such  as  creating  a  blended  environ¬ 
ment,  are  highly  context  dependent,  with  an  almost 
infinite  number  of  possible  solutions.  Researchers 
should  better  articulate  conceptual  frameworks  that 
will  serve  blended  contexts  (Shea,  2007).  The  commu¬ 
nity  of  inquiry  model  is  one  possible  framework  (Gar¬ 
rison  et  al.,  2000;  Garrison  and  Vaughan,  2007),  but  it 
would  be  more  helpful  to  have  a  range  of  rich  blended 
learning  models  so  designers  could  design  tradeoffs  in 
their  own  specific  contexts. 

Role  of  Live  Interaction  vs. 

Computer-Mediated  Communication 

Under  what  conditions  is  human  interaction  important 
to  the  learning  outcomes  and  learner  satisfaction  with 
the  experience?  When  and  why  should  we  be  consid¬ 
ering  human  interaction  such  as  collaboration  and 
learning  communities  (Alavi  and  Dufner,  2005)?  How 
does  live  interaction  vs.  low-fidelity  asynchronous 
interaction  affect  the  learning  experience?  These  ques¬ 
tions  are  just  a  few  that  are  directly  relevant  to  deciding 
when  to  have  face-to-face  or  technology-mediated 
interactions  between  participants  or  with  a  nonhuman 
instructional  system.  Some  evidence  indicates  that 
learners  in  blended  environments  place  greater  value 
or  emphasis  on  the  face-to-face  components,  while 
other  findings  suggest  that  the  face-to-face  elements 
are  unnecessary  (Graham,  2006).  Research  related  to 
better  understanding  the  nature  of  human  interaction 
in  blended  learning  environments  is  a  promising  direc¬ 
tion  of  inquiry  (Shea,  2007). 

Role  of  Learner  Choice  and  Self -Regulation 

How  are  learners  making  choices  about  the  kinds  of 
blends  in  which  they  are  participating?  Are  choices 
being  made  primarily  on  the  basis  of  convenience  and 
flexibility?  How  much  information  and  guidance  are 
being  provided  to  learners  to  help  them  make  decisions 
about  how  different  blends  will  affect  their  learning 
experience?  Online  components  are  perceived  as 
requiring  a  greater  amount  of  discipline  for  learners  to 
succeed  (Allen  and  Seaman,  2005),  so  how  can  blended 
environments  be  designed  to  support  increasing  learner 
maturity  and  capabilities  for  self-regulation? 


CONCLUSIONS 

Learning  environments  have  affordances  that  facilitate 
or  constrain  different  types  of  interactions  and  activities. 
Although  much  can  be  learned  and  synthesized  from 
research  in  both  distance  and  traditional  learning  envi¬ 
ronments,  blended  environments  provide  a  paradigm 
that  is  different  than  just  a  linear  combination  of  the  two; 
for  example,  reactions  to  the  use  of  computer- mediated 
discussions  can  be  quite  different  in  a  course  that  also 
meets  face-to-face  vs.  a  completely  online  course  (An 
and  Frick,  2006;  Schweizer  et  al.,  2003;  Yanes,  2004). 
In  a  completely  online  course,  the  computer- mediated 
discussion  may  be  valued  as  the  only  means  of  human 
interaction,  while  in  the  blended  context  learners  might 
perceive  it  as  a  low-fidelity,  time-consuming  channel  for 
communication.  Faculty  and  learners  will  take  advan¬ 
tage  of  the  opportunities  in  their  learning  environments 
based  on  their  expectations,  goals,  and  understanding  of 
the  learning  possibilities  within  the  environment.  Cur¬ 
rently,  only  a  small  (but  growing)  body  of  research  is 
specifically  related  to  blended  environments.  We  need 
more  research  on  the  design  of  blended  environments 
and  how  instructors  and  learners  engage  in  the  act  of 
teaching  and  learning  in  these  environments. 
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ABSTRACT 

This  chapter  describes  research  and  development 
efforts  related  to  adaptive  technologies,  which  can  be 
combined  with  other  technologies  and  processes  to 
form  an  adaptive  system.  The  goal  of  an  adaptive  sys¬ 
tem,  in  the  context  of  this  chapter,  is  to  create  an 
instructionally  sound  and  flexible  environment  that 
supports  learning  for  students  with  a  range  of  abilities, 
disabilities,  interests,  backgrounds,  and  other  charac¬ 
teristics.  After  defining  key  terms  and  establishing  a 
rationale  for  adaptation,  we  present  a  general  frame¬ 
work  to  organize  adaptive  technologies.  We  then 
describe  experts’  thoughts  on  what  to  adapt  and  how 
to  adapt.  We  conclude  with  a  summary  of  key  chal¬ 
lenges  and  potential  futures  of  adaptive  technologies. 

KEYWORDS 

Adaptivity:  The  capability  exhibited  by  an  organic  or 
an  artificial  organism  to  alter  its  behavior  according 
to  the  environment.  In  the  context  of  an  instruc¬ 
tional  system,  this  capability  allows  the  system  to 
alter  its  behavior  according  to  learner  needs  and 
other  characteristics.  This  is  typically  represented 
within  a  learner  model. 


Hard  technologies:  These  represent  devices  may  be 
used  in  adaptive  systems  to  capture  learner 
information  (e.g.,  eye-tracking  devices)  or  to 
present  content  to  a  learner  (e.g.,  tactile  tablet). 
These  devices  can  be  used  to  detect  and  classify 
learners’  performance  data  or  affective  states 
such  as  confusion,  frustration,  excitement,  dis¬ 
appointment,  boredom,  confidence,  content¬ 
ment,  and  so  on. 

Learner  model:  A  representation  of  the  learner  that 
is  maintained  by  an  adaptive  system.  Learner 
models  can  be  used  to  provide  personalized 
instruction  to  a  particular  individual,  and  may 
include  cognitive  and  noncognitive  aspects  of 
the  learner.  Learner  models  have  been  used  in 
many  areas,  such  as  adaptive  educational  and 
training  systems,  help  systems,  recommender 
systems,  and  others. 

Soft  technologies:  These  are  usually  algorithms, 
programs,  or  even  environments  that  broaden  the 
types  of  interaction  between  students  and  com¬ 
puters;  for  example,  an  adaptive  algorithm  may 
be  employed  in  a  program  that  selects  an  assess¬ 
ment  task  that  provides  the  most  information 
about  a  particular  learner  at  a  particular  point  in 
time. 
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INTRODUCTION 

Air-conditioning  systems  monitor  and  adjust  room 
temperature;  cruise-control  systems  monitor  and 
adjust  vehicle  speed.  Similarly,  adaptive  educational 
systems  monitor  important  learner  characteristics  and 
make  appropriate  adjustments  to  the  instructional 
milieu  to  support  and  enhance  learning.  In  this  chapter, 
we  describe  research  and  development  related  to  adap¬ 
tive  technologies,  which  can  be  combined  with  other 
technologies  and  processes  to  form  an  adaptive  system. 

The  goal  of  an  adaptive  system,  in  the  context  of 
this  chapter,  is  to  create  an  instructionally  sound  and 
flexible  environment  that  supports  learning  for  stu¬ 
dents  with  a  range  of  abilities,  disabilities,  interests, 
backgrounds,  and  other  characteristics.  The  challenge 
of  accomplishing  this  goal  depends  largely  on  accu¬ 
rately  identifying  characteristics  of  a  particular  learner 
or  group  of  learners— such  as  type  and  level  of  knowl¬ 
edge,  skills,  personality  traits,  affective  states— and 
then  determining  how  to  leverage  the  information  to 
improve  student  learning  (Conati,  2002;  Park  and  Lee, 
2003;  Shute  et  al„  2000;  Snow,  1989,  1994). 

After  defining  key  terms  and  establishing  a  ratio¬ 
nale  for  adaptation,  we  present  a  general  framework 
to  organize  adaptive  technologies.  We  then  describe 
experts’  thoughts  on:  (1)  the  variables  to  be  taken  into 
account  when  implementing  an  adaptive  system  (i.e., 
what  to  adapt),  and  (2)  the  best  technologies  and  meth¬ 
ods  to  accomplish  adaptive  goals  (i.e.,  how  to  adapt). 
We  conclude  with  a  summary  of  key  challenges  and 
future  applications  of  adaptive  tools  and  technologies. 
Challenges  include:  (1)  obtaining  useful  and  accurate 
learner  information  on  which  to  base  adaptive  deci¬ 
sions,  (2)  maximizing  benefits  to  the  learner  while 
minimizing  costs  associated  with  adaptive  technolo¬ 
gies,  (3)  addressing  issues  of  learner  control  and  pri¬ 
vacy,  and  (4)  figuring  out  the  bandwidth  problem, 
which  has  to  do  with  the  amount  of  relevant  learner 
data  that  can  be  acquired  at  any  time. 

Definitions 

Before  we  begin  our  discussion  on  adaptive  technolo¬ 
gies  that  support  learners  in  educational  settings,  we  will 
briefly  define  relevant  terms.  Most  generally,  to  adapt 
means  making  an  adjustment  from  one  situation  or  con¬ 
dition  to  another  (e.g.,  software  programs  and  persons 
are  capable  of  adaptation).  Technology  refers  to  the 
application  of  science  (methods  or  materials,  electronic 
or  digital  products  or  systems)  to  achieve  a  particular 
objective,  such  as  the  enhancement  of  learning.  A  system 
in  this  context  refers  to  a  network  of  related  computer 
software,  hardware,  and  data  transmission  devices. 


An  adaptive  system  adjusts  itself  to  suit  particular 
learner  characteristics  and  needs  of  the  learner.  Adap¬ 
tive  technologies  help  achieve  this  goal  and  are  typi¬ 
cally  controlled  by  the  computational  devices,  adapting 
content  for  different  learners’  needs  and  sometimes 
preferences.  Information  is  usually  maintained  within 
a  learner  model ,  which  is  a  representation  of  the  learner 
managed  by  an  adaptive  system.  Learner  models  pro¬ 
vide  the  basis  for  deciding  how  to  provide  personalized 
content  to  a  particular  individual  and  may  include  cog¬ 
nitive  as  well  as  noncognitive  information.  Learner 
models  have  been  used  in  many  areas,  such  as  adaptive 
educational  and  training  systems  (e.g.,  intelligent  tutor¬ 
ing  systems),  help  systems,  and  recommender  systems. 

Adaptive  systems  may  consist  of  hard  or  soft  tech¬ 
nologies  (e.g.,  devices  vs.  algorithms).  Hard  technolo¬ 
gies  are  devices  used  in  adaptive  systems  to  capture 
learner  information  (e.g.,  eye-tracking  devices),  thus 
they  can  be  used  to  detect  and  classify  learners’  per¬ 
formance  data  or  affective  states  such  as  confusion, 
frustration,  excitement,  and  boredom.  Hard  technolo¬ 
gies  can  also  be  used  to  present  content  in  various 
formats  (e.g.,  tactile  tablet  to  accommodate  visual  dis¬ 
abilities).  Soft  technologies  represent  algorithms,  pro¬ 
grams,  or  environments  that  broaden  the  types  of  inter¬ 
action  between  students  and  computers;  for  example, 
an  adaptive  algorithm  may  be  employed  in  a  program 
that  selects  an  assessment  task  or  learning  object  most 
appropriate  for  a  learner  at  a  particular  point  in  time. 

The  effectiveness  of  adaptive  technologies  hinge 
on  accurate  and  informative  student  or  learner  models. 
For  the  remainder  of  this  chapter,  we  use  the  terms 
student  model  and  learner  model  interchangeably,  and 
abbreviate  them  as  either  SM  or  LM.  Because  this 
chapter  focuses  on  the  educational  functions  of  adap¬ 
tive  systems,  we  limit  our  modeling  discussion  to  the 
context  of  students  or  learners ,  rather  than  more 
broadly  defined  users. 

Rationale  for  Adapting  Content 

The  attractiveness  of  adaptive  technologies  derives 
from  the  wide  range  of  capabilities  that  these  technol¬ 
ogies  afford.  As  discussed,  one  capability  involves  the 
real-time  delivery  of  assessments  and  instructional 
content  that  adapt  to  learners’  needs  and  preferences. 
Other  technology  interventions  include  simulations  of 
dynamic  events,  extra  practice  opportunities  on  emer¬ 
gent  skills,  and  alternative  multimedia  options,  partic¬ 
ularly  those  that  allow  greater  access  to  individuals 
with  disabilities.  We  now  provide  evidence  that  sup¬ 
ports  the  importance  of  adapting  content  to  students 
to  improve  learning.  These  arguments  concern  individ¬ 
ual  and  group  differences  among  students. 
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Differences  in  Incoming 
Knowledge,  Skills,  and  Abilities 

The  first  reason  for  adapting  content  to  the  learner  has 
to  do  with  general  individual  differences  in  relation  to 
incoming  knowledge  and  skills  among  students.  These 
differences  are  real,  often  large,  and  powerful;  how¬ 
ever,  our  educational  system’s  traditional  approach  to 
teaching  is  not  working  well  in  relation  to  the  diverse 
population  of  students  in  U.S.  schools  today  (Shute, 
2006).  Many  have  argued  that  incoming  knowledge  is 
the  single  most  important  determinant  of  subsequent 
learning  (Alexander  and  Judy,  1988;  Glaser,  1984; 
Tobias,  1994).  Thus,  it  makes  sense  to  assess  students’ 
incoming  knowledge  and  skills  to  provide  a  sound 
starting  point  for  teaching.  A  second  reason  to  adapt 
content  to  learners  has  to  do  with  differences  among 
learners  in  terms  of  relevant  abilities  and  disabilities. 
The  latter  addresses  issues  of  equity  and  accessibility. 
To  illustrate,  a  student  with  visual  disabilities  will  have 
great  difficulty  acquiring  visually  presented  material, 
regardless  of  prior  knowledge  and  skill  in  the  subject 
area.  Student  abilities  and  disabilities  can  usually  be 
readily  identified  and  content  adapted  to  accommodate 
the  disability  or  leverage  an  ability  to  support  learning 
(Shute  et  al.,  2005). 

Differences  in  Demographic 
and  Sociocultural  Variables 

Another  reason  to  adapt  content  to  learners  relates  to 
demographic  and  sociocultural  differences  among  stu¬ 
dents,  which  can  affect  learning  outcomes  and  ulti¬ 
mately  achievement  (Conchas,  2006;  Desimone,  1999; 
Fan  and  Chen,  2001).  Adaptive  technologies  can  help 
reduce  some  major  gaps  that  persist  in  the  United 
States  (e.g.,  differential  access  to  information  and  other 
resources);  for  example,  some  researchers  (see,  for 
example.  Snow  and  Biancarosa,  2003)  have  argued 
that  the  achievement  gap  in  the  United  States  is  largely 
due  to  differential  language  proficiencies.  In  response 
to  this  need,  adaptive  technologies  that  support  English 
language  learners  are  being  developed  (Yang  et  al., 
2006). 

Differences  in  Affective  Variables 

In  addition  to  cognitive,  physical,  and  sociocultural 
differences,  students  differ  in  relation  to  affective 
states— many  of  which  influence  learning— such  as 
frustration,  boredom,  motivation,  and  confidence  (Con- 
ati,  2002;  Craig  et  al.,  2004;  Ekman,  2003;  Kapoor  and 
Picard,  2002;  Litman  and  Forbes-Riley,  2004;  Picard, 
1997;  Qu  et  al.,  2005).  Various  noninvasive  measures 


infer  learners’  states  and  alter  the  instructional  envi¬ 
ronment  to  suit  different  needs;  for  example,  sensory 
input  systems  detect,  classify,  and  analyze  learners’ 
facial  expressions  (Yeasin  and  Bullot,  2005),  eye 
movements  (Conati  et  al.,  2005),  head  position  (Seo  et 
al.,  2004),  body  posture  and  position  (Chu  and  Cohen, 
2005),  gestures  (Kettebekov  et  al.,  2003),  and  speech 
(Potamianos  et  al.,  2005).  Bayesian  networks  and  other 
statistical  classifier  systems  can  render  inferences 
about  states  from  a  variety  of  inputs  (e.g.,  excessive 
fidgeting  implying  inattention). 

In  summary,  there  are  a  number  of  compelling 
reasons  to  adapt  content  to  learners.  We  now  provide 
context  and  coherence  for  adaptive  technologies  by 
way  of  a  general  four-process  model.  This  model  has 
been  extended  from  (1)  a  simpler  two-process  model 
that  lies  at  the  heart  of  adaptive  technology  (diagnosis 
and  prescription),  and  (2)  a  process  model  to  support 
assessment  (Mislevy  et  al.,  2003). 

Four-Process  Adaptive  Cycle 

The  success  of  any  adaptive  technology  to  promote 
learning  requires  accurate  diagnosis  of  learner  charac¬ 
teristics  (e.g.,  knowledge,  skill,  motivation,  persis¬ 
tence).  The  collection  of  learner  information  can  then 
be  used  as  the  basis  for  the  prescription  of  optimal 
content,  such  as  hints,  explanations,  hypertext  links, 
practice  problems,  encouragement,  metacognitive  sup¬ 
port,  and  so  forth.  Our  framework  involves  a  four- 
process  cycle  connecting  the  learner  to  appropriate 
educational  materials  and  resources  (e.g.,  other  learn¬ 
ers,  learning  objects,  applications,  and  pedagogical 
agents)  through  the  use  of  a  learner  model  (see  Figure 
24.1).  The  components  of  this  four-process  cycle 
include  capture,  analyze,  select,  and  present. 

Capture 

This  process  entails  gathering  personal  information 
about  the  learner  as  the  learner  interacts  with  the  envi¬ 
ronment  (depicted  in  Figure  24.1  by  the  larger  human 
figure).  Relevant  information  can  include  cognitive  as 
well  as  noncognitive  aspects  of  the  learner.  This  infor¬ 
mation  is  used  to  update  internal  models  maintained 
by  the  system. 

Analyze 

This  process  requires  the  creation  and  maintenance  of 
a  model  of  the  learner  in  relation  to  the  domain,  typ¬ 
ically  representing  information  in  terms  of  inferences 
on  current  states.  In  Figure  24.1 ,  this  is  depicted  as  the 
smaller  human  figure  (i.e.,  the  SM). 
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Figure  24.1  Four-process  adaptive  cycle. 

Select 

Information  (i.e.,  content  in  the  broadest  sense)  is 
selected  according  to  the  model  of  the  learner  main¬ 
tained  by  the  system  and  the  goals  of  the  system  (e.g., 
next  learning  object  or  test  item) .  This  process  is  often 
required  to  determine  how  and  when  to  intervene. 

Present 

Based  on  results  from  the  select  process,  specific  con¬ 
tent  is  presented  to  the  learner.  This  entails  appropriate 
use  of  different  media,  devices,  and  technologies  to 


efficiently  convey  information  to  the  learner.  This 
model  accommodates  alternative  scenarios.  Table  24.1 
describes  some  of  these  scenarios  that  involve  different 
types  of  adaptation  starting  with  a  completely  adaptive 
cycle  and  continuing  to  a  nonadaptive  presentation. 
Figure  24.2  illustrates  the  evolving  nature  of  the  four- 
process  adaptive  loop;  that  is,  as  time  passes,  the  learner 
model  becomes  more  refined  and  accurate,  as  repre¬ 
sented  in  the  figure  by  different  degrees  of  saturation. 

In  general,  the  architecture  of  adaptive  applications 
has  evolved  in  a  way  that  reflects  the  evolution  of 
software  systems  architecture;  for  example,  it  is  pos¬ 
sible  to  find  stand-alone  adaptive  applications  where 
the  complete  adaptive  system— including  its  student 
model— resides  in  a  single  machine.  Also,  adaptive 
applications  have  been  implemented  using  a  distrib¬ 
uted  architecture  model.  Some  examples  of  distributed 
applications  include:  (1)  client-server  adaptive  appli¬ 
cations  that  make  use  of  student  modeling  servers  and 
shells  (Fink  and  Kobsa,  2000);  (2)  distributed  agent- 
based  platforms  (Azambuja  Silveira  et  al.,  2002;  Vassi- 
leva  et  al.,  2003);  (3)  hybrid  approaches  involving 
distributed  agents  and  a  student  modeling  server 
(Brusilovsky  et  al.,  2005;  Zapata- Rivera  and  Greer, 
2004);  (4)  peer-to-peer  architectures  (Bretzke  and 
Vassileva,2003);  and  (5)  service-oriented  architectures 
(Froschl,  2005;  Gonzalez  et  al.,  2005;  Kabassi  and 
Virvou,  2003;  Trella  et  al.,  2005;  Winter  et  al.,  2005). 

To  illustrate  how  our  four-process  adaptive  model 
can  accommodate  more  distributed  scenarios,  Figure 
24.3  depicts  an  extended  version  of  our  model,  which 
includes  a  group  of  agents:  application,  personal,  and 
pedagogical.  Each  agent  maintains  a  personal  view  of 


TABLE  24.1 

Scenarios  Represented  in  the  Four-Process  Adaptive  Cycle 


Scenario 

A  complete  cycle  (1 , 2, 3, 4, 5,  and  6) 


Modifying  the  adaptive  cycle  (1,2,3, 
4,5,6,  and  9) 


Monitoring  path  (1,2,  and  3) 


Short  (or  temporary)  memory  cycle 
(1,7,  5,  and  6) 

Short  (or  temporary)  memory,  no 
selection  cycle  (1,  2,  8,  and  6) 


Description 

All  processes  of  the  cycle  are  exercised:  capturing  relevant  information,  analyzing  it,  updating  the  variables, 
selecting  appropriate  resources  and  strategies  that  meet  the  current  needs  of  the  learner,  and  making  them 
available  to  the  student  in  an  appropriate  manner.  This  cycle  will  continue  until  the  goals  have  been  met. 

The  learner  is  allowed  to  interact  with  the  learner  model.  The  nature  of  this  interaction  and  the  effects  on 
the  learner  model  can  vary.  Allowing  human  interaction  with  the  model  may  reduce  the  complexity  of  the 
diagnostic  and  selection  processes  by  decreasing  uncertainty.  It  can  also  benefit  the  learner  by  increasing 
awareness  and  self-reflection. 

The  learner  is  continuously  monitored;  information  gathered  is  analyzed  and  used  to  update  profiles  (e.g., 
homeland  security  surveillance  system,  analyzing  profiles  of  individuals  for  risk-analysis  purposes).  This 
path  can  be  seen  as  a  cycle  that  spins  off  to  a  third  party  instead  of  returning  to  the  learner. 

The  selection  of  content  and  educational  resources  is  done  by  using  the  most  recent  information  (e.g.,  current 
test  results  and  navigation  commands).  No  permanent  learner  model  is  maintained.  Adaptation  is  performed 
using  information  gathered  from  the  latest  interaction  between  learner  and  the  system. 

A  predefined  path  on  the  curriculum  structure  is  followed.  No  learner  model  is  maintained.  This  predefined 
path  dictates  which  educational  resources  and  testing  materials  are  presented  to  the  learner. 


281 


Diego  Zapata-Rivera 


Figure  24.2  Flow  of  diagnosis  and  adaptation  over  time. 


the  learner.  Learner  model  information  and  educational 
resources  can  be  distributed  in  different  places.  Agents 
communicate  with  each  other  directly  or  through  an 
LM  server  to  share  information  that  can  be  used  to 
help  learners  achieve  their  learning  goals. 

SUMMARY  OF  CURRENT 
ADAPTIVE  TECHNOLOGIES 

This  section  describes  adaptive  technologies  currently 
in  use  and  relevant  to  the  context  of  this  chapter.  The 
technologies  have  been  divided  into  two  main  sections: 
soft  and  hard  technologies.  As  described  earlier,  this 
distinction  may  be  likened  to  programs  vs.  devices, 
respectively,  and  may  be  used  across  the  array  of  pro¬ 
cesses  described  in  the  previous  section  (i.e.,  capturing 
student  information,  analyzing  it,  selecting  content,  and 
presenting  it).  The  technologies  selected  for  inclusion 
in  this  section  are  those  that  make  use  of,  to  some  extent, 
a  learner  model  in  its  formulation.  Also,  this  listing  is 
intended  to  be  illustrative  and  not  exhaustive.  For  a  more 
thorough  description  of  adaptive  technologies  in  the 
context  of  e-learning  systems,  see  Froschl  (2005), 
Kobsa  (2006),  Jameson  (2006a),  and  Buxton  (2006),  the 
latter  for  a  directory  of  sources  for  input  technologies. 

Figure  24.4  provides  examples  of  both  soft  and  hard 
technologies  (in  shaded  boxes)  operating  within  an 
adaptive  learning  environment  in  relation  to  our  four- 
process  adaptive  cycle;  for  example,  technologies  for 
analyzing  and  selecting  LM  information  include  Baye¬ 
sian  networks  and  machine  learning  techniques.  These 
technologies  are  examined  in  relation  to  both  learner 
variables  (cognitive  and  noncognitive)  and  modeling 
approaches  (quantitative  and  qualitative).  Similarly, 
examples  of  soft  and  hard  technologies  are  provided  for 
the  processes  of  capturing  and  presenting  information. 


Soft  Technologies 

Soft  technologies  represent  programs  or  approaches 
that  capture,  analyze,  select,  or  present  information. 
Their  primary  goals  are  to  create  LMs  (diagnostic 
function)  and  to  utilize  information  from  LMs  (pre¬ 
scriptive  function). 

Quantitative  Modeling 

In  general,  quantitative  modeling  of  learners  obtains 
estimates  about  the  current  state  of  some  attribute  .  This 
involves  models  and  datasets,  as  well  as  typically  com¬ 
plex  relationships  and  calculations.  To  begin  modeling, 
relationships  are  established  and  tested,  in  line  with  a 
hypothesis  that  forms  the  basis  of  the  model  and  its 
test.  To  quantify  the  relationships,  one  can  use  graph¬ 
ical  models  to  create  graphs  of  the  relationships  and 
statistical  models  that  will  define  quantitative  equa¬ 
tions  of  expected  relationships  to  model  uncertainty 
(for  more,  see  Jameson,  1995). 

Qualitative  Modeling 

Qualitative  modeling  supports  learners  by  constructing 
conceptual  models  of  systems  and  their  behavior  using 
qualitative  formalisms.  According  to  Bredeweg  and 
Forbus  (2003),  qualitative  modeling  is  a  valuable  tech¬ 
nology  because  much  of  education  is  concerned  with 
conceptual  knowledge  (e.g.,  causal  theories  of  physical 
phenomena).  Environments  using  qualitative  models 
may  use  diagrammatic  representations  to  facilitate 
understanding  of  important  concepts  and  relationships. 
Evaluations  in  educational  settings  provide  support  for 
the  hypothesis  that  qualitative  modeling  tools  can  be 
valuable  aids  for  learning  (Frederiksen  and  White, 
2002;  Leelawong  et  al.,  2001). 
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Learner  1 


Learner  2 


Learner  3 


Figure  24.3  Communication  among  agents  and  learners. 


•  Performance  data 

•  Eye/gaze  tracker 

•  Speech  capture 

•  Gesture/posture 

•  Haptic  devices 


Cognitive  Noncognitive 

Variables  Variables 


•  Bayesian  nets 

•  Machine  learning 

•  Stereotypes 

•  Plan  recognition 


•  Personalized 
content 

•  Multiple 
representations 

•  Meaning 
equivalence 

•  Accomodations 


Adaptive  Learning  Environments 


Figure  24.4  Overview  of  technologies  to  support  learner  modeling. 


Cognitive  Modeling 

Cognitive  models  may  be  quantitative  or  qualitative. 
They  help  predict  complex  human  behaviors,  includ¬ 
ing  skill  learning,  problem  solving,  and  other  types  of 
cognitive  activities.  Generally,  cognitive  models  may 


apply  across  various  domains,  serve  different  func¬ 
tions,  and  model  well-  or  ill-defined  knowledge  (e.g., 
design  problems).  The  range  of  cognitive  modeling 
approaches  includes,  for  example,  symbolic,  connec- 
tionist,  hybrid,  neural,  probabilistic,  and  determini¬ 
stic  mathematical  models.  Probably  the  best  known 
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examples  of  cognitive  models  come  from  the  cognitive 
tutoring  research  by  John  Anderson  and  colleagues 
(Anderson,  1993;  Anderson  and  Lebiere,  1998;  Ander¬ 
son  et  al.,  1990, 1995;  Koedinger  and  Anderson,  1998; 
Koedinger  et  al.,  1997;  Matsuda  et  al.,  2005). 

Machine  Learning 

Machine  learning  methods  applicable  for  learner  mod¬ 
eling  include  rule/tree  (analogy)  learning  methods, 
probabilistic  learning  methods,  and  instance  or  case- 
based  learning  approaches.  A  learner  model  can  take 
advantage  of  machine  learning  methods  and  thus 
increase  accuracy,  efficiency,  and  extensibility  in  areas 
not  modeled  before  (Sison  and  Shimura,  1998). 
According  to  Webb  and  colleagues  (2001),  machine 
learning  methods  can  be  used  to  model:  (1)  cognitive 
processes  underlying  the  learner’s  actions,  (2)  differ¬ 
ences  between  the  learner’s  skills  and  expert  skills,  (3) 
the  learner’s  behavioral  patterns  or  preferences,  and 
(4)  other  characteristics  of  the  learner. 

Bayesian  Networks 

Bayesian  networks  (Pearl,  1988)  are  related  to  the 
machine  learning  methods  (see  above)  and  are  used 
within  learner  models  to  handle  uncertainty  by  using 
probabilistic  inference  to  update  and  improve  belief  val¬ 
ues  (e.g.,  regarding  learner  proficiencies).  The  inductive 
and  deductive  reasoning  capabilities  of  Bayesian  nets 
support  “what  if’  scenarios  by  activating  and  observing 
evidence  that  describes  a  particular  case  or  situation  and 
then  propagating  that  information  through  the  network 
using  the  internal  probability  distributions  that  govern 
the  behavior  of  the  Bayesian  net.  Resulting  probabilities 
inform  decision  making,  as  needed  in,  for  example,  our 
select  process.  Examples  of  Bayesian  net  implementa¬ 
tions  for  LMs  may  be  found  in  Conati  et  al.  (2002), 
Shute  et  al.  (2005),  and  VanLehn  and  colleagues  (2005). 

Stereotype  Methods 

A  stereotype  is  a  collection  of  frequently  occurring 
characteristics  of  users  (e.g.,  physical  characteristics, 
social  background,  computer  experience).  Adaptive 
methods  are  used  to  initially  assign  users  to  specific 
classes  (stereotypes)  so  previously  unknown  charac¬ 
teristics  can  be  inferred  on  the  basis  of  the  assumption 
that  they  will  share  characteristics  with  others  in  the 
same  class  (Kobsa,  2006).  Creating  stereotypes  is  a 
common  approach  to  user  modeling,  whereby  a  small 
amount  of  initial  information  is  used  to  assume  a  large 
number  of  default  assumptions.  When  more  informa¬ 


tion  about  individuals  becomes  available,  the  default 
assumptions  may  be  altered  (Rich,  1979).  The  two 
types  of  stereotyping  are  fixed  and  default.  In  fixed 
stereotyping,  learners  are  classified  according  to  their 
performance  into  a  predefined  stereotype  that  is  deter¬ 
mined  by,  for  example,  an  academic  level.  Default 
stereotyping  is  a  more  flexible  approach.  At  the  begin¬ 
ning  of  a  session,  learners  are  stereotyped  to  default 
values,  but,  as  the  learning  process  proceeds  and 
learner  performance  data  is  obtained,  the  settings  of 
the  initial  stereotype  are  gradually  replaced  by  more 
individualized  settings  (Kay,  2000). 

Overlay  Methods 

An  overlay  model  is  a  novice-expert  difference  model 
representing  missing  conceptions,  often  implemented 
as  either  an  expert  model  annotated  for  missing  items 
or  an  expert  model  with  weights  assigned  to  each 
element  in  the  expert  knowledge  base.  One  of  the  first 
uses  of  an  overlay  model  was  done  with  the  WUSOR 
program  (Stansfield  et  al.,  1976).  Current  applications 
of  this  overlay  approach  can  be  found  in  a  variety  of 
research  projects  (Kay,  1999;  Vassileva,  1998;  Zapata- 
Rivera  and  Greer,  2000). 

Plan  Recognition 

A  plan  is  a  sequence  of  actions  to  achieve  a  certain 
goal,  and  it  reflects  the  learner’s  intentions  and  desires. 
Plan  recognition  is  based  on  observing  the  learner’s 
input  actions  and  the  system  then  inferring  all  possible 
learner  plans  based  on  the  observed  actions.  According 
to  Kobsa  (1993),  two  main  techniques  are  used  to 
recognize  the  learner’s  plan:  (1)  establishing  a  plan 
library  containing  all  possible  plans  where  the  selec¬ 
tion  of  the  actual  plan  is  based  on  the  match  between 
observed  actions  and  a  set  of  actions  in  the  library,  and 
(2)  plan  construction,  where  the  system  controls  a 
library  of  all  possible  learner  actions  combined  with 
the  effects  and  the  preconditions  of  these  actions.  Pos¬ 
sible  next  actions  may  be  calculated  by  comparing  the 
effects  of  preceding  actions  with  the  preconditions  of 
actions  stored  in  the  actions  library.  To  read  more  about 
applying  plan-recognition  techniques  in  relation  to 
instructional  planning  efforts,  see  Kobsa  (1993)  and 
Vassileva  and  Wasson  (1992). 

Cumulative/Persistent  Student  Model 

The  cumulative  student  model  represents  the  more  tra¬ 
ditional  approach  where  the  SM  is  analyzed  and 
updated  in  response  to  the  learner’s  activities.  This 
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involves  building  a  student  model  that  captures  and 
represents  emerging  knowledge,  skills,  and  other 
attributes  of  the  learner,  with  the  computer  responding 
to  updated  observations  with  modified  content  that  can 
be  minutely  adjusted.  The  selection  and  presentation 
of  subsequent  content  are  dependent  on  individual 
response  histories  (Shute  and  Psotka,  1996;  VanLehn 
et  al„  2005;  Wenger,  1987). 

Temporary  Student  Model 

Temporary  student  models  usually  do  not  persist  in  the 
system  after  the  learner  has  logged  out.  In  artificial 
intelligence,  formalisms  used  to  describe  the  world 
often  face  something  called  the  frame  problem,  which 
is  the  problem  of  inferring  whether  something  that  was 
true  is  still  true;  for  example,  the  accuracy  of  cumula¬ 
tive  (or  persistent)  student  models  can  degrade  as  stu¬ 
dents  forget  things.  Brooks  (1999)  and  others  have 
circumvented  the  frame  problem  by  using  the  world 
as  its  own  model  (e.g.,  if  you  want  to  know  if  a  window 
is  closed,  check  the  actual  window  rather  than  consult 
an  internal  model).  The  same  idea  applies  to  student 
modeling;  that  is,  if  you  want  to  know  if  a  student  can 
still  multiply  two  fractions,  ask  the  student  to  multiply 
two  fractions.  This  is  what  human  tutors  do,  and  their 
one-time  students  yield  a  student  model  that  is  always 
up  to  date  and  corresponds  to  the  short  memory  cycle 
scenario  shown  in  Table  24.1. 

Pedagogical  Agents 

Pedagogical  means  that  these  programs  are  designed 
to  teach,  and  agent  suggests  that  the  programs  are 
semiautonomous,  possessing  their  own  goals  and  mak¬ 
ing  decisions  on  what  actions  to  take  to  achieve  their 
goals  (i.e.,  a  programmer  has  not  predefined  every 
action  for  them) .  The  current  generation  of  pedagogi¬ 
cal  agents  is  interactive  and  sometimes  animated;  for 
example,  students  can  speak  to  agents  that  can  speak 
back,  often  have  faces  and  bodies,  use  gestures,  and 
can  move  around  a  computer  screen.  Some  well- 
known  agents  include  Steve  (Johnson  et  al.,  2000), 
AutoTutor  (Graesser  et  al.,  2001),  AdeLE  (Shaw  et  al., 
1999),  and  the  Tactical  Language  Training  System 
(Johnson  et  al.,  2004). 

An  interesting  application  of  agent  technologies 
is  teachable  agents,  which  have  been  successfully 
used  to  promote  student  learning  of  mathematics 
and  science  (Biswas  et  al.,  2001).  This  computer- 
based  environment  involves  a  multi-agent  system 
(Betty’s  Brain)  that  implements  a  learning  by  teach¬ 
ing  paradigm.  Students  teach  Betty  by  using  concept 


map  representations  with  a  visual  interface.  Betty 
is  intelligent,  not  because  she  learns  on  her  own  but 
because  she  can  apply  qualitative -reasoning  tech¬ 
niques  to  answer  questions  that  are  directly  related 
to  what  she  has  been  taught.  Another  class  of  agents 
is  emotional  agents  (affective  computing),  which 
have  been  employed  to  support  student  learning 
(Picard,  1997;  Wright,  1997).  Getting  students  moti¬ 
vated  and  sustaining  their  motivation  have  histori¬ 
cally  been  major  obstacles  in  education.  Emotional 
(or  affective)  agents  create  a  learning  environment 
involving  learners  and  interactive  characters  (or 
believable  agents).  Two  important  aspects  of  such 
characters  are  that  they  appear  emotional  and  can 
engage  in  social  interactions.  This  requires  a  broad 
agent  architecture  and  some  degree  of  modeling  of 
other  agents  in  the  environment.  Finally,  pedagogi¬ 
cal  or  virtual  agents  can  collaborate  with  students, 
enabling  new  types  of  interactions  and  support  for 
learning  (Johnson  et  al.,  2000). 

Hard  Technologies 

In  this  section,  we  review  several  hardware-based  tech¬ 
nologies.  These  are  mainly  used  for  input  (i.e.,  data 
capture)  and  presentation  purposes. 

Biologically  Based  Devices 

Some  biologically  based  devices  were  originally 
developed  to  support  learners  with  disabilities  (i.e., 
assistive  technologies);  however,  many  are  being  cre¬ 
ated  or  repurposed  to  support  learner  models  for  both 
cognitive  and  noncognitive  student  data.  As  an  exam¬ 
ple,  obtaining  information  about  where  on  the  com¬ 
puter  the  learner  is  looking  during  learning  provides 
evidence  about  the  learner’s  current  state  and  atten¬ 
tiveness  (for  good  reviews  of  eye-tracking  research, 
see  Conati  et  al.,  2005;  Merten  and  Conati,  2006).  This 
information  can  inform  the  system  about  what  is  the 
next  optimal  path  to  take  for  this  particular  learner.  In 
terms  of  eye-tracking  technology,  eye  movements, 
scanning  patterns,  and  pupil  diameter  are  indicators  of 
thought  and  mental  processing  that  occurs  during 
learning  from  visual  sources  (Rayner,  1998);  conse¬ 
quently,  eye-tracking  data  can  be  used  as  the  basis  for 
supporting  and  guiding  learners  during  the  learning 
process.  To  illustrate  the  approach,  consider  a  novel 
application  of  this  technology  known  as  AdeLE 
(Garcla-Barrios  et  al.,  2004).  This  introduces  a  real¬ 
time,  eye-tracking  procedure  for  intelligent  user  profile 
deduction  as  well  as  the  use  of  a  dynamic  background 
library  to  support  learning. 


285 


Diego  Zapata-Rivera 


Speech-Capture  Devices 

These  devices  allow  users  to  interact  with  the  computer 
via  speech,  instead  of  relying  on  typing  their  input; 
consequently,  this  approach  is  valuable  for  individuals 
with  physical  disabilities  that  preclude  typing,  for 
young  children  who  cannot  yet  type,  and  so  on.  One 
example  project  using  speech-capture  technology  is 
Project  LISTEN  (Literacy  Innovation  that  Speech 
Technology  ENables),  by  Jack  Mostow  and  col¬ 
leagues.  This  is  an  automated  reading  tutor  that  dis¬ 
plays  stories  on  a  computer  screen  and  listens  to  chil¬ 
dren  read  aloud.  It  intervenes  when  the  reader  makes 
mistakes,  gets  stuck,  clicks  for  help,  or  is  likely  to 
encounter  difficulty  (Project  LISTEN,  2006). 

Head-Gesture  Capture  Devices 

Many  computers  currently  are  equipped  with  video 
cameras.  Processing  the  image  provides  a  means  to 
track  head  position  and  movement.  Software  by 
Visionics  Corp.,  for  example,  provides  this  capability. 
Zelinsky  and  Heinzmann  (1996)  developed  a  system 
that  can  recognize  13  different  head  and  face  gestures. 
In  addition,  researchers  in  areas  such  as  animated  ped¬ 
agogical  and  conversational  agents  have  used  sensors 
and  a  video  camera  for  recognizing  facial  gestures. 
This  information  is  used  to  facilitate  human-agent 
interaction  (Cassell  et  al.,  2001). 

Assistive  Technologies 

Disabilities  and  non-native  language  status  can  be 
major  obstacles  to  learning  from  a  computer.  Examin¬ 
ing  adaptations  in  light  of  a  validity  framework  can  be 
valuable  if  not  essential  for  ensuring  effectiveness  (for 
more  on  this  topic,  see  Hansen  and  Mislevy,  2005; 
Hansen  et  al.,  2005).  Currently,  a  growing  number  of 
sites  on  the  Web  provide  information  for  persons  with 
special  needs.  See  the  Special  Needs  Opportunity  Win¬ 
dow  (SNOW,  2006)  website  for  information  about  the 
different  kinds  of  adaptive  technologies  for  people 
with  disabilities. 


Adaptive  Environments 

When  several  technologies  (soft  and  hard)  are  inte¬ 
grated  into  a  single  environment  or  platform  to  accom¬ 
plish  the  goal  of  enhancing  student  learning  via  adap¬ 
tation,  this  is  called  an  adaptive  environment.  We  now 
examine  several  well-known  types  of  adaptive  envi¬ 
ronments. 


Adaptive  Hypermedia  Environment 

Adaptive  hypermedia  environments  or  systems 
(AHSs)  are  extended  from  an  intelligent  tutoring  sys¬ 
tem  foundation  and  combine  adaptive  instructional 
systems  and  hypermedia-based  systems  (Brusilovsky, 
1996).  An  AHS  combines  hypertext  and  hypermedia, 
utilizes  features  of  the  learner  in  the  model,  and  applies 
the  LM  during  adaptation  of  visible  aspects  of  the 
system  to  the  learner.  Brusilovsky  (2001)  distinguished 
between  two  different  types  of  AHS:  (1)  adapting  the 
presentation  of  content  (i.e.,  different  media  formats 
or  orderings) ,  and  (2)  adapting  the  navigation  or  learn¬ 
ing  path,  via  direct  guidance;  hiding,  reordering,  or 
annotating  links;  or  even  disabling  or  removing  links 
(Kinshuk  and  Lin,  2004). 

Adaptive  Educational  Hypermedia  Environment 

A  particular  type  of  AHS  is  an  adaptive  educational 
hypermedia  system  (AEHS).  The  hyperspace  of  AEHS 
is  kept  relatively  small  given  its  focus  on  a  specific 
topic;  consequently,  the  focus  of  the  LM  is  entirely  on 
the  domain  knowledge  of  the  learner  (Brusilovsky, 
1996).  Henze  and  Nejdl  (2003)  have  described  AEHS 
as  consisting  of  a  document  space,  a  learner  model, 
observations,  and  an  adaptation  component.  The  doc¬ 
ument  space  belongs  to  the  hypermedia  system  and  is 
enriched  with  associated  information  (e.g.,  annota¬ 
tions,  domain  or  knowledge  graphs).  The  LM  stores, 
describes,  and  infers  information,  knowledge,  and 
preferences  about  a  learner.  Observations  represent  the 
information  about  the  interaction  between  the  learner 
and  the  AEHS  and  are  used  for  updating  the  LM. 

Collaborative  Learning  Environment 

An  alternative  approach  to  individualized  learning  is 
collaborative  learning— that  is,  the  notion  that  stu¬ 
dents,  working  together,  can  learn  more  than  by  them¬ 
selves,  especially  when  they  bring  complementary, 
rather  than  identical,  contributions  to  the  joint  enter¬ 
prise  (Cumming  and  Self,  1989).  Collaboration  is  a 
process  by  which  “individuals  negotiate  and  share 
meanings  relevant  to  the  problem-solving  task  at  hand” 
(Teasley  and  Roschelle,  1993,  p.  229).  Research  in  this 
area  examines  methods  to  accurately  capture  and  ana¬ 
lyze  student  interactions  in  collaborative  or  distance 
learning  environments;  for  example,  Soller  (2004) 
described  various  techniques  (e.g.,  probabilistic 
machine  learning)  for  modeling  knowledge  sharing 
interactions  among  different  learners. 
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TABLE  24.2 
What  to  Adapt 

Learner  Variables 

Cognitive  abilities  (e.g.,  math  skills,  reading  skills,  cognitive 
development  stage,  problem  solving,  analogical  reasoning) 
Metacognitive  skills  (e.g.,  self-explanation,  self-assessment, 
reflection,  planning) 

Affective  states  (e.g.,  motivation,  attention,  engagement) 
Additional  variables  (e.g.,  personality,  learner  styles,  social 
skills  such  as  collaboration,  and  perceptual  skills) 


Instructional  Variables 

Feedback  type  (e.g.,  hints,  explanations)  and  timing 
(e.g.,  immediate,  delayed) 

Content  sequencing  (e.g.,  concepts,  learning  objects,  tasks, 
items,  and/or  problems  to  solve) 

Scaffolding  (e.g.,  support  and  fading  as  warranted,  rewards) 
View  of  material  (e.g.,  overview,  preview,  review, 
visualization  of  goal  and/or  solution  structure) 


Simulation  and  Immersive  Environment 

Although  simulations  and  immersive  environments 
(e.g.,  virtual  reality)  change  in  response  to  specific 
user  actions,  typically  the  change  is  not  due  to  an 
underlying  LM  but  rather  is  a  function  of  a  predefined 
set  of  rules.  Some  simulations  and  immersive  envi¬ 
ronments,  however,  do  maintain  a  learner  model 
(Rickel  and  Johnson,  1997).  Smithtown  (Shute  and 
Glaser,  1990;  Shute  et  al.,  1989)  is  a  simulated  envi¬ 
ronment  where  students  change  parameters  in  the 
hypothetical  town— such  as  per-capita  income,  pop¬ 
ulation,  the  price  of  gasoline  — and  see  immediate 
changes  in  various  markets,  thus  learning  the  laws  of 
supply  and  demand.  Smithtown  actually  maintains 
two  learner  models:  one  to  model  students’  microeco¬ 
nomic  knowledge  and  skills  and  the  other  to  model 
their  scientific  inquiry  skills. 

As  we  have  just  shown,  many  different  programs 
and  devices  are  available  to  capture,  analyze,  select, 
or  present  information  to  a  learner  based  on  current  or 
perceived  needs  or  wants.  We  now  turn  our  attention 
to  what  some  experts  in  the  field  have  to  say  about 
adaptive  technologies.  Our  goal  is  to  provide  addi¬ 
tional  perspectives  on  relevant  topics. 


EXPERTS’  THOUGHTS 
ON  ADAPTIVE  TECHNOLOGIES 

To  supplement  our  literature  review  on  adaptive  tech¬ 
nologies,  we  asked  leading  adaptive-technology 
experts  to  address  two  questions:  (1)  what  to  adapt 
(i.e.,  what  variables  should  be  taken  into  account  when 
implementing  an  adaptive  system?),  and  (2)  how  to 
adapt  (i.e.,  what  are  the  best  technologies  and  methods 
that  you  use  or  recommend?).  The  experts  who 
responded  to  our  e-mail  queries  include  Cristina  Con- 
ati,  Jim  Greer,  Tanja  Mitrovic,  Julita  Vassileva,  and 
Beverly  Woolf. 


What  to  Adapt? 

Our  experts  responded  to  the  what-to-adapt  question 
in  two  ways:  (1)  input  data  or  learner  variables  to  be 
measured  and  used  as  the  basis  for  adaptation,  and  (2) 
output  or  instructional  variables  that  adapt  to  learners’ 
needs  and  occasionally  to  preferences.  Table  24.2  sum¬ 
marizes  their  collective  responses  and  illustrates  a 
wide  range  of  student  variables  and  adaptive  pedagog¬ 
ical  responses. 

How  to  Adapt? 

Responses  to  this  question  tended  to  focus  on  domain 
independent  approaches  and  technologies  based  on 
analysis  of  student  and  pedagogical  models.  Table  24.3 
lists  the  methods  suggested  by  our  experts  which  rep¬ 
resent  innovative  implementations  of  the  adaptive 
technologies  discussed  earlier. 

In  this  section,  we  have  presented  a  variety  of 
learner  traits  and  states  that  are  judged  relevant  to 
modeling  in  educational  contexts.  In  addition  to  these 
variables  to  be  captured  and  analyzed  in  the  learner 
model,  new  data- mining  technologies  permit  the  dis¬ 
covery  of  even  more  learning  variables  for  a  more 
refined  just-in-time  collection  of  student  information. 
This  will  allow  systems  to  discover  new  things  about 
a  learner  based  on  multiple  sources  of  information 
from  a  single  learner  as  well  as  from  different  learners. 
This  sets  the  stage  for  accomplishing  more  accurate 
individual,  as  well  as  distributed  and  collaborative 
learner  modeling  in  the  future.  Challenges  and  envi¬ 
sioned  futures  are  discussed  next. 


CHALLENGES  AND  FUTURE 
OF  ADAPTIVE  TECHNOLOGIES 

Several  major  obstacles  must  be  overcome  for  the  area 
of  adaptive  technologies  to  move  forward.  As  in  the 
previous  section,  we  have  augmented  this  section  by 
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TABLE  24.3 
How  to  Adapt 

Adaptive  Approach 

Probability  and  decision  theory 


Constraint-based  tutoring 


Concept  mapping 


Unsupervised  machine  learning 


Exploiting  learning  standards 


Analyzing  expert  teachers 


Matching  instructional  support 
to  cognitive  ability 


Rationale 

Rule-based  approaches  are  typically  used  in  adaptive  systems, 
but  using  probabilistic  learner  models  provides  formal  theories 
of  decision  making  for  adaptation.  Decision  theory  takes  into 
account  the  uncertainty  in  both  model  assessment  and 
adaptation  action  outcomes  and  combines  it  with  a  formal 
representation  of  system  objectives  to  identify  optimal  actions. 

The  domain  model  is  represented  as  a  set  of  constraints  on 
correct  solutions;  the  long-term  student  model  contains 
constraint  histories,  and  these  can  be  used  to  generate  the 
system’s  estimate  of  students’  knowledge.  Constraint  histories 
can  also  be  used  to  generate  a  population  student  model  (e.g., 
probabilistic  model),  which  can  later  be  adapted  with  the 
student’s  data  to  provide  adaptive  actions  (e.g.,  problem  or 
feedback  selection). 

Content  (e.g.,  sequences  of  concepts,  learning  objects,  hints)  is 
adapted  to  the  student  by  employing  a  concept  map  with 
prerequisite  relationships,  an  overlay  model  of  the  students’ 
knowledge,  and  a  reactive  planning  algorithm. 

Most  existing  student  models  are  built  by  relying  on  expert 
knowledge,  either  for  direct  model  definition  or  for  labeling 
data  to  be  used  by  supervised  machine  learning  techniques. 
But,  relying  on  expert  knowledge  can  be  very  costly,  and  for 
some  innovative  applications  it  may  be  even  impossible 
because  the  necessary  knowledge  does  not  exist.  An  alternative 
is  to  use  unsupervised  machine  learning  to  build  student  models 
from  unlabeled  data  using  clustering  techniques  for  defining 
classes  of  user  behaviors  during  learning  environment 
interactions. 

Adapting  around  standardized  content  packages  (e.g.,  IMS  QTI, 
IEEE  LOM)  can  make  use  (and  reuse)  of  large  quantities  of 
high-quality  content.  This  is  done  by  extending  the  SCORM 
runtime  environment  specification  to  include  user-modeling 
functionality.  This  permits  content  authors  to  take  advantage 
of  (and  update)  learner  models  in  a  content  management 
system.  Content  recommendations  to  students  are  based  on  the 
learner  model,  and  recommendation  is  done  in  a  lightweight 
manner  with  minimal  demands  on  content  developers. 

Studying  expert  teachers/tutors  is  an  invaluable  source  of 
information  on  how  to  adapt  instructional  content,  but  it  is  not 
always  possible.  Moreover,  for  some  innovative  systems  (e.g., 
educational  games)  human  tutors  may  not  know  how  provide 
effective  pedagogical  support.  An  alternative  is  to  run  so-called 
“Wizard  of  Oz”  studies  to  test  adaptation  strategies  defined  via 
pedagogical  or  cognitive  theories  or  through  intuition. 

Adapting  instructional  support  to  match  students’  cognitive 
needs  (i.e.,  developmental  stage  and  different  abilities)  has 
been  shown  to  promote  better  learning  in  a  couple  of 
experimental  studies.  The  rationale  is  that  if  students  receive 
instructional  support  that  they  are  not  cognitively  ready  to  use 
it  will  be  less  effective  in  promoting  learning. 
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directly  asking  leading  researchers  in  the  held  of  adap¬ 
tive  technologies  to  summarize  their  views  on  chal¬ 
lenges  and  the  future  of  adaptive  technologies.  Our 
experts  include  Anthony  Jameson,  Judy  Kay,  and  Gord 
McCalla. 

Practical  and  Technical  Challenges 

The  main  barriers  to  moving  ahead  in  the  area  of 
adaptive  educational  technologies  are  obtaining  useful 
and  accurate  learner  information  on  which  to  base 
adaptive  decisions,  maximizing  benefits  to  learners 
while  minimizing  costs  associated  with  adaptive  tech¬ 
nologies,  addressing  issues  relating  to  learner  control 
and  privacy,  and  figuring  out  the  bandwidth  problem, 
relating  to  the  scope  of  learner  data.  Each  of  these  is 
now  described. 

Developing  Useful  Learner  Models 

A  core  challenge  of  developing  effective  adaptive  tech¬ 
nologies  is  building  useful  LMs.  According  to  Judy 
Kay  (Kay,  2006),  collecting  meaningful  learning  traces 
(i.e.,  data  obtained  from  records  and  student  log  hies) 
should  help  overcome  this  challenge;  that  is,  the  large 
and  increasing  volume  of  learning  trace  data  associated 
with  individuals  is  generally  trapped  within  logs  of 
individual  tools.  As  a  consequence,  these  data  repre¬ 
sent  a  wasted,  untapped  resource  that  might  be  used 
to  build  rich  LMs.  To  transform  learning  trace  data 
into  a  LM,  a  process  must  interpret  the  data  to  infer 
relevant  learner  attributes,  such  as  knowledge  and  pref¬ 
erences.  This  would  require  the  addition  of  a  knowl¬ 
edge  layer  that  maps  learner  trace  data  (evidence)  to 
a  set  of  inferences  about  the  learner’s  knowledge. 

Acquiring  Valid  Learner  Data 

A  related  barrier  to  overcome  involves  the  acquisition 
of  valid  learner  data,  particularly  when  accomplished 
via  self  reports  (Kay,  2006).  Self-report  information 
has  at  least  two  problems.  First,  learners  may  enter 
inaccurate  data  either  purposefully  (e.g.,  based  on  con¬ 
cerns  about  privacy  or  a  desire  to  present  themselves 
in  a  flattering  light)  or  by  accident  (e.g.,  lack  of  knowl¬ 
edge  about  the  characteristics  they  are  providing).  This 
problem  may  be  solved  by  maintaining  separate  views 
of  the  LM  (e.g.,  the  learner’s  view)  and  providing 
mechanisms  for  reconciling  different  views  into  one 
LM.  Second,  when  additional  interactions  are  required 
during  the  learning  process  (e.g.,  completing  online 
questionnaires),  this  increases  the  time  imposition  and 
can  lead  to  frustration  (Kay,  2006)  as  well  as  poten¬ 
tially  invalid  data  from  students  simply  trying  to  get 


to  the  content  quickly  (Greer  and  Brooks,  2006).  Gath¬ 
ering  such  information,  however,  can  not  only  reduce 
the  complexity  of  diagnosis  but  also  encourage  stu¬ 
dents  to  become  more  active  participants  in  learning 
and  assume  greater  responsibility  for  their  own  LMs. 

Maximizing  Benefits 

Currently,  the  cost  of  developing  and  employing  adap¬ 
tive  technologies  is  often  quite  high  while  the  return 
on  investment  is  equivocal.  This  challenge  is  a  practi¬ 
cal  one— how  to  maximize  the  benefit-to-cost  ratio  of 
adaptive  technologies.  Despite  a  growing  number  of 
adaptive  technologies  available  today,  there  are  too  few 
controlled  evaluations  of  the  technologies  and  systems. 
According  to  Jameson  (2006b),  addressing  this  prob¬ 
lem  should  begin  with  the  identification  of  specific 
conditions  that  warrant  adaptation.  There  are  at  least 
two  standards  of  comparison  for  adaptivity:  (1)  fixed 
sequencing,  and  (2)  learner  control  of  content.  The 
question  is  whether  these  comparison  conditions 
accomplish  the  same  goals  that  could  be  achieved  via 
adaptation.  Jameson  (2006b)  offered  a  strategy  for 
finding  appropriate  adaptivity  applications  — look  for 
cases  where  the  learner  is  in  a  poor  position  to  select 
content  herself,  such  as:  (1)  the  learner  wants  to  choose 
an  item  from  a  very  large  set  of  items  whose  properties 
the  learner  is  not  familiar  with,  and  (2)  the  learner  is 
in  a  situation  lacking  in  the  resources  that  would  be 
required  for  effective  performance. 

Minimizing  Costs 

One  straightforward  way  to  minimize  the  technical 
costs  associated  with  adaptivity  involves  the  use  of 
more  or  less  off-the-shelf  technology  for  user  adaptiv¬ 
ity  (Fink  and  Kobsa,  2000;  Jameson,  2006b).  Another 
cost-minimizing  option  has  been  suggested  by  Greer 
and  Brooks  (2006)  that  involves  leveraging  existing 
content.  They  note  that  adaptive  algorithms  are  often 
domain  specific,  requiring  the  hand  coding  of  content 
to  fit  the  specific  form  of  adaptation.  But,  with  the 
growing  use  of  standardized  content  management  sys¬ 
tems  and  content  available  with  descriptive  metadata, 
the  adaptive  learning  community  has  the  opportunity 
to  get  in  on  the  ground  floor  in  creating  standards  for 
content  adaptation.  Their  approach  involves  creating 
formal  ontologies  to  capture  content,  context,  and 
learning  outcomes.  Instances  of  these  ontologies  can 
be  reasoned  over  by  a  learning  environment  to  provide 
content  (and  peer  help)  recommendations.  Formal 
ontologies  may  then  be  shared  (e.g.,  via  Semantic  Web 
specifications)  and  provide  a  clear  set  of  deduction 
rules  as  well  as  extensive  tool  support. 
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Dealing  with  Learner  Control  Issues 

Learners  often  want  to  control  their  learning  environ¬ 
ment.  One  strategy  that  addresses  this  desire  is  to  allow 
them  partial  control  of  the  process.  According  to  Jame¬ 
son  (2006b),  there  are  several  ways  to  divide  the  job 
of  making  a  learning-path  decision  by  the  system  vs. 
the  learner  (see  Wickens  and  Hollands,  2000,  chap. 
13).  The  system  can  (1)  recommend  several  possibil¬ 
ities  and  allow  the  learner  to  choose  from  that  list,  (2) 
ask  the  learner  for  approval  of  a  suggested  action,  or 
(3)  proceed  with  a  particular  action  but  allow  the 
learner  to  interrupt  its  execution  of  the  action.  Choos¬ 
ing  the  right  point  on  this  continuum  can  be  just  as 
important  as  ensuring  high  accuracy  of  the  system’s 
modeling  and  decision  making. 

Addressing  Privacy  and  Obtrusiveness  Concerns 

When  a  system  has  control  of  the  learning  environment 
and  automatically  adapts,  its  behavior  may  be  viewed 
by  learners  as  relatively  unpredictable,  incomprehen¬ 
sible,  or  uncontrollable  (Jameson,  2006a).  Moreover, 
the  actions  that  the  system  performs  to  acquire  infor¬ 
mation  about  the  learner  or  to  obtain  confirmation  for 
proposed  actions  may  make  the  system  seem  obtrusive 
or  threaten  the  learner’s  privacy  (Kobsa,  2002). 
According  to  Kay  (2006),  one  way  to  address  this 
concern  is  to  build  all  parts  of  the  learner  modeling 
system  in  a  transparent  manner  to  ensure  that  the 
learner  can  scrutinize  the  system’s  management  of 
their  data  and  the  way  in  which  those  data  are  inter¬ 
preted  (Cook  and  Kay,  1994). 

Considering  the  Scope  of  the  Learner  Model 

According  to  McCalla  (2006),  adapting  to  individual 
differences  is  essential  to  making  adaptive  systems 
more  effective.  Despite  some  support  for  this  claim 
(Arroyo  et  al.,  2004,  2006),  significantly  more  exper¬ 
imental  studies  are  needed.  The  traditional  approach 
to  achieving  adaptivity  has  required  the  system  to 
maintain  a  LM  that  captures  certain  characteristics  of 
each  learner  and  then  use  those  data  as  the  basis  for 
adapting  content  (Greer  and  McCalla,  1994).  One 
major  problem  concerns  obtaining  sufficient  band¬ 
width  of  learner  interactions  to  allow  the  capture  of  a 
sufficient  range  of  characteristics  to  paint  an  accurate 
picture  of  the  learner  for  appropriate  adaptation.  Band¬ 
width  in  this  case  refers  to  the  amount  of  relevant 
learner  data  that  can  be  passed  along  a  communica¬ 
tions  channel  in  a  given  period  of  time.  The  bad  news 
is  that  it  is  difficult  to  maintain  a  consistent  model  as 
learners’  knowledge  and  motivations  change  over 


time,  but  the  good  news  is  that  the  bandwidth  problem 
is  diminishing  as  learners  are  currently  spending  more 
time  interacting  with  technology  (McCalla,  2006),  and 
it  is  possible  to  gather  a  broad  range  of  information 
about  them.  Moreover,  learners’  interactions  can  now 
be  recorded  at  a  fine  enough  grain  size  to  produce  more 
depth  in  the  LM.  The  maintenance  problem  may  be 
addressed  by  the  simple  expedient  of  not  trying  to 
maintain  a  persistent  LM  but  instead  making  sense  of 
a  learner’s  interactions  with  an  adaptive  system  just  in 
time  to  achieve  particular  pedagogical  goals. 

Having  summarized  the  main  challenges  surround¬ 
ing  adaptive  technologies  and  possible  ways  to  over¬ 
come  them,  we  now  present  some  visions  of  where  the 
field  may  be  heading  in  the  future,  through  the  eyes 
of  our  three  experts. 

The  Future  of  Adaptive  Technology 

Judy  Kay’s  Views 

A  long-term  vision  for  adaptive  technologies  involves 
the  design  and  development  of  life-long  learner  models 
under  the  control  of  each  learner.  This  idea  draws  on 
the  range  of  learning  traces  available  from  various 
tools  and  contexts.  Learners  could  release  relevant 
parts  of  their  life-long  LMs  to  new  learning  environ¬ 
ments.  Realizing  such  a  vision  requires  that  all  aspects 
of  the  LM  and  its  use  are  amenable  to  learner  control. 
Part  of  the  future  for  LMs  of  this  type  must  include 
the  aggregation  of  information  across  models.  This 
relates  back  to  two  major  challenges:  privacy  and  user 
control  of  personal  data,  as  well  as  its  use  and  reuse. 
An  important  part  of  addressing  these  issues  will  be 
to  build  LMs  and  associated  applications  so  learners 
can  always  access  and  control  their  LMs  and  their  use. 
This  approach  must  go  beyond  just  making  the  LM 
more  open  and  inspectable  to  ensuring  that  learners 
actually  take  control  of  its  use. 

Gord  McCalla’ s  Views 

The  next  envisioned  future  of  adaptive  technologies 
relates  to  the  ecological  approach.  The  learning  envi¬ 
ronment  is  assumed  to  be  a  repository  of  known  learn¬ 
ing  objects,  but  both  learning  object  and  repository  are 
defined  broadly  to  include  a  variety  of  learning  envi¬ 
ronments.  To  further  enhance  flexibility,  the  repository 
may  also  include:  (1)  artificial  agents  representing 
learning  objects,  and  (2)  personal  agents  representing 
users  (e.g.,  learners,  tutors,  and  teachers).  In  this  vision, 
each  agent  maintains  models  of  other  agents  and  users 
that  help  the  agent  achieve  its  goals.  The  models  con¬ 
tain  raw  data  tracked  during  interactions  between  the 
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agents  and  users  (and  other  agents),  as  well  as  infer¬ 
ences  drawn  from  the  raw  data.  Such  inferences  are 
only  made  as  needed  (and  as  resources  allow)  while  an 
agent  is  trying  to  achieve  a  pedagogical  goal.  This  is 
called  active  modeling  (McCalla  et  al.,  2000). 

After  a  learner  has  interacted  with  a  learning  object, 
a  copy  of  the  model  that  her  personal  agent  has  been 
keeping  can  be  attached  to  the  learning  object.  This 
copy  is  called  a  learner  model  instance  and  represents 
the  agent’s  view  of  the  learner  during  this  particular 
interaction,  both  what  the  personal  agent  inferred  about 
the  learner’s  characteristics  and  how  the  learner  inter¬ 
acted  with  the  system.  Over  time,  each  learning  object 
slowly  accumulates  LM  instances  that  collectively  form 
a  record  of  the  experiences  of  many  different  learners 
as  they  have  interacted  with  the  learning  object.  To 
achieve  various  pedagogical  goals,  agents  can  mine  LM 
instances  — attached  to  one  or  more  learning  objects  — 
for  patterns  about  how  learners  interacted  with  the 
learning  objects.  The  approach  is  called  ecological 
because  the  agents  and  objects  in  the  environment  must 
continuously  accumulate  information,  and  there  can  be 
natural  selection  as  to  which  objects  are  useful  or  not. 
Useless  objects  and  agents  can  thus  be  pruned.  More¬ 
over,  ecological  niches  may  exist  that  are  based  on 
goals  (e.g.,  certain  agents  and  learning  objects  are  use¬ 
ful  for  a  given  goal  while  others  are  not).  Finally,  the 
whole  environment  evolves  and  changes  naturally 
through  interaction  among  the  agents  and  ongoing 
attachment  of  LM  instances  to  learning  objects.  The 
ecological  approach  will  require  research  into  many 
issues  (e.g.,  experimentation  to  discover  algorithms  that 
work  for  particular  kinds  of  pedagogical  goals). 

Anthony  Jameson’s  Views 

Although  there  are  many  improvements  that  can  and 
should  be  made  in  terms  of  tools  and  techniques  for 
adaptation,  it  is  even  more  important  to  focus  on  the 
central  problem  of  getting  the  benefits  to  exceed  the 
costs.  Adaptivity,  like  many  other  novel  technologies, 
is  a  technology  that  is  worthwhile,  albeit  within  a 
restricted  range  of  settings.  It  is  thus  critically  impor¬ 
tant  to  clearly  identify  these  settings  and  to  solve  the 
adaptation  problems  therein.  The  ultimate  goal  is  to 
enhance  (in  the  short  or  middle  term)  the  usability  and 
effectiveness  of  real  systems  in  the  real  world. 


SUMMARY  AND  DISCUSSION 

Adaptive  systems  have  been  evolving,  and  will  con¬ 
tinue  to  evolve,  as  new  technologies  appear  in  the  field 
and  old  ones  transform  and  become  more  established. 


The  future  of  the  field  is  wide  open  in  that  it  can  evolve 
in  different  ways  depending  on  factors  such  as  the 
emergence  of  new  technologies,  new  media,  advances 
in  learning,  measurement,  and  artificial  intelligence, 
and  general  policies  and  standards  that  take  hold  (or 
not)  in  relation  to  adaptive  instruction  and  learning. 

One  shift  that  we  see  as  critically  important  to  the 
field,  particularly  in  the  near  term,  is  toward  conducting 
controlled  evaluations  of  adaptive  technologies  and 
systems.  This  will  enable  the  community  to  gauge  the 
value-added  of  these  often  expensive  technologies  in 
relation  to  improving  student  learning  or  other  valued 
proficiencies  (e.g.,  self  esteem,  motivation) .  Our  review 
has  shed  light  on  a  range  of  technologies,  but  the  bot¬ 
tom  line  has  not  yet  been  addressed:  What  works,  for 
whom,  and  under  which  conditions  and  contexts? 

Conati  (2006)  asserts  and  we  agree  that  learners’ 
traits  targeted  for  adaptation  should  clearly  improve 
the  pedagogical  effectiveness  of  the  system.  This 
depends  on  whether  or  not:  (1)  a  given  trait  is  relevant 
to  achieve  the  system’s  pedagogical  goals,  (2)  there  is 
enough  learner  variability  on  the  trait  to  justify  the 
need  for  individualized  interaction,  and  (3)  there  is 
sufficient  knowledge  on  how  to  adapt  to  learner  dif¬ 
ferences  along  this  trait.  Along  the  same  lines,  Jameson 
(2006b)  argued  that  the  benefits  of  adaptation  should 
be  weighed  against  the  cost  of  modeling  each  candi¬ 
date  trait,  to  focus  on  traits  that  provide  the  highest 
benefit  given  the  available  resources. 

A  similar  appeal  for  conducting  controlled  evalu¬ 
ations  was  made  more  than  a  decade  ago,  during  the 
heyday  of  intelligent  tutoring  system  development. 
Now,  as  then,  the  call  for  evaluations  of  adaptive  tech¬ 
nologies  and  systems  is  crucial  for  future  development 
efforts  to  succeed  in  terms  of  promoting  learning. 
Building  adaptive  systems  and  not  evaluating  them  is 
like  “building  a  boat  and  not  taking  it  in  the  water” 
(Shute  and  Regian,  1993,  p.  268).  Evaluation  is  not 
only  important  to  the  future  of  the  field  but  can  also 
be  as  exciting  as  the  process  of  developing  the  tools 
and  systems.  And,  although  the  results  may  be  surpris¬ 
ing  or  humbling,  they  will  always  be  informative. 
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ABSTRACT 

Generational  differences  are  the  subject  of  much  pop¬ 
ular  speculation  but  relatively  little  substantive 
research.  Among  the  speculations  are  suggestions  that 
instructional  designers  should  take  generational  differ¬ 
ences  into  account  when  developing  instruction  and 
that  games  and  simulations  will  be  more  effective  learn¬ 
ing  environments  with  today’s  younger  generation  than 
they  have  been  with  earlier  ones.  This  review  examines 
the  evidence  in  both  the  research  and  popular  literature 
that  supports  (or  fails  to  support)  these  speculations. 
Most  of  the  popular  literature  on  the  subject  of  gener¬ 
ational  differences  appears  to  rest  on  limited  data, 
almost  always  conducted  by  survey  methods  character¬ 
ized  by  a  lack  of  reliability  and  validity  data.  The  most 
recent  research  based  on  rigorous  analysis  of  previous 
psychological  studies  does  yield  some  evidence  of  sub¬ 
stantive  generational  differences,  especially  between 
those  generations  bom  before  and  after  1970.  Recom¬ 
mendations  for  further  research  in  this  area  include 


examining  generational  differences  across  the  whole 
spectrum  of  socioeconomic  status  and  in  international 
contexts,  as  well  as  implementing  innovative  design- 
based  research  approaches  to  accommodating  genera¬ 
tion  differences  in  the  process  of  instructional  design. 


KEYWORDS 

Educational  technology  research:  Research  focused  on 
describing,  predicting,  understanding,  and  designing 
effective  applications  of  technology  to  serve  the  goals 
of  education,  training,  and  performance  support. 

Generational  differences:  The  theory  that  people  bom 
within  an  approximately  20-year  time  period  share  a 
common  set  of  characteristics  based  on  the  historical 
experiences,  economic  and  social  conditions,  techno¬ 
logical  advances,  and  other  societal  changes  they  have 
in  common;  the  term  first  came  into  popularity  in 
the  1960s  when  it  was  used  to  distinguish  the  rebel¬ 
lious  Baby  Boomer  Generation  from  their  parents. 
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Instructional  design:  The  systematic  process  of  ana¬ 
lyzing,  designing,  developing,  implementing,  and 
evaluating  instruction;  also  known  as  instructional 
systems  design. 

INTRODUCTION 

Generational  differences,  especially  the  differences 
between  generations  defined  variously  as  the  Baby 
Boom  Generation,  Generation  X,  and  the  Millennial 
Generation,  are  widely  discussed  in  the  popular  press 
as  well  as  in  a  few  scholarly  publications.  Business 
entrepreneurs  as  well  as  social  pundits  have  speculated 
that  the  various  generations  of  students  enrolled  in 
today’s  higher  education  institutions  as  well  as  the 
different  generations  of  employees  in  the  corporate 
workplace  require  a  different  approach  to  education 
and  training.  Extensions  of  this  speculation  are  that 
instructional  designers  should  take  generational  differ¬ 
ences  into  account  when  developing  instruction  and 
that  generational  differences  represent  a  meaningful 
variable  for  research  designed  to  examine  the  differ¬ 
ential  efficacy  of  various  applications  of  educational 
technology.  The  purpose  of  this  chapter  is  to  review 
the  research  and  popular  literature  that  support  (or  fail 
to  support)  this  speculation. 

Definitions  of  Generations 

The  nomenclature  used  to  label  various  generations  is 
not  standardized  because  the  different  researchers  and 
consultants  exploring  and  writing  about  generational 


differences  have  come  up  with  a  variety  of  different 
names  to  label  the  specific  generations.  In  addition, 
there  is  significant  disagreement  among  the  various 
authors  about  which  span  of  years  should  be  encom¬ 
passed  within  any  one  generation.  Table  25.1  presents 
a  comparison  of  the  different  labels  given  to  various 
generations  as  well  as  the  different  chronological 
schemes  used  to  assign  people  born  in  any  given  year 
to  one  of  the  generations  defined  by  the  sources  listed 
in  the  first  column.  As  illustrated  in  the  table,  some 
authorities  state  that  Generation  Y  workers  were  born 
as  early  as  1978  (Martin  and  Tulgan,  2002),  whereas 
others  (Howe  and  Strauss,  2000)  have  established  a 
start  date  as  late  as  1982.  Interestingly,  both  of  these 
sources  (Howe  and  Strauss,  2000;  Martin  and  Tulgan, 
2002)  define  the  end  date  for  Generation  Y  as  2000. 
The  focus  of  this  review  is  on  the  three  middle  gener¬ 
ations  (Boomer,  X,  and  Millennial),  because  members 
of  these  three  generations  will  be  in  higher  education 
and  the  workforce  over  the  next  15  years.  For  purposes 
of  this  review,  the  generations  will  be  labeled  and 
delineated  as  illustrated  in  Table  25.2;  however,  other 
synonymous  terms  for  the  three  major  generations  are 
used  in  various  sections  of  this  review,  especially  when 
referring  to  specific  literature  resources  that  employ 
alternative  terms. 

Do  Generations  Really  Differ? 

There  is  relatively  little  consensus  of  opinion  and 
scholarship  about  whether  generational  differences 
exist  that  are  worth  taking  into  consideration  in  the 
workplace,  colleges,  and  universities,  and  other  con- 


TABLE  25.1 

Generational  Labels  and  Dates  Reported  in  Different  Sources 


Source 

Labels 

Howe  and  Strauss  (2000) 

Silent  Generation 

(1925-1943) 

Boom  Generation 

(1943-1960) 

13th  Generation 

(1961-1981) 

Millennial  Generation 

(1982-2000) 

— 

Lancaster  and  Stillman  (2002) 

Traditionalists 

(1900-1945) 

Baby  Boomers 
(1946-1964) 

Generation  Xers 

(1965-1980) 

Millennial  Generation;  Echo 
Boomer;  Generation  Y;  Baby 
Busters;  Generation  Next 
(1981-1999) 

Martin  and  Tulgan  (2002) 

Silent  Generation 
(1925-1942) 

Baby  Boomers 
(1946-1960) 

Generation  X 
(1965-1977) 

Millennials  (1978-2000) 

— 

Oblinger  and  Oblinger  (2005) 

Matures  (<1946) 

Baby  Boomers 
(1947-1964) 

Gen-Xers 

(1965-1980) 

Gen-Y;  NetGen;  Millennials 
(1981-1995) 

Post-Millennials 

(1995-present) 

Tapscott  (1998) 

Baby  Boom 
Generation 

(1946-1964) 

Generation  X 
(1965-1975) 

Digital  Generation 
(1976-2000) 

Zemke  et  al.  (2000) 

Veterans 

(1922-1943) 

Baby  Boomers 
(1943-1960) 

Gen-Xers 

(1960-1980) 

Nexters  (1980-1999) 

— 
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TABLE  25.2 

Generational  Labels  and  Dates 

Label:  Mature  Generation  Boom  Generation  Generation  X  Millennial  Generation  Generation  Z 

Date:  1925-1945  1946-1964  1965-1980  1981-2000  2001-present 


texts.  For  starters,  as  noted  above,  notable  differences 
can  be  found  in  the  labels  used  for  the  generations  by 
different  researchers  and  experts  as  well  as  across  the 
spans  of  years  used  to  delineate  the  different  genera¬ 
tions.  In  addition,  Lancaster  and  Stillman  (2002), 
among  others,  have  made  a  distinction  between  people 
born  on  the  edges  of  various  generational  spans  and 
those  caught  between  two  generations  by  labeling 
them  cuspers.  Lancaster  and  Stillman  (2002)  main¬ 
tained  that  the  Traditionalist/Baby  Boom  cuspers  were 
born  between  1940  and  1945,  Baby  Boom/Generation 
X  cuspers  were  bom  between  1960  and  1965,  and  the 
Generation  X/Millennial  cuspers  were  bom  between 
1975  and  1980.  The  existence  of  cuspers  further  limits 
the  generalization  of  generational  traits  to  individuals 
based  on  their  categorization  with  regard  to  generation. 

Despite  the  lack  of  consistency  in  nomenclature 
and  chronology,  most  authorities  agree  that  a  great  deal 
of  variance  exists  among  the  distinguishing  character¬ 
istics  within  any  given  generation;  thus,  it  is  inadvis¬ 
able  to  assume  that  if  a  person  was  bom  in  1985  then 
that  person  would  have  most  of  the  characteristics  of 
a  Gen  Y  person  or  that  someone  born  in  1960  (and 
thus  a  late  Boomer)  would  be  not  as  technologically 
sophisticated  as  a  person  bom  into  Gen  X  or  the  Mil¬ 
lennial  Generation.  In  other  words,  it  is  definitely 
unjustified  to  make  assumptions  about  any  one  indi¬ 
vidual  based  on  that  person’s  membership  in  a  chro¬ 
nological  generational  cohort. 

Indeed,  birth  years  are  only  one  factor  to  consider 
in  distinguishing  among  generations,  and  a  relatively 
minor  one  at  that.  Instead,  most  experts  have  argued 
that  generations  are  shaped  much  more  by  history  than 
by  chronological  dates.  According  to  Howe  and 
Strauss  (2000),  three  attributes  that  more  clearly  iden¬ 
tify  the  nature  of  a  generation  than  years  of  birth  are: 

•  Perceived  membership  —  The  self-perception 
of  membership  within  a  generation  that 
begins  during  adolescence  and  coalesces 
during  young  adulthood 

•  Common  beliefs  and  behaviors— The  atti¬ 
tudes  (toward  family,  career,  personal  life, 
politics,  religion,  etc.)  and  behaviors 
(choices  made  in  regard  to  jobs,  marriage, 
children,  health,  crime,  sex,  dmgs,  etc.)  that 
characterize  a  generation 


•  Common  location  in  history— The  turning 
points  in  historical  trends  (e.g.,  from  liberal 
to  conservative  politics)  and  significant 
events  (e.g.,  the  Vietnam  War)  that  occur 
during  a  generation’s  formative  years  (ado¬ 
lescence  and  young  adulthood) 

Many  of  the  theories  and  conclusions  about  gen¬ 
eration  differences  reported  in  popular  books  such  as 
those  written  by  Howe  and  Strauss  (2000)  as  well  as 
the  information  found  in  research  papers  are  based  on 
survey  data  collected  from  young  people  from  middle 
and  upper  middle  socioeconomic  groups.  No  national 
surveys  related  to  generational  differences  that  cut 
across  the  full  range  of  socioeconomic  status  (SES) 
groups  can  be  found  in  the  literature.  In  addition,  the 
majority  of  the  research  papers  and  other  literature 
sources  are  focused  on  people  who  will  enter  colleges 
and  universities  and  eventually  pursue  white-collar  or 
knowledge-worker  careers.  Virtually  no  literature  can 
be  found  that  specifically  addresses  the  generational 
differences  among  those  who  will  not  enter  higher 
education  or  who  are  more  likely  to  assume  blue-collar 
jobs.  In  addition,  the  literature  is  remarkably  ethno¬ 
centric  ,  taking  a  perspective  that  ignores  generational 
differences  in  cultures  other  than  those  associated  with 
Western  capitalism. 

As  an  example,  a  survey  conducted  by  Universum 
Communications  (see  http://www.universumusa. 
com/)  involved  37,000  “soon-to-graduate  college  stu¬ 
dents  of  the  millennial  generation”  (Stone,  2006,  p. 
1)  from  207  colleges  and  universities  in  the  United 
States.  According  to  the  survey  conducted  in  the 
spring  of  2006,  the  career  goals  of  today’s  Net  Gen 
college  graduates  are  “to  balance  their  personal  and 
professional  life  (59  percent),  pursue  further  educa¬ 
tion  (46  percent),  build  a  sound  financial  base  (32 
percent),  and  contribute  to  society  (27  percent).” 
Whether  these  types  of  career  ambitions  would  sur¬ 
face  among  non-college  workers  in  the  same  Millen¬ 
nial  cohort  has  not  been  addressed,  nor  have  research¬ 
ers  investigated  the  ambitions  of  people  in  developing 
countries  by  generational  membership. 

The  most  frequently  cited  sources  for  issues  related 
to  generational  differences  are  the  works  of  Neil  Howe 
and  William  Strauss,  authors  of  a  series  of  popular 
books  that  includes: 
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•  Generations:  The  History  of  America’s 
Future,  1584  to  2069 

•  13th  Gen:  Abort,  Retry,  Ignore,  Fail? 

•  The  Fourth  Turning,  Millennials  Rising:  The 
Next  Great  Generation 

•  Millennials  Go  to  College:  Strategies  for  a 
New  Generation  on  Campus 

•  Millennials  and  the  Pop  Culture 

Their  most  controversial  projection  is  that  the  Mil¬ 
lennial  Generation  will  be  the  most  successful  gener¬ 
ation  since  the  so-called  greatest  generation  that  fought 
World  War  II.  Howe  and  Strauss  (2000,  p.  4)  predict 
a  rosy  future  for  Generation  Y: 

As  a  group,  Millennials  are  unlike  any  other  youth  gen¬ 
eration  in  living  memory.  They  are  more  numerous,  more 
affluent,  better  educated,  and  more  ethnically  diverse. 
More  important,  they  are  beginning  to  manifest  a  wide 
array  of  positive  social  habits  that  older  Americans  no 
longer  associate  with  youth,  including  a  new  focus  on 
teamwork,  achievement,  modesty,  and  good  conduct. 

Secondary  sources  such  as  Forrester  Research,  an 
independent  technology  and  market  research  company, 
have  extrapolated  from  Howe  and  Strauss  and  other 
pollsters  to  make  optimistic  conjectures  about  the  work 
habits  of  Generation  Y  (Schooley,  2005,  Executive 
Summary): 

The  “Millennials”— born  between  1980  and  2000— have 
an  innate  ability  to  use  technology,  are  comfortable  multi¬ 
tasking  while  using  a  diverse  range  of  digital  media,  and 
literally  demand  interactivity  as  they  construct  knowl¬ 
edge.  Millennials  lack  the  workaholic  drive  of  their 
burned-out  predecessors,  but  they  compensate  by  using 
many  technologies — often  simultaneously — to  get  the 
job  done  quickly  and  have  a  personal  life  as  well. 

Of  course,  some  critics  take  issue  with  such  opti¬ 
mistic  predictions,  especially  with  the  generalizations 
made  from  limited  survey  data  by  Howe  and  Strauss. 
Brooks  (2000,  p.  1) ,  a  reviewer  for  The  New  York  Tunes, 
wrote: 

Now  if  you're  going  to  get  through  this  book,  you  can't 
hurl  it  against  the  wall  every  time  Howe  and  Strauss  make 
a  huge  generalization  about  an  entire  age  category, 
because  you'll  either  knock  down  your  house  or  tear  your 
rotator  cuff.  This  is  not  a  good  book,  if  by  good  you  mean 
the  kind  of  book  in  which  the  authors  have  rigorously 
sifted  the  evidence  and  carefully  supported  their  asser¬ 
tions  with  data.  But  it  is  a  very  good  bad  book.  It’s  stuffed 
with  interesting  nuggets.  It's  brightly  written.  And  if  you 
get  away  from  the  generational  mumbo  jumbo,  it  illumi¬ 
nates  changes  that  really  do  seem  to  be  taking  place. 


O’Neill  (2000;  see  also  http://www.millennialsris- 
ing.com  and  www.fourthturning.com)  also  criticized 
Howe  and  Strauss: 

This  latest  book,  like  the  others,  mixes  statistics  from 
responsible  data-collectors  such  as  the  Institute  for  Social 
Research  with  results  from  scientifically  unrepresentative 
surveys  of  200  high  schoolers  in  Virginia  and  from  post¬ 
ings  on  their  websites. 

Twenge  (2006,  pp.  6-7)  criticized  the  optimistic 
conclusions  drawn  by  Howe  and  Strauss: 

My  perspective  on  today’s  young  generations  differs 
from  that  of  Neil  Howe  and  William  Strauss,  who  argue 
in  their  2000  book,  Millennials  Rising,  that  those  born 
since  1982  will  usher  in  a  return  to  duty,  civic  responsi¬ 
bility,  and  teamwork.  Their  book  is  subtitled  The  Next 
Great  Generation  and  contends  that  today’s  young  peo¬ 
ple  will  resemble  the  generation  who  won  World  War  II. 

I  agree  that  in  an  all-encompassing  crisis  today's  young 
people  would  likely  rise  to  the  occasion  -  people  usually 
do  what  needs  to  be  done.  But  I  see  no  evidence  that 
today’s  young  people  feel  much  attachment  to  duty  or  to 
group  cohesion.  Instead,  as  you’ll  see  in  the  following 
pages,  young  people  have  been  consistently  taught  to  put 
their  own  needs  first  and  to  focus  on  feeling  good  about 
themselves.  This  is  not  an  attitude  conducive  to  following 
social  rules  or  favoring  the  group's  needs  over  the  indi¬ 
vidual's.  . .  .Our  childhood  of  constant  praise,  self-esteem 
boosting,  and  unrealistic  expectations  did  not  prepare  us 
for  an  increasingly  competitive  workplace  and  the  eco¬ 
nomic  squeeze  created  by  sky-high  housing  costs  and 
rapidly  accelerating  health  care  costs.  After  a  childhood 
of  buoyancy,  GenMe  is  working  harder  to  get  less. 

It  is  difficult  to  find  the  details  about  the  surveys 
upon  which  most  of  the  popular  books  about  genera¬ 
tional  differences  are  based.  By  contrast,  Howe  and 
Strauss  (2000),  authors  of  Millennials  Rising,  do  pro¬ 
vide  information  about  their  surveys  on  the  companion 
website  for  their  book  at:  http://www.millennialsris- 
ing.com/.  They  report  that  a  total  of  202  teachers  and 
655  students  from  the  class  of  2000  completed  the 
surveys  upon  which  their  book  is  based.  Some  critics 
have  suggested  that  the  fact  that  all  the  surveys  were 
conducted  in  Fairfax  County,  Virginia,  an  affluent  sub¬ 
urban  area  just  outside  Washington,  D.C.,  limits  the 
generalizability  of  the  optimistic  conclusions  that 
Howe  and  Strauss  have  drawn  to  less  affluent  areas  of 
the  country.  Howe  and  Strauss  have  defended  their 
results  as  follows  (http://www.millennialsrising.com/ 
survey,  shtml): 

Comprising  the  western  suburbs  of  Washington,  D.C., 
Fairfax  County  (with  a  total  population  of  one  million, 
including  170,000  school-age  kids)  has  one  of  the  largest 
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TABLE  25.3 

Generational  Differences  on  12  Criteria 


Views  toward 

Level  of  trust 
Loyalty  to  institutions 
Most  admire 
Career  goals 
Rewards 

Parent-child  involvement 

Having  children 

Family  life 

Education 

Evaluation 

Political  orientation 

The  big  question 


Boomers 

Confident  of  self,  not  authority 

Cynical 

Taking  charge 

Build  a  stellar  career 

Title  and  corner  office 

Receding 

Controlled 

Indulged  as  children 

Freedom  of  expression 

Once  a  year  with  documentation 

Attack  oppression 

What  does  it  mean? 


Gen  Xers 

Low  toward  authority 
Considered  naive 
Creating  enterprise 
Build  a  portable  career 
Freedom  not  to  do 
Distant 
Doubtful 

Alienated  as  children 
Pragmatic 

“Sorry,  but  how  am  I  doing?” 
Apathetic,  individual 
Does  it  work? 


Millennial 

High  toward  authority 
Committed 

Following  a  hero  of  integrity 
Build  parallel  careers 
Meaningful  work 
Intruding 
Definite 

Protected  as  children 
Structure  of  accountability 
Feedback  whenever  I  want  it 
Crave  community 
How  do  we  build  it? 


Source:  Debard,  R.D.,  in  Serving  the  Millennial  Generation:  New  Directions  for  Student  Services ,  Debard,  R.D.  and  Coomes, 
M.D.,  Eds.,  Jossey-Bass,  San  Francisco,  CA,  2004,  pp.  33 — 45 .  With  permission. 


and  most  renowned  school  systems  in  the  nation.  Its 
student  population  is  ethnically  diverse  — 67  percent  Cau¬ 
casian,  13  percent  Asian,  10  percent  Hispanic,  8  percent 
African-American,  and  3  percent  other.  Three  of  every 
ten  students  live  in  homes  where  a  language  other  than 
English  is  spoken.  Though  Fairfax  County  is  relatively 
well-to-do,  with  a  median  household  income  nearly  twice 
the  national  average,  eighteen  percent  of  all  students  are 
eligible  for  free  or  reduced-price  school  lunches,  nine  per¬ 
cent  live  in  households  with  annual  incomes  under  $25,000, 
and  five  percent  live  beneath  the  official  poverty  line. 

Whereas  Howe  and  Strauss  have  focused  on  high- 
school  students,  others  have  focused  their  generational 
research  on  higher  education  students.  In  2005,  Diana 
G.  Oblinger,  Vice  President  of  EDUCAUSE,  a  non¬ 
profit  organization  focused  on  improving  the  applica¬ 
tion  of  information  technology  in  higher  education, 
and  James  L.  Oblinger,  Chancellor  of  North  Carolina 
State  University,  coedited  a  book  titled  Educating  the 
Net  Gen  (see  http://www.educause.edu/educatingth- 
enetgen).  The  editors  and  the  various  authors  of  the 
chapters  in  this  book  argue  that  the  current  generation 
of  students  entering  higher  education  has  information 
technology  skills  that  exceed  those  of  the  faculty  mem¬ 
bers  who  will  be  teaching  them,  a  trend  that  demands 
significant  changes  in  the  way  that  programs,  courses, 
and  learning  environments  are  designed  and  imple¬ 
mented.  Oblinger  and  Oblinger  (2005,  p.  2.5),  how¬ 
ever,  admitted  that  the  technological  sophistication  of 
today’s  Net  Gen  students  may  be  somewhat  superficial: 

Having  grown  up  with  widespread  access  to  technology, 
the  New  Gen  is  able  to  intuitively  use  a  variety  of  IT 
devices  and  navigate  the  Internet.  Although  they  are  com¬ 


fortable  using  technology  without  an  instruction  manual, 
their  understanding  of  the  technology  or  source  quality 
may  be  shallow. 

Debard  (2004)  has  summarized  the  values  of 
Boomers,  Gen  Xers,  and  Gen  Yers  that  he  thinks  have 
implications  for  higher  education  administrators  and 
faculty  (see  Table  25.3);  for  example,  Debard  main¬ 
tained  that  members  of  the  Millennial  Generation  have 
high  trust  in  authority  and  are  primarily  rewarded  by 
work  they  consider  meaningful. 

Although  it  is  obvious  that  some  in  higher  educa¬ 
tion  are  taking  generational  differences  seriously,  oth¬ 
ers  view  efforts  to  adjust  teaching  and  other  campus 
activities  to  the  needs  and  preferences  of  the  new  gen¬ 
eration  of  students  with  skepticism.  Academic  critics 
of  optimistic  predictions  about  the  Millennial  Gener¬ 
ation  may  be  closer  to  the  realities  of  higher  education 
challenges  today  than  pollsters  and  pundits  such  as 
Howe  and  Strauss  (2000).  In  2006,  The  Chronicle  of 
Higher  Education  published  a  special  supplement 
focused  on  the  relationship  between  school  and  col¬ 
lege.  The  cover  of  the  March  10,  2006,  supplement 
reads  as  follows: 

The  facts  are  stunning.  More  than  40  percent  of  students 
arrive  on  college  campuses  needing  remedial  work. 
Only  about  half  of  the  high  school  graduates  who  enter 
college  have  pursued  a  college-preparatory  curriculum. 
Colleges  and  universities  spend  billions  of  dollars  a  year 
trying  to  bring  those  unprepared  students  up  to  speed. 

But  most  of  those  institutions  play  only  a  minor  role  in 
the  movement  to  reform  the  schools.  The  whole  nation 
is  suffering  as  a  result.  America  is  fast  losing  its  lead  in 
critical  fields. 
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A  similarly  dismal  picture  of  the  lack  of  preparation 
that  America’s  students  entering  colleges  and  universi¬ 
ties  today  exhibit  was  presented  by  Hersh  and  Merrow 
(2005)  in  a  two-hour  PBS  television  special  and  book 
that  have  the  same  title,  Declining  by  Degrees:  Higher 
Education  at  Risk.  The  book  and  video  present  inter¬ 
views  and  extensive  data  that  undermine  the  argument 
that  the  Millennial  Generation  is  well  prepared  and 
motivated  to  succeed  in  higher  education. 

Taking  an  innovative  approach  to  generational 
research,  Twenge  (2006)  presented  convincing  evi¬ 
dence  that  most  of  today’s  American  young  people 
(which  she  labels  Generation  Me  or  GenMe)  have  been 
raised  to  think  that  they  will  be  highly  successful,  even 
stars,  although  the  reality  is  that  they  will  find  it  more 
difficult  than  ever  to  get  into  and  afford  the  best  col¬ 
leges,  find  a  high-paying  and  personally  rewarding  job, 
and  buy  a  decent  home.  Whereas  most  other  genera¬ 
tional  researchers  have  taken  a  cross-sectional 
approach  to  their  research  wherein  they  distribute  sur¬ 
veys  to  or  conduct  interviews  with  members  of  differ¬ 
ent  generations  at  the  same  point  in  time,  Twenge 
(2006)  has  painstakingly  analyzed  the  results  of  studies 
that  involved  school  children,  adolescents,  and  college 
students  completing  well -designed,  validated  ques¬ 
tionnaires  in  the  1950s,  1960s,  1970s,  1980s,  1990s, 
and  today.  This  enabled  her  to  compare,  for  example, 
the  attitudes  of  the  Baby  Boomer  Generation  expressed 
when  they  were  adolescents  with  the  attitudes  of 
GenMe  expressed  during  their  adolescence.  A  sample 
of  her  findings  derived  from  data  collected  from  1 .3 
million  young  Americans  since  the  1950s  include: 

•  In  2002, 74%  of  high-school  students  admit¬ 
ted  to  cheating  whereas  in  1969  only  34% 
admitted  such  a  failing  (p.  27). 

•  In  1967,  86%  of  incoming  college  students 
said  that  “developing  a  meaningful  philoso¬ 
phy  of  life”  was  an  essential  life  goal 
whereas  in  2004  only  42%  of  GenMe  fresh¬ 
men  agreed  (p.  48). 

•  In  2004, 48%  of  American  college  freshmen 
reported  earning  an  A  average  in  high 
school,  whereas  in  1968  only  18%  of  fresh¬ 
men  reported  being  an  A  student  in  high 
school  (p.  63). 

•  In  the  1950s,  only  12%  of  young  teens 
agreed  with  the  statement  “I  am  an  important 
person”  whereas  by  the  late  1980s,  80% 
claimed  they  were  important  (p.  69). 

•  In  the  1960s,  42%  of  high-school  students 
expected  to  work  in  professional  jobs, 
whereas  in  the  late  1990s  70%  of  high  school¬ 
ers  expected  to  work  as  professionals  (p.  78). 


•  In  a  recent  poll,  53%  of  GenMe  mothers 
agreed  with  the  statement  that  a  person’s 
main  responsibility  is  to  themselves  and 
their  children  rather  than  making  the  world 
a  better  places,  whereas  only  28%  of 
Boomer  mothers  agreed  (p.  78). 

Turning  to  the  corporate  world,  a  veritable  cottage 
industry  has  sprung  up  around  the  theory  that  manag¬ 
ers  should  attend  to  generational  differences  and  make 
changes  to  their  management  styles  and  other  business 
practices  accordingly.  Among  the  major  providers  of 
consulting  services  focused  on  generational  differ¬ 
ences  are: 

•  Eric  Chester  (2002),  author  of  Employing 
Generation  Why?  (http://www.generation- 
why.com/) 

•  Lynne  Lancaster  and  David  Stillman  (2002), 
authors  of  When  Generations  Collide  (http:// 
www.generations  .com/) 

•  Carolyn  Martin  and  Bruce  Tulgan  (2002), 
authors  of  Managing  the  Generational  Mix 
(http  ://www.rainmakerthinking  .com/) 

•  Claire  Raines  (2003),  author  of  Connecting 
Generations :  The  Sourcebook  for  a  New 
Workplace  (http://www.generationsatwork. 
com/) 

Summing  up  generational  differences  at  work  and 
the  implications  for  corporate  business  managers,  Lan¬ 
caster  and  Stillman  (2002)  also  described  a  set  of  dis¬ 
tinguishing  characteristics  among  the  three  genera¬ 
tions  of  interest  in  this  review  (see  Table  25.4). 
Unfortunately,  as  with  the  predictions  made  by  Howe 
and  Strauss  (2000),  little  substantive  research  backs  up 
the  generational  distinctions  that  Lancaster  and  Still¬ 
man  (2002),  Chester  (2002),  Martin  and  Tulgan 
(2002),  and  Raines  (2003)  make  for  managers  and 
trainers  in  the  corporate  world. 


DISCUSSION 

One  of  the  most  frustrating  aspects  of  the  research 
focused  on  cognitive,  affective,  and  psychomotor  dif¬ 
ferences  among  the  Boomer,  X,  and  Millennial  gen¬ 
erations  is  that  for  the  most  part  it  is  based  on  small, 
highly  selective  surveys  rather  than  national  datasets 
that  cut  across  important  variables  such  as  socioeco¬ 
nomic  status  and  level  of  education.  Some  critics  have 
been  especially  critical  of  the  data  used  by  Howe  and 
Strauss  (2000)  for  making  their  optimistic  predictions 
about  the  Millennial ,  maintaining  that  their  data  are 
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TABLE  25.4 

Lancaster  and  Stillman’s  Generational  Differences 


Factor 

Attitude 

Overview 


Description 


Work  habits 


Baby  Boomers 

Optimistic 

They  believe  in  possibilities,  and  often 
idealistically  strive  to  make  a  positive 
difference  in  the  world.  They  are  also 
competitive  and  seek  ways  to  change  the 
system  to  get  ahead. 

Numbered  at  80  million,  the  largest  of  the 
groups,  Boomers  were  born  between 
1946  and  1964.  They  were  influenced 
by  Martin  Luther  King,  JFK,  Gloria 
Steinem,  and  The  Beatles.  Places  such 
as  the  Hanoi  Hilton,  Woodstock,  and 
Kent  State  resonate  for  this  group. 
Television  changed  their  world 
dramatically.  In  general,  they  can  be 
described  as  optimistic.  This  was  the 
generation  that  believed  anything  was 
possible — that  they  really  could  change 
the  world. 


They  have  an  optimistic  outlook. 

They  are  hard  workers  who  want  personal 
gratification  from  the  work  they  do. 
They  believe  in  self-improvement  and 
growth. 


Generation  Xers 

Skeptical 

The  most  misunderstood  generation,  they 
are  very  resourceful  and  independent 
and  do  not  depend  on  others  to  help 
them  out. 

Born  between  1965  and  1980,  this 
relatively  small  (46  million)  segment  of 
the  workforce  saw  the  likes  of  Bill 
Clinton,  A1  Bundy,  Madonna,  Beavis 
and  Butthead,  and  Dennis  Rodman 
make  headlines  during  their  formative 
years.  Their  world  shape  changed  to 
include  the  former  Soviet  Union, 
Lockerbie,  Scotland,  and  the 
Internet— in  fact,  this  is  the  generation 
that,  more  than  any  other,  is  defined  by 
media  and  technology.  For  Gen-Xers, 
the  watchword  is  skepticism— this 
group  puts  more  faith  in  the  individual, 
in  themselves,  than  in  any  institution, 
from  marriage  to  their  employer. 

They  are  aware  of  diversity  and  think 
globally. 

They  want  to  balance  work  with  other 
parts  of  life. 

They  tend  to  be  informal. 

They  rely  on  themselves. 

They  are  practical  in  their  approach  to 
work. 

They  want  to  have  fun  at  work. 

They  like  to  work  with  the  latest 
technology. 


Millennial  Generation 

Realistic 

They  appreciate  diversity,  prefer  to 
collaborate  instead  of  being  ordered, 
and  are  very  pragmatic  when  solving 
problems. 

The  youngest  members  of  what  will  be 
the  next  Boomer  wave,  some  76  million 
Millennial  were  born  between  1981 
and  1999.  Although  they  are  just  starting 
to  trickle  into  the  workforce,  this  group 
grew  up  with  everybody  from  Prince 
William  to  Winky  Tinky,  Felicity, 
Marilyn  Manson,  Venus  and  Serena 
Williams,  and  Britney  Spears.  They 
have  already  lived  through  Columbine, 
the  Columbia  Space  Shuttle  disaster, 
and  September  1 1 .  Stillman  and 
Lancaster  describe  this  group  as 
realistic,  confident,  and  pragmatic. 
Raised  by  optimistic  Boomers, 
Millennial  feel  empowered  to  take 
positive  action  when  things  go  wrong. 

They  have  an  optimistic  outlook. 

They  are  self-assured  and  achievement 
focused. 

They  believe  in  strong  morals  and  serving 
the  community. 

They  are  aware  of  diversity. 


Source:  Adapted  from  Lancaster,  L.C.  and  Stillman,  D.,  When  Generations  Collide.  Who  They  Are.  Why  They  Clash.  How  to  Solve  the  Generational 
Puzzle  at  Work ,  Collins  Business,  New  York,  2002  (excerpted  from  D’Addono,  2004). 


primarily  drawn  from  interviews  and  focus  groups 
involving  young  people,  conducted  at  times  in  the 
presence  of  their  parents.  Furthermore,  they  are  crit¬ 
icized  that  their  convenience  samples  have  largely 
been  drawn  from  wealthy  suburbs  in  Northern  Vir¬ 
ginia  near  Washington,  D.C.,  where  the  authors  them¬ 
selves  live.  Critics  maintain  that  the  research  con¬ 
ducted  by  Howe  and  Strauss  does  not  reflect  regional 
differences  sufficiently,  much  less  issues  related  to 
socioeconomic  status,  race,  culture,  and  other  impor¬ 
tant  factors. 

Regardless  of  whether  the  survey  samples  are  rep¬ 
resentative  of  the  general  population  of  the  United 


States,  it  is  important  to  remember  that  the  data  from 
these  surveys  are  all  based  on  self-reports  rather  than 
behavioral  observations  or  other  measures.  In  addition, 
the  vast  majority  of  the  cases  presented  in  the  popular 
books  about  this  topic  are  anecdotal  in  nature,  fre¬ 
quently  reflecting  stereotypes  reported  in  the  news  or 
represented  in  popular  media.  For  example,  television 
programs  such  as  NBC’s  Dateline ,  CBS’s  60  Minutes, 
and  ABC’s  20-20  regularly  feature  stories  about  gen¬ 
erational  differences  such  as  the  impact  of  the  Baby 
Boomers’  pending  retirement  on  younger  generations 
or  the  phenomenon  of  “helicopter  parents”  who  hover 
over  their  Millennial  children. 
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Speaking  about  what  would  become  known  as  the 
Baby  Boomer  Generation  in  1 959 ,  Clark  Kerr,  President 
of  the  University  of  California  at  Berkeley  said  that 
“employers  are  going  to  love  this  generation.  ...They 
are  going  to  be  easy  to  handle.  There  aren’t  going  to  be 
any  riots”  (quoted  in  Manchester,  1974).  Kerr’s  naive 
optimism  about  the  Baby  Boomers  illustrates  the  danger 
of  predicting  the  behavior  of  future  generations.  Many 
members  of  the  Baby  Boomer  Generation  protested, 
and  even  occasionally  rioted,  about  civil  rights,  the  Viet¬ 
nam  War,  feminism,  and  the  environment  (Gillon, 
2004).  Arguably,  the  optimistic  predictions  made  about 
the  Millennials  by  Howe  and  Strauss  (2000)  and  others 
may  be  just  as  mistaken  as  Kerr’s  errant  prophecy. 

Clearly,  more  rigorous  research  is  needed  to  test 
whether  the  generational  differences  described  by 
Howe  and  Strauss  (2000),  Lancaster  and  Stillman 
(2002),  and  others  hold  up  when  specifically  looking 
at  people  living  in  the  lower  middle  and  lower  socio¬ 
economic  groups,  especially  with  respect  to  the  more 
than  35  million  American  living  below  the  poverty 
line.  Race  and  gender  are  other  individual  differences 
that  most  of  the  researchers  and  self-professed  author¬ 
ities  represented  in  the  generational  literature  address 
inadequately.  In  addition,  research  about  generational 
differences  in  other  cultures,  if  it  has  been  done  at  all, 
cannot  be  found  in  English  language  publications. 

The  best  contemporary  research  on  this  topic 
comes  from  Twenge  (2006),  who  devotes  considerable 
attention  to  race  and  gender,  although  unfortunately 
she  also  largely  ignores  socioeconomic  status.  With 
respect  to  race,  her  analysis  of  numerous  studies  con¬ 
ducted  between  the  1950s  and  today  led  her  to  the 
conclusion  that  GenMe  “will  continue  the  shift  toward 
equality  across  races”  [and]  "that  race  will  become  less 
important  as  a  defining  characteristic”  (p.  214).  With 
respect  to  gender,  she  predicted  that  women  will  soon 
be  the  majority  of  young  professionals  such  as  doctors 
and  lawyers  whereas  engineering  and  physics  “will 
remain  majority  male”  (p.  215). 

Generational  differences  are  weak  as  a  research- 
able  variable  in  a  manner  similar  to  learning  styles.  A 
recent  review  of  the  learning  styles  literature  con¬ 
ducted  in  England  (Coffield  et  al.,  2004)  throws  grave 
doubt  on  the  validity  and  utility  of  employing  learning 
styles  as  a  basis  for  accommodating  students  of  any 
generation.  The  reviewers  found  little  rigorous  scien¬ 
tific  support  for  the  very  existence  of  the  more  than 
70  learning  styles  models  reported  in  hundreds  of  pub¬ 
lished  studies  from  the  educational  and  psychological 
research  literature.  According  to  Coffield  et  al.  (2004), 
even  the  most  widely  studied  models  have  not  held  up 
to  scrutiny.  The  same  can  be  said  of  the  vast  majority 
of  scholarship  focused  on  generational  differences. 


CONCLUDING  REMARKS 

The  bottom  line  on  generational  differences  is  that 
educational  technology  researchers  should  treat  this 
variable  as  failing  to  meet  the  rigor  of  definition  and 
measurement  required  for  robust  individual  differ¬ 
ences  variables.  The  gross  generalizations  based  on 
weak  survey  research  and  the  speculations  of  profit- 
oriented  consultants  should  be  treated  with  extreme 
caution  in  a  research  and  development  context. 
Twenge’s  (2006)  work  appears  to  provide  the  most 
solid  foundation  for  distinguishing  between  two 
major  generations:  the  large  Baby  Boomer  Generation 
that  was  born  before  1970  and  Generation  Me  that 
was  bom  after  1970. 

That  said,  there  may  be  merit  in  examining  the 
preferences  of  today  generation  of  college  students  and 
workers  for  instructional  designs  that  utilize  video 
games,  instant  messaging,  podcasts,  and  other  cutting- 
edge  technologies  in  higher  education  and  the  work¬ 
place;  for  example,  there  is  growing  evidence  that 
video  games  do  more  good  than  bad  with  respect  to 
cognitive  outcomes  (Gee,  2003),  a  finding  that  some 
still  find  heretical.  Today’s  younger  university  students 
and  workers  have  spent  large  amounts  of  their  free 
time  playing  video  games,  and  survey  results  indicate 
that  today’s  students  and  workers  alike  prefer  to  be 
engaged  in  learning  via  games  than  by  traditional 
classroom  instruction.  Nonetheless,  similar  to  genera¬ 
tional  differences,  this  is  an  area  in  need  of  more 
substantive  research  for  it  to  be  assumed.  It  is  recom¬ 
mended  that  design-based  research  approaches  would 
be  most  amenable  to  future  inquiry  in  this  area  because 
such  approaches  seek  to  solve  real-world  problems 
related  to  teaching,  learning,  and  performance  support 
as  well  as  to  identifying  reusable  design  principles  (van 
den  Akker  et  al.,  2006).  Both  are  sorely  needed  at  the 
intersection  of  generational  differences  and  educa¬ 
tional  technology. 
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ABSTRACT 

People  prefer  to  learn  in  many  different  ways.  Further¬ 
more,  individuals  have  different  cognitive  abilities  that 
influence  the  way  effective  learning  takes  place.  Incor¬ 
porating  individual  differences  such  as  learning  styles 
and  cognitive  abilities  into  education  makes  learning 
easier  and  increases  the  learner’s  performance.  In  con¬ 
trast,  learners  whose  needs  are  not  supported  by  the 
learning  environment  experience  problems  in  the 
learning  process.  In  this  chapter,  we  introduce  some 
cognitive  traits  that  are  important  for  learning  and  also 
discuss  how  to  incorporate  different  abilities  in  educa¬ 
tional  systems.  In  regard  to  learning  styles,  some  major 
approaches  as  well  as  possible  strategies  for  involving 
learning  styles  in  online  courses  are  presented.  In  the 
next  section,  recent  research  dealing  with  identifying 
learning  styles  and  cognitive  traits  based  on  the  behav¬ 
ior  of  students  during  a  course  is  presented.  This  infor¬ 
mation  is  necessary  to  provide  adaptive  courses. 
Finally,  the  relationship  between  cognitive  traits  and 
learning  styles  is  discussed.  This  relationship  leads  to 
additional  information  and  therefore  to  a  more  reliable 
student  model. 


KEYWORDS 

Cognitive  abilities:  Abilities  to  perform  any  of  the 
functions  involved  in  cognition  whereby  cognition 
can  be  defined  as  the  mental  process  of  knowing, 
including  aspects  such  as  awareness,  perception, 
reasoning,  and  judgment. 

Learning  styles:  There  is  no  single  agreed  definition 
of  learning  styles;  a  general  definition  is  provided 
by  Honey  and  Mumford  (1992),  who  said  that  a 
learning  style  is  a  description  of  the  attitudes  and 
behaviors  that  determine  an  individual’s  preferred 
way  of  learning. 

Student  modeling:  Student  models  store  information 
about  students,  including  domain  competence  and 
individual  domain-independent  characteristics. 
Student  modeling  is  the  process  of  building  and 
updating  the  student  model. 


INTRODUCTION 

Individual  learners  play  a  central  role  in  a  technology- 
enhanced  learning  environment.  Each  learner  has  indi¬ 
vidual  characteristics  such  as  different  cognitive  abil¬ 
ities,  learning  style  preferences,  prior  knowledge, 
motivation,  and  so  on.  These  individual  differences 
affect  the  learning  process  and  are  the  reason  why 


some  students  find  it  easy  to  learn  in  a  particular  course 
whereas  others  find  the  same  course  difficult  (Jonassen 
and  Grabowski,  1993). 

The  context  in  which  learning  takes  place  also 
plays  an  important  role.  This  learning  context  includes 
learning  objectives,  learning  activities,  learning  assess¬ 
ments,  the  technology  or  tools  used,  information 
resources,  and  teachers,  tutors,  or  assistants.  Learners 
with  their  individual  differences  as  well  as  the  men¬ 
tioned  aspects  of  the  learning  context  can  be  seen  as 
components  of  a  learning  system.  Each  of  these  com¬ 
ponents  and  especially  the  interactions  among  these 
components  influence  the  learning  process. 

Gagne  (1985),  for  example,  argued  that  an  inter¬ 
action  between  the  learning  objectives  and  the  learning 
activities  exists  and  that  different  conditions  on  the 
structure  and  kind  of  learning  activities  are  necessary 
for  different  types  of  learning  objectives.  He  identified 
five  categories  of  learning:  verbal  information,  intel¬ 
lectual  skills,  cognitive  strategies,  motor  skills,  and 
attitudes.  For  learning  attitudes,  persuasive  arguments 
or  a  kind  of  role  model  are  necessary.  In  contrast,  to 
learn  motor  skills  an  important  condition  of  learning 
is  to  practice  these  skills.  On  the  other  hand,  when 
learning  verbal  information,  such  as  facts,  no  practices, 
arguments,  or  role  models  are  necessary. 

Another  example  is  the  interaction  between  infor¬ 
mation  resources  and  individual  differences  in  learning 
styles.  The  information  resources  can  be  presented  in 
different  forms,  such  as  text,  images,  animations,  sim¬ 
ulations,  graphs,  and  so  on,  and  therefore  match  or 
mismatch  each  learner’s  preferred  way  of  receiving 
information.  The  better  these  aspects  match,  the  better 
the  learning  that  can  take  place.  Furthermore,  the  infor¬ 
mation  might  be  comprised  of  concrete  material  such 
as  facts  and  data  or  the  information  might  deal  with 
more  abstract  material  such  as  concepts  and  theories. 
Again,  matching  or  mismatching  influences  the  learn¬ 
ing  process. 

Many  other  links  among  the  above-mentioned 
components  are  investigated  in  this  chapter,  and  influ¬ 
ences  on  learning  are  elaborated.  We  focus  on  research 
dealing  with  the  links  between  aspects  of  individual 
learners  — in  particular,  cognitive  abilities  and  learning 
styles,  how  instruction  can  be  designed  to  match  or 
mismatch  with  these  needs,  and  how  these  instructions 
can  be  supported  by  technology. 

Concerning  individual  differences,  much  research 
has  been  done  on  prior  knowledge  and  its  influence  on 
learning.  Jonassen  and  Grabowski  (1993)  suggested 
that  prior  knowledge  is  one  of  the  strongest  and  most 
consistent  individual  difference  predictors  of  achieve¬ 
ment.  Although  prior  knowledge  seems  to  account  for 
more  variance  in  learning  than  other  individual 
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differences,  more  recently  educational  researchers 
have  focused  on  aspects  of  cognitive  abilities  and 
learning  styles,  their  influence  on  learning,  and  how 
they  can  be  incorporated  in  technology  enhanced 
learning. 

Cognitive  abilities  and  learning  styles  play  an 
important  role  in  education;  for  example,  cognitive 
overload  may  hinder  the  process  of  learning  and  yield 
to  poor  performance.  With  regard  to  learning  styles, 
Felder  (Felder  and  Silverman,  1988;  Felder  and  Solo- 
man,  1997)  pointed  out  that  learners  with  strong  pref¬ 
erence  for  a  specific  learning  style  may  have  difficul¬ 
ties  in  learning  if  the  teaching  style  does  not  match 
their  learning  style.  From  a  theoretical  point  of  view, 
we  can  therefore  argue  that  incorporating  the  cognitive 
abilities  and  learning  style  of  students  makes  learning 
easier  and  increases  the  learning  efficiency  of  the  stu¬ 
dents.  On  the  other  hand,  learners  who  are  not  sup¬ 
ported  by  the  learning  environment  may  experience 
problems  in  the  learning  process. 

Although  these  hypotheses  seem  to  be  intuitive  and 
supported  by  educational  theories,  inconsistent  results 
have  been  obtained  by  studies  investigating  the  effects 
on  achievement  of  matched  and  mismatched  instruc¬ 
tion  for  learners  with  different  abilities  and  prefer¬ 
ences.  Jonassen  and  Grabowski  (1993,  p.  28)  noted 
that  several  reasons  for  such  inconsistent  results  are 
known  in  the  field  of  aptitude-treatment  interaction 
(ATI)  research.  Limitations  might  include  “small  sam¬ 
ples  size,  abbreviated  treatments,  specialized  aptitude 
constructs  or  standardized  tests,  and  a  lack  of  concep¬ 
tual  or  theoretical  linkage  between  aptitudes  and  the 
information-processing  requirements  of  the  treat¬ 
ment.” 

An  example  of  a  supporting  study  is  the  study 
performed  by  Bajraktarevic  et  al.  (2003)  showing  that 
students  attending  an  online  course  that  matched  their 
preferred  learning  style  (either  sequential  or  global) 
achieved  significantly  better  results  than  those  who 
completed  a  mismatched  course.  Another  supporting 
example  is  the  study  by  Ford  and  Chen  (2001)  where 
they  investigated  the  performance  of  students  attend¬ 
ing  courses  that  either  matched  or  mismatched  with 
their  cognitive  styles  (field  dependency  or  field  inde¬ 
pendency).  Also,  in  this  case,  students  who  undertook 
the  matched  courses  achieved  significantly  better 
results  than  those  who  attended  the  mismatched 
courses.  In  contrast,  a  study  by  Brown  et  al.  (2006) 
focused  on  the  visual  and  verbal  preference  of  learners. 
They  concluded  that,  “It  did  not  seem  to  matter 
whether  a  student  was  a  visual  or  bimodal  learner,  nor 
if  they  were  presented  with  visual,  verbal  or  mixed 
representations  of  data”  (Brown  et  al.,  2006,  p.  333). 
Another  example  of  a  study  that  did  not  yield  signifi¬ 


cant  results,  described  in  Tillema  (1982),  dealt  with 
the  serial  and  holistic  cognitive  styles.  These  inconsis¬ 
tent  results  show  that  more  future  work  is  necessary. 

Much  recent  research,  however,  has  addressed 
aspects  of  incorporating  cognitive  abilities  and  learn¬ 
ing  styles  in  technology-enhanced  learning  systems. 
This  chapter  aims  at  providing  an  overview  of  these 
aspects.  First,  an  introduction  into  cognitive  traits  and 
learning  styles  is  provided,  taking  into  account  instruc¬ 
tional  strategies  to  support  specific  cognitive  traits  and 
learning  styles  of  students  in  educational  systems.  The 
next  section  then  discusses  and  gives  examples  of  how 
cognitive  traits  and  learning  styles  can  be  identified. 
Finally,  the  relationship  between  cognitive  traits  and 
learning  style  is  discussed. 


COGNITIVE  TRAITS 

Humans  typically  have  a  number  of  cognitive  abilities. 
In  this  section,  we  focus  on  cognitive  abilities  that  are 
important  for  learning.  For  these  abilities,  we  discuss 
how  instruction  is  related  with  these  cognitive  abilities 
and  how  to  support  learners  with  low  and  high  cogni¬ 
tive  abilities  in  educational  systems. 

Working  Memory  Capacity 

Working  memory  allows  us  to  keep  active  a  limited 
amount  of  information  (about  five  to  nine  items)  for  a 
brief  period  of  time  (Miller,  1956).  In  earlier  times, 
working  memory  was  also  referred  to  as  short-term 
memory.  Despite  varying  views  on  the  structure  of  the 
working  memory,  researchers  now  agree  that  it  con¬ 
sists  of  both  storage  and  operational  subsystems  (Rich¬ 
ards- Ward,  1996).  Deficiencies  in  working  memory 
capacity  result  in  different  performances  on  a  variety 
of  tasks.  Examples  of  affected  tasks  include  natural 
language  use  (e.g.,  comprehension,  production),  rec¬ 
ognition  of  declarative  memory,  skill  acquisition,  and 
so  on  (Byrne,  1996). 

The  dual-code  hypothesis  is  based  on  the  assump¬ 
tion  that  the  working  memory  consists  of  two  separate 
components,  one  concerned  with  verbal  materials  and 
one  concerned  with  nonverbal  materials  (Clark  and 
Paivio,  1991).  According  to  this  hypothesis,  cognitive 
load  is  reduced  when  both  channels  (verbal  and  non¬ 
verbal)  are  attracted,  thus  better  learning  can  take 
place.  A  supporting  example  is  the  study  conducted  by 
Moreno  and  Valdez  (2005)  where  students  were  pre¬ 
sented  with  diagrams  (nonverbal  information)  and  an 
explaining  text  (verbal  information),  with  only  dia¬ 
grams,  or  with  only  the  explaining  text.  Students  who 
received  both  types  of  information  achieved  the  best 
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results  in  tests  of  retention  as  well  as  the  transfer  of 
knowledge.  Another  supporting  study  of  the  dual-code 
hypothesis  was  performed  by  Wey  and  Waugh  (1993). 
They  found  that  field-dependent  learners,  who  tend  to 
have  low  working  memory  capacity  (A1  Naeme,  1991; 
Bahar  and  Hansell,  2000),  experienced  difficulties  in 
learning  text-only  material  and  benefited  from  material 
containing  both  text  and  graphics. 

In  some  conditions,  dual-code  presentation  has 
positive  effects.  According  to  Mayer  (1997)  and  Kal- 
yuga  et  al .  ( 1999) ,  information  should  not  be  redundant 
and  should  be  integrated  so  students  are  not  forced  to 
split  their  attention.  Presenting  a  text  in  written  format 
as  well  as  in  audio  format  imposes  an  additional  cog¬ 
nitive  load  and  has  a  negative  effect  on  learning.  Fur¬ 
thermore,  incorporating  the  domain  experience  of  the 
learners  seems  to  be  important.  Kalyuga  et  al.  (2000) 
found  in  their  study  that  the  effectiveness  of  dual-code 
presentation  of  information  decreases  with  the  increas¬ 
ing  learner  experience.  Although  novice  learners  can 
achieve  better  results  when  learning  from  diagrams 
with  audio  text  than  with  diagrams  only,  more  experi¬ 
enced  learners  achieved  better  results  from  the  dia- 
grams-only  situation  due  to  the  reduced  cognitive  load 
imposed  by  the  diagram-only  presentation. 

Based  on  the  exploration  space  control  elements 
(Kashihara  et  al.,  2000),  different  versions  of  courses 
can  be  created  that  suit  different  needs.  These  elements 
include  the  number  and  relevance  of  paths;  the  amount, 
concreteness,  and  structure  of  the  content;  and  the 
number  of  information  resources.  The  instructional 
design  in  learning  systems  should  assist  learners  by 
considering  their  abilities  and  avoiding  cognitive  over¬ 
load.  For  learners  with  low  working  memory  capacity, 
this  can  be  achieved  by  decreasing  the  number  and 
increasing  the  relevance  of  paths  in  a  course.  Further¬ 
more,  less  but  more  concrete  content  should  be  pre¬ 
sented.  Moreover,  the  number  of  available  media 
resources  should  increase.  In  contrast,  for  learners  with 
high  working  memory  capacity,  fewer  relevant  paths 
can  be  presented,  and  the  amount  of  content  and  its 
abstractness  can  be  increased  (Kinshuk  and  Lin,  2003). 

Reasoning  Ability 

With  respect  to  reasoning  abilities,  we  can  distinguish 
among  inductive,  deductive,  and  abductive  reasoning. 
In  the  following  discussion,  we  focus  on  inductive  rea¬ 
soning,  as  this  ability  is  the  most  important  one  with 
regard  to  learning.  We  also  address  deductive  reasoning. 

Inductive  reasoning  skills  relate  to  the  ability  to 
construct  concepts  from  examples.  Students  who  face 
complicated  problems  look  for  known  patterns  and 
use  them  to  construct  temporary  internal  hypotheses 


or  schema  in  which  to  work  (Bower  and  Hilgard, 
1981).  It  is  easier  for  students  who  possess  better 
inductive  reasoning  skill  to  recognize  an  already 
known  pattern  and  generalize  higher  order  rules.  As 
a  result,  the  load  on  working  memory  is  reduced,  and 
the  learning  process  is  more  efficient.  In  other  words, 
the  higher  the  inductive  reasoning  ability,  the  easier 
it  is  to  build  up  the  mental  model  of  the  information 
learned.  According  to  Harverty  et  al.  (2000) ,  inductive 
reasoning  ability  is  the  best  predictor  for  academic 
performance. 

For  simulation-based  discovery  learning,  students 
are  asked  to  infer  characteristics  of  a  model  through 
experimentations  by  using  a  computer  simulation,  thus 
they  are  being  asked  to  use  their  inductive  reasoning 
skills.  According  to  Veermans  and  van  Joolingen 
(1998),  simulation-based  discovery  learning  results  in 
deeper  rooting  of  the  knowledge,  enhanced  transfer, 
the  acquisition  of  regulatory  skills,  and  thus  better 
motivation;  however,  discovery  learning  does  not 
always  lead  to  better  learning  results.  One  reason  is 
that  students  find  it  difficult  to  perform  the  required 
processes.  To  improve  the  learning  progress  and  sup¬ 
port  learners  with  low  inductive  reasoning  abilities, 
Veermans  and  van  Joolingen  designed  a  mechanism 
that  provides  advice  based  on  the  experiments  per¬ 
formed  in  the  simulation.  This  mechanism  is  integrated 
in  SimQuest,  an  authoring  system  for  simulation-based 
discovery  (van  Joolingen  and  de  Jong,  2003). 

Considering  again  exploratory  learning  and  the 
exploration  space  control  elements,  many  opportuni¬ 
ties  for  observation  should  be  provided  to  learners  with 
low  inductive  reasoning  skills.  Learning  systems  can 
support  these  learners  by  providing  a  great  amount  of 
well-structured  and  concrete  information  with  many 
paths.  For  learners  with  high  inductive  reasoning  skills, 
the  amount  of  information  and  number  of  paths  should 
decrease  to  reduce  the  complexity  of  the  hyperspace 
and  enable  the  learners  to  grasp  the  concepts  more 
quickly.  Moreover,  information  can  be  presented  in  a 
more  abstract  way  (Kinshuk  and  Lin,  2003). 

Deduction  is  defined  as  drawing  logical  conse¬ 
quences  from  premises.  An  application  for  deductive 
reasoning  is,  for  example ,  naturalistic  decision  making 
(Zsambok  and  Klein,  1997),  or  what  people  do  in  real- 
world  situations.  It  has  been  observed  that  experienced 
decision  makers  recognize  appropriate  actions  to  take 
in  various  situations  that  might  arise,  whereas  inexpe¬ 
rienced  decision  makers  perform  an  unorganized  and 
almost  random  search  of  alternatives.  When  it  comes 
to  complex  problems,  humans  often  fail  to  find  appro¬ 
priate  solutions.  According  to  Ddmer  (1996),  several 
reasons  exist  for  such  failures;  for  example,  humans 
tend  to  oversimplify  the  mental  model  of  the  complex 
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system,  tend  to  be  slow  in  thinking  when  it  comes  to 
conscious  thoughts,  or  tend  to  ignore  the  possibility  of 
side  effects.  Dorner’s  experiments,  however,  showed 
that  leaders  from  business  and  industry  tend  to  make 
more  effective  decisions  in  complex  situations,  and  he 
argued  that  the  necessary  behavior  and  skills  can  be 
acquired  and  learned. 

Information  Processing  Speed 

Information  processing  speed  determines  how  quickly 
learners  can  acquire  information  correctly.  Instruc¬ 
tional  designers  should  take  into  account  a  learner’s 
information  processing  speed;  for  example,  some 
learners  have  such  slow  reading  speeds  that  they  are 
unable  to  hold  enough  details  in  working  memory  to 
permit  decoding  of  the  overall  meaning  (Bell,  2001). 
Based  on  the  exploration  space  control  elements, 
learners  with  low  information  processing  speed  should 
be  presented  with  only  the  important  points.  The  num¬ 
ber  of  paths  and  amount  of  information  should 
decrease,  and  the  relevance  of  the  paths  should 
increase.  Additionally,  the  structure  of  the  information 
should  increase  to  speed  up  the  learning  process.  In 
contrast,  for  learners  with  high  information  processing 
speed,  the  information  space  can  be  enlarged  by  pro¬ 
viding  a  greater  amount  of  information  and  number  of 
paths  (Kinshuk  and  Lin,  2003). 

Associative  Learning  Skills 

The  associative  learning  skills  link  new  knowledge  to 
existing  knowledge.  The  association  process  requires 
pattern-matching  to  discover  the  space  of  existing 
information,  analysis  of  the  relationships  between  the 
existing  and  new  knowledge,  and  retention  of  the  new 
knowledge  in  long-term  memory  (or,  more  specifi¬ 
cally,  maintaining  links  to  the  new  knowledge).  To 
assist  the  association  processes  during  a  student’s 
learning,  the  instruction  must  assist  the  recall  (revisit) 
of  learned  information,  clearly  show  the  relationships 
of  concepts  (new  to  existing),  and  facilitate  new  or 
creative  association  or  insight  formation  by  providing 
information  of  the  related  domain  area.  A  great  amount 
of  information,  various  media  resources,  and  many 
relevant  paths  help  a  learner  with  low  associative  learn¬ 
ing  skills  to  associate  one  concept  with  another.  Fur¬ 
thermore,  well-structured  information  makes  linkage 
between  concepts  easier.  In  contrast,  for  learners  with 
high  associative  learning  skills,  providing  less  struc¬ 
tured  information  allows  them  to  navigate  more  freely 
and  enhances  learning  speed.  Additionally,  the  rele¬ 
vance  of  the  paths  should  decrease  to  enlarge  the  infor¬ 
mation  space  (Kinshuk  and  Lin,  2003). 


Metacognition 

The  concept  of  metacognition  was  introduced  by  John 
Flavell  (1976).  Several  definitions  for  metacognition 
exist;  for  example,  according  to  Flavell  (1976,  p.  232), 
“Metacognition  refers  to  one’s  knowledge  concerning 
one’s  own  cognitive  processes  and  products  or  any¬ 
thing  related  to  them.”  Panaoura  and  Philippou  (2005) 
defined  metacognition  as  the  awareness  and  monitor¬ 
ing  of  one’s  own  cognitive  system  and  its  functioning. 
Metacognition  consists  of  several  dimensions;  how¬ 
ever,  self-representation  (the  knowledge  about  cogni¬ 
tion)  and  self-regulation  (the  regulation  of  cognition) 
are  the  main  ones.  Recent  research  suggests  that  meta¬ 
cognition  plays  an  important  role  in  the  learning  pro¬ 
cess  (Alexander  et  al„  2003;  Mayer,  1998;  Panaoura 
and  Philippou,  2004);  for  example,  it  is  known  that 
learners  with  high  metacognitive  abilities  perform  bet¬ 
ter  in  problem-solving  tasks  (Lester  et  al.,  1989; 
Mayer,  1998). 


LEARNING  STYLES 

Several  different  learning  style  models  exist  in  the 
literature,  each  proposing  different  descriptions  and 
classifications  of  learning  types.  To  date,  no  single 
definition  of  learning  style  has  been  identified. 
Honey  and  Mumford  (1992,  p.  1)  defined  learning 
styles  as  “a  description  of  the  attitudes  and  behaviors 
which  determine  an  individual’s  preferred  way  of 
learning.”  James  and  Gardner  (1995,  p.  20)  defined 
it  more  precisely  by  suggesting  that  learning  style  is 
the  “complex  manner  in  which,  and  conditions  under 
which,  learners  most  efficiently  and  most  effectively 
perceive,  process,  store,  and  recall  what  they  are 
attempting  to  learn.”  Furthermore,  researchers  do  not 
agree  on  whether  learning  styles  are  stable  over  time. 
In  some  studies,  learning  style  changed  quite  quickly 
(see,  for  example,  Clariana,  1997),  whereas  other 
researchers  found  learning  styles  to  be  stable  over  a 
long  period  of  time  (Felder  and  Spurlin,  2005;  Keefe, 
1979;  Kolb,  1981). 

In  this  section,  we  introduce  some  of  the  most 
common  classifications  of  learning  styles  — namely, 
the  Myers-Briggs  Type  Indicator  (Briggs-Myers, 
1962),  Kolb’s  learning  style  model  (Kolb,  1984), 
Honey  and  Mumford’s  learning  style  model  (Honey 
and  Mumford,  1982),  and  the  Felder-Silverman 
learning  style  model  (Felder  and  Silverman,  1988). 
Focusing  on  this  last  model,  we  also  discuss  possible 
teaching  strategies  that  can  be  used  to  support  learn¬ 
ers  with  different  learning  styles  in  educational  sys¬ 
tems. 
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Myers-Briggs  Type  Indicator 

The  Myers-Briggs  Type  Indicator  (MBTI)  (Briggs- 
Myers,  1962)  is  a  personality  test  and  is  not  focused 
specifically  on  learning;  nevertheless,  the  personality 
of  a  learner  influences  how  that  person  learns,  so  some 
learning  style  models  are  based  on  considerations  of 
MBTI.  Based  on  Jung’s  theory  of  psychological  types 
(Jung,  1923),  the  MBTI  distinguishes  a  person’s  type 
according  to  four  dichotomies:  extroversion/introver¬ 
sion,  sensing/intuitive,  thinking/feeling,  and  judg¬ 
ing/perceiving.  All  possible  combinations  can  occur, 
resulting  in  a  total  number  of  16  types. 

Extrovert  and  introvert  refer  to  how  a  person  ori¬ 
ents  and  receives  his  or  her  energy.  The  preferred  focus 
of  people  with  an  extrovert  attitude  is  on  their  sur¬ 
roundings,  such  as  other  people  and  things,  whereas 
introverts’  preferred  focus  is  on  their  own  thoughts  and 
ideas.  Sensing  and  intuition  deal  with  the  way  people 
prefer  to  perceive  data.  Whereas  sensing  people  prefer 
to  perceive  data  from  their  five  senses,  intuitive  people 
use  their  intuition  and  prefer  to  perceive  data  from  the 
unconscious.  The  judgment  based  on  the  perceived 
data  can  be  distinguished  between  thinking  and  feel¬ 
ing.  Thinking  means  that  the  judgment  is  based  on 
logical  connections,  such  as  true-false  or  if-then,  but 
feeling  refers  to  more-less  or  better-worse  evalua¬ 
tions;  however,  judgment  and  decisions  are  in  both 
cases  based  on  rational  considerations.  The  last  dichot¬ 
omy  describes  whether  a  person  is  more  extroverted 
in  his  or  her  stronger  judgment  function  (thinking  or 
feeling)  or  in  the  perceiving  function  (sensing  or  intu¬ 
ition).  Judging  people  prefer  step-by-step  approaches 
and  structure,  as  well  as  coming  to  a  quick  closure. 
Perceiving  people  have  a  preference  for  keeping  all 
options  open  and  tend  to  be  more  flexible  and  sponta¬ 
neous.  The  four  preferences  interact  with  each  other 
rather  then  being  independent,  and  for  a  complete 
description  of  a  person’s  type  the  combination  of  all 
four  preferences  must  be  considered. 

Kolb’s  Learning  Style  Model 

The  learning  style  theory  proposed  by  Kolb  (1984)  is 
based  on  the  experiential  learning  theory,  which  mod¬ 
els  the  learning  process  and  incorporates  the  important 
role  of  experience  in  this  process.  Following  this  the¬ 
ory,  learning  is  conceived  as  a  four-stage  cycle.  Con¬ 
crete  experience  is  the  basis  for  observations  and 
reflections.  These  observations  are  used  to  form 
abstract  concepts  and  generalizations,  which  again  act 
as  basis  for  testing  implementations  of  concepts  in  new 
situations.  Testing  implementations  results  in  concrete 
experience,  which  closes  the  learning  cycle.  According 


to  this  theory,  learners  need  four  abilities  for  effective 
learning:  (1)  concrete  experience  abilities,  (2)  reflec¬ 
tive  observation  abilities,  (3)  abstract  conceptualiza¬ 
tion  abilities,  and  (4)  active  experimentation  abilities. 
On  closer  examination,  there  are  two  polar  opposite 
dimensions:  concrete/abstract  and  active/reflective. 
Kolb  (1981)  noted  that,  “as  a  result  of  our  hereditary 
equipment,  our  particular  past  life  experience,  and  the 
demands  of  our  present  environment,  most  of  us 
develop  learning  styles  that  emphasize  some  learning 
abilities  over  others.”  Based  on  this  assumption,  Kolb 
identified  four  statistically  prevalent  types  of  learning 
styles. 

Convergers'  dominant  abilities  are  abstract  con¬ 
ceptualization  and  active  experimentation;  therefore, 
their  strengths  lie  in  the  practical  applications  of  ideas. 
The  name  “Convergers”  is  based  on  Hudson’s  theory 
of  thinking  styles  (Hudson,  1966),  where  convergent 
thinkers  are  people  who  are  good  at  gathering  infor¬ 
mation  and  facts  and  putting  them  together  to  find  a 
single  correct  answer  to  a  specific  problem.  In  contrast. 
Divergers  excel  at  the  opposite  poles  of  the  two  dimen¬ 
sions— namely,  concrete  experimentation  and  reflec¬ 
tive  observation.  They  are  good  in  viewing  concrete 
situations  in  many  different  perspectives  and  in  orga¬ 
nizing  relationships  to  a  meaningful  shape.  According 
to  Hudson,  a  dominant  strength  of  Di vergers  is  to 
generate  ideas;  therefore.  Divergers  tend  to  be  more 
creative.  Assimilators  excel  in  abstract  conceptualiza¬ 
tion  and  reflective  observation.  Their  greatest  strength 
lies  in  creating  theoretical  models.  They  are  good  in 
inductive  reasoning  and  in  assimilating  disparate 
observations  into  an  integrated  explanation.  Accommo- 
dators’  strengths  are  opposite  those  of  the  Assimila¬ 
tors.  Their  dominant  abilities  are  concrete  experience 
and  active  experimentation.  Their  strengths  lie  in  doing 
things  actively,  carrying  out  plans  and  experiments, 
and  becoming  involved  in  new  experiences.  They  are 
also  characterized  as  risk  takers  and  as  people  who 
excel  in  situations  that  call  for  adaptation  to  specific 
immediate  circumstances. 

Honey  and  Mumford’s  Learning  Style  Model 

The  learning  style  model  by  Honey  and  Mumford 
(1982)  is  based  on  Kolb’s  experiential  learning  theory 
and  is  developed  further  on  the  four  types  of  Kolb’s 
learning  style  model.  The  active/reflective  and  sensing/ 
intuitive  dimensions  are  strongly  involved  in  the 
defined  types  as  well.  Furthermore,  Honey  and  Mum- 
ford  stated  that  the  similarities  between  Kolb’s  model 
and  theirs  are  more  significant  than  the  differences 
(Honey  and  Mumford,  1992).  In  Honey  and  Mum- 
ford’s  learning  style  model,  the  types  are  referred  to 
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as  Activist  (similar  to  Accommodator),  Theorist  (sim¬ 
ilar  to  Assimilator) ,  Pragmatist  (similar  to  Converger) , 
and  Reflector  (similar  to  Diverger).  Activists  involve 
themselves  fully  in  new  experiences,  are  enthusiastic 
about  anything  new,  and  learn  best  by  doing  something 
actively.  Theorists  excel  in  adapting  and  integrating 
observations  into  theories;  they  need  models,  concepts, 
and  facts  to  engage  in  the  learning  process.  Pragmatists 
are  interested  in  real-world  applications  of  the  learned 
material;  they  like  to  try  out  various  ideas,  theories, 
and  techniques  to  see  if  they  work  in  practice.  Reflec¬ 
tors  are  people  who  like  to  observe  other  people  and 
their  experiences  from  many  different  perspectives  and 
reflect  about  them  thoroughly  before  coming  to  a  con¬ 
clusion;  also,  learning  occurs  for  these  people  by 
observing  and  analyzing  the  observed  experiences. 

Felder-Silverman  Learning  Style  Model 

Whereas  Honey  and  Mumford’s  and  Kolb’s  learning 
style  models  focus  on  a  few  statistically  prevalent 
types,  in  the  Felder-Silverman  learning  style  model 
(Felder  and  Silverman,  1988),  learners  are  character¬ 
ized  by  values  on  four  dimensions.  These  dimensions 
can  be  viewed  independently,  and  they  show  how 
learners  prefer  to  process  (active/reflective),  perceive 
(sensing/intuitive),  receive  (verbal/visual),  and  under¬ 
stand  (sequential/global)  information.  Because  the 
range  of  each  dimension  in  the  Felder-Silverman 
learning  style  model  extends  from  +11  to  -11,  a  bal¬ 
anced  preference  can  also  be  expressed.  These  values 
represent  tendencies,  suggesting  that  even  a  learner 
with  a  strong  positive  or  negative  value  sometimes  can 
act  differently. 

The  active/reflective  dimension  is  analogous  to  the 
respective  dimension  in  Kolb’s  model;  active  learners 
learn  best  by  working  actively  with  the  learning  mate¬ 
rial— for  example, by  working  in  groups, by  discussing 
the  material,  or  by  applying  it.  To  support  these  learn¬ 
ers  in  technology-enhanced  educational  systems ,  exer¬ 
cises,  interactive  animations,  and  group  work  tasks  can 
be  provided  to  allow  them  to  deal  with  the  subject 
actively.  In  contrast,  reflective  learners  prefer  to  think 
about  and  reflect  the  material,  and  they  need  enough 
time  for  doing  so.  Learning  systems  support  this  by 
allowing  learners  to  learn  in  their  own  pace. 

The  sensing! intuitive  dimension  is  taken  from  the 
Myers-Briggs  Type  Indicator  and  is  similar  to  the 
sensing/intuitive  dimension  in  Kolb’s  model.  Learners 
who  prefer  a  sensing  learning  style  like  to  learn  facts 
and  concrete  learning  material.  They  tend  to  be  more 
patient  with  details  and  also  more  practical  than  intu¬ 
itive  learners  and  like  to  relate  the  learned  material  to 
the  real  world.  Intuitive  learners  prefer  to  learn  abstract 


learning  material,  such  as  theories  and  their  underlying 
meanings.  They  like  to  discover  possibilities  and  rela¬ 
tionships  and  tend  to  be  more  innovative  and  creative 
than  sensing  learners;  therefore,  they  score  better  in 
open-ended  tests  than  in  tests  with  a  single  answer  to 
a  problem.  Whereas  intuitive  learners  are  good  at 
learning  abstract  concepts  and  theories,  sensing  learn¬ 
ers  require  a  higher  number  of  examples  and  various 
kinds  of  media  resources  that  address  their  senses,  such 
as  audio  or  video  objects,  to  support  their  learning 
process. 

The  verbal/visual  dimension  differentiates 
between  learners  who  remember  best  what  they  have 
seen  (e.g.,  pictures,  diagrams,  flow-charts)  and  learners 
who  get  more  out  of  words,  regardless  of  whether  they 
are  written  or  spoken.  Accordingly,  visual  learners  can 
be  assisted  by  including  visual  elements  such  as  pic¬ 
tures  or  diagrams  in  the  learning  material.  For  verbal 
learners,  communication  tools  such  as  forum  or  chat 
are  helpful. 

The  fourth  dimension,  sequential! global,  distin¬ 
guishes  between  sequential  and  global  ways  of  under¬ 
standing.  Sequential  learners  learn  in  small  incremen¬ 
tal  steps  and  have  a  linear  learning  progress.  They  tend 
to  follow  logical  stepwise  paths  to  find  solutions.  In 
contrast,  global  learners  use  a  holistic  thinking  process 
and  learn  in  large  leaps.  They  tend  to  absorb  learning 
material  almost  randomly  without  seeing  connections , 
but  after  they  have  learned  enough  material  they  sud¬ 
denly  get  the  whole  picture  .  They  are  then  able  to  solve 
complex  problems  and  put  things  together  in  novel 
ways;  however,  they  find  it  difficult  to  explain  how 
they  did  it.  For  sequential  learners,  it  is  important  to 
provide  a  well-structured  path  through  the  course 
material  and  not  overtax  them  by  providing  too  many 
links.  In  contrast,  global  learners  prefer  to  go  their  own 
way  through  the  course.  To  help  global  learners  gain 
an  understanding  of  the  whole  picture  of  the  course, 
overviews  should  be  presented. 

IDENTIFYING  COGNITIVE  TRAITS 
AND  LEARNING  STYLES 

To  incorporate  cognitive  traits  or  learning  styles  in 
educational  systems,  information  about  cognitive 
traits  and  learning  styles  must  first  be  collected.  One 
approach  is  to  let  students  perform  comprehensive 
tests  or  questionnaires  to  reveal  their  cognitive  traits 
or  learning  styles.  Such  an  approach,  however,  has  the 
potential  to  suffer  from  the  biases  and  indecisiveness 
of  the  learners.  A  more  meaningful  approach  is  to 
track  students’  behavior  and  infer  the  required  infor¬ 
mation  from  their  behavior.  The  cognitive  trait  model 
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(Kinshuk  and  Lin,  2004;  Lin  and  Kinshuk,  2005)  uses 
this  approach  to  profile  learners  according  to  their 
cognitive  traits.  To  identify  learning  styles,  approaches 
for  detecting  the  various  dimensions  of  the  Felder-Sil- 
verman  learning  style  model  are  used. 

Identification  of  Cognitive  Traits 

The  cognitive  trait  model  (CTM)  is  a  student  model 
that  profiles  learners  according  to  their  cognitive  traits. 
Four  cognitive  traits  — working  memory  capacity, 
inductive  reasoning  ability,  processing  speed,  and 
associative  learning  skills  — are  addressed  in  the  CTM. 
The  CTM  offers  the  role  of  learning  companion,  which 
can  be  consulted  by  and  interact  with  a  particular 
learner’s  learning  environments.  The  CTM  can  still  be 
valid  after  a  long  period  of  time  due  to  the  more  or 
less  persistent  nature  of  cognitive  traits  of  humans 
(Deary,  2004).  When  a  student  encounters  a  new  learn¬ 
ing  environment,  the  learning  environment  can  utilize 
the  CTM  of  the  particular  student  and  does  not  need 
to  reacquaint  itself  with  the  student.  Identification  of 
the  cognitive  traits  is  based  on  the  behaviors  of  learners 
in  the  system.  Various  patterns— manifests  of  traits 
(MOTs)— are  defined  for  each  cognitive  trait.  Each 
MOT  is  a  piece  of  an  interaction  pattern  manifesting 
a  learner’s  characteristics.  A  neural  network  (Lin  and 
Kinshuk,  2004)  is  responsible  for  calculating  the  cog¬ 
nitive  traits  of  the  learners  based  on  the  information 
of  the  MOTs. 

Identification  of  Learning  Styles 

A  number  of  adaptive  systems  are  available  in  the 
literature  that  incorporate  learning  styles;  for  example, 
CS383  (Carver  et  al.,  1999)  was  the  first  adaptive 
hypermedia  system  based  on  the  Felder-Silverman 
learning  style  model  (FSLSM) .  The  course  conducted 
in  the  system  included  the  comprehensive  collection 
of  media  objects.  The  system  offered  students  the 
option  to  order  these  objects  in  accordance  with  how 
well  they  fit  the  learning  styles  of  the  students.  Also 
based  on  FSLSM  is  MAS-PLANG  (Pena  et  al.,  2002), 
a  multi-agent  system  that  has  been  developed  to  enrich 
the  intelligent  tutoring  system  USD  (Fabregat  et  al., 
2000)  with  adaptivity  with  respect  to  learning  styles. 
Another  example  is  INSPIRE  (Papanikolaou  and 
Grigoriadou,  2003),  which  is  based  on  Honey  and 
Mumford’s  learning  style  theory.  In  all  of  these  sys¬ 
tems  and  in  most  other  systems  that  incorporate  learn¬ 
ing  styles,  the  learning  style  is  identified  based  on  a 
questionnaire  that  is  filled  out  by  learners  before  using 
the  system.  These  questionnaires  are  based  on  the 
assumption  that  learners  are  aware  of  how  they  learn. 


Jonassen  and  Grabowski  (1993,  p.  234)  pointed  out 
that,  “because  learning  styles  are  based  on  self- 
reported  measures,  rather  than  ability  tests,  validity  is 
one  of  their  most  significant  problems.” 

Garcia  et  al.  (2007)  studied  the  use  of  Bayesian 
networks  (Jensen,  1996)  to  detect  students’  learning 
styles  based  on  their  behavior  in  the  educational  sys¬ 
tem  SAVER.  Based  on  the  Felder-Silverman  learning 
style  model,  they  determined  patterns  of  behavior  rep¬ 
resentative  of  the  respective  dimensions,  as  well  as  the 
different  states  that  these  variables  or  patterns  can  take. 
Because  SAVER  does  not  incorporate  the  visual/verbal 
dimension,  this  dimension  was  not  included  in  their 
investigations. 

The  above  approaches  were  developed  for  specific 
systems,  but  Graf  and  Kinshuk  (2006)  proposed  an 
approach  to  detecting  learning  styles  in  learning  man¬ 
agement  systems  in  general.  Similar  to  the  approach  of 
Garcia  et  al.  (2006),  the  Felder-Silverman  learning 
style  model  is  used  as  the  basis,  but  in  this  case  all  four 
dimensions  are  incorporated.  The  patterns  of  behavior 
are  derived  from  commonly  used  features  in  learning 
management  systems,  such  as  forums  and  exercises. 
The  learning  styles  are  determined  by  applying  the 
index  of  learning  styles  (Felder  and  Soloman,  1997),  a 
questionnaire  used  to  identify  learning  styles  according 
to  the  Felder-Silverman  learning  style  model. 

THE  RELATIONSHIP  BETWEEN 
LEARNING  STYLES  AND 
COGNITIVE  TRAITS 

So  far,  cognitive  traits  and  learning  styles  have  been 
discussed  separately.  Consideration  of  their  relation¬ 
ship  with  each  other  makes  it  possible  to  obtain  addi¬ 
tional  information  about  a  learner.  In  educational  sys¬ 
tems  that  consider  either  only  learning  styles  or  only 
cognitive  traits,  the  relationship  leads  to  more  informa¬ 
tion.  This  additional  information  can  be  used  to  provide 
more  adaptivity —namely,  with  regard  to  learning  styles 
and  cognitive  traits.  In  systems  that  incorporate  learn¬ 
ing  styles  as  well  as  cognitive  traits,  the  interaction  can 
be  used  to  improve  the  detection  process  of  the  coun¬ 
terpart.  This  leads  to  a  more  reliable  student  model. 

Graf  et  al.  (2005)  investigated  the  relationship 
between  the  Felder-Silverman  learning  style  model  and 
the  cognitive  trait  of  working  memory  capacity.  Based 
on  the  literature,  a  relationship  between  high  working 
memory  capacity  and  a  reflective,  intuitive,  and 
sequential  learning  style  can  be  identified.  Learners 
with  low  working  memory  capacity  tend  to  prefer  an 
active,  sensing,  and  global  learning  style.  With  regard 
to  the  visual/verbal  dimension,  it  can  be  concluded  that 
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learners  with  low  working  memory  capacity  tend  to 
prefer  a  visual  learning  style,  but  learners  with  a  visual 
learning  style  do  not  necessarily  have  low  working 
memory  capacity.  To  verify  the  proposed  relationship, 
an  exploratory  study  with  39  students  was  conducted 
(Graf  et  al.,  2006).  The  results  supported  the  existence 
of  a  relationship  between  working  memory  capacity 
and  two  of  the  four  dimensions  of  the  learning  style 
model— the  sensing/intuitive  and  the  verbal/visual 
dimensions.  For  the  two  remaining  dimensions,  only 
tendencies  but  no  significant  correlations  were  found; 
therefore,  a  further  study  with  a  larger  sample  size  is 
planned. 


CONCLUSION 

Incorporating  cognitive  traits  and  learning  styles  in 
technology-enhanced  educational  systems  supports 
learners  and  makes  learning  easier  for  them;  neverthe¬ 
less,  only  a  few  systems  have  considered  these  needs. 
Although  at  least  some  attention  is  paid  to  cognitive 
traits  and  learning  styles  in  adaptive  systems,  learning 
management  systems  that  are  commonly  used  in  e- 
education  today  do  not  incorporate  such  personal 
needs.  This  chapter  has  focused  on  three  issues:  (1) 
introducing  cognitive  traits  that  are  important  for 
learning  and  the  major  learning  style  theories,  as  well 
as  strategies  that  incorporate  both  in  educational  sys¬ 
tems;  (2)  approaches  to  identifying  cognitive  traits  and 
learning  style  as  a  requirement  to  adapting  to  them; 
and  (3)  how  the  relationships  between  cognitive  traits 
and  learning  styles  can  be  used  to  improve  student 
modeling.  Linking  learning  styles  and  cognitive  traits 
with  instruction  and  incorporating  them  in  educational 
systems  is  an  important  and  beneficial  issue  for  stu¬ 
dents.  Obviously,  more  research  on  learning  styles  and 
especially  on  cognitive  traits  is  necessary  to  further 
establish  their  importance  in  e-education. 
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ABSTRACT 

A  variety  of  technology-based  interventions  have 
been  proposed  to  accelerate  learning.  Although  these 
technologies  have  enjoyed  some  success,  we  need  a 
better  understanding  of  the  manner  in  which  specific 
technologies  affect  the  pedagogical  factors  associ¬ 


ated  with  enhanced  learning.  In  this  chapter,  we 
attempt  to  articulate  a  path  forward  for  the  develop¬ 
ment  of  synthetic  learning  environments  by  identi¬ 
fying  variables  that  are  important  to  their  effective¬ 
ness.  Further,  we  attempt  to  articulate  the  conditions 
under  which  such  technologies  are  most  likely  to  be 
successful. 
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KEYWORDS 

Simulation:  A  working  representation  of  reality;  used 
in  training  to  represent  devices  and  process  and 
may  be  low  or  high  in  terms  of  physical  or  func¬ 
tional  fidelity. 

Synthetic  learning  environments:  A  learning  environ¬ 
ment  characterized  in  terms  of  a  particular  technol¬ 
ogy,  subject  matter,  learner  characteristics,  and 
pedagogical  principles;  a  synthetic  experience,  as 
opposed  to  a  real-world  interaction  with  an  actual 
device  or  process,  is  created  for  the  learner  through 
a  simulation,  game,  or  other  technology. 


INTRODUCTION 

The  use  of  technology  in  education  has  received  tre¬ 
mendous  attention  over  the  last  30  years,  and  its  role 
is  likely  to  increase  even  more  as  the  demand  for 
teaching  students  efficiently  and  effectively  continues 
to  grow.  Indeed,  a  wide  variety  of  computer-aided 
instructional  packages,  educational  games ,  automated 
tutors,  and  other  technologies  have  been  developed 
to  teach  or  aid  in  teaching.  Although  some  of  these 
development  efforts  have  been  successful,  others  have 
not,  and  still  others  (the  majority  perhaps)  are  of 
unknown  value  because  they  have  never  undergone 
rigorous  empirical  testing.  Moreover,  past  technol¬ 
ogy-based  education  development  efforts  have  too 
often  been  driven  by  the  technology  rather  than  the 
training  needs  or  an  attempt  to  make  use  of  the  newest 
gadget  to  come  along  (Gee,  2003).  For  all  these  rea¬ 
sons,  we  believe  that  the  use  of  technology-based 
educational  approaches  is  suboptimal. 

To  address  this  issue,  it  must  be  recognized  that  to 
create  an  optimal  environment  for  learning  we  must 
consider  the  confluence  of  not  only  technology  and 
subject  matter  but  also  learner  characteristics  and 
pedagogical  principles.  These  are  the  ingredients  of  a 
true  synthetic  learning  environment  (SLE)  (Dede  et  al., 
1999).  Such  environments,  we  suggest,  are  an  excel¬ 
lent  opportunity  to  realize  the  benefits  of  technology- 
enabled  education.  In  an  effort  to  guide  the  develop¬ 
ment  of  SLEs,  this  chapter  reviews  areas  of  the  scien¬ 
tific  literature  that  are  most  pertinent,  but  often  over¬ 
looked,  in  the  development  of  technology-enhanced 
learning  instruments. 

Definitions 

Before  we  proceed,  it  is  first  important  to  define  the 
key  terms  that  we  are  using  in  this  chapter:  synthetic 
learning  environments  (SLEs),  simulations,  games. 


and  virtual  worlds.  We  define  SLEs  as  tools  that  com¬ 
bine  the  key  elements  of  learning  theory  and  technol¬ 
ogy  to  create  a  context  where  optimal  learning  is 
achieved.  Importantly,  SLEs  create  a  synthetic  experi¬ 
ence  as  the  basis  for  learning;  that  is,  they  support 
learning  by  augmenting,  replacing,  creating,  or  man¬ 
aging  a  learner’s  actual  experience  with  the  world. 
Some  of  the  technologies  used  to  create  synthetic  expe¬ 
rience  include  simulations  and  games,  which  are  typ¬ 
ically  computer  based. 

The  term  simulation  typically  refers  to  a  training 
device  characterized  by  “a  working  representation  of 
reality  ...  [that]  may  be  an  abstracted,  simplified,  or 
accelerated  model  of  process”  (Galvao  et  al.,  2000,  p. 
1692).  As  such,  simulations  can  vary  from  relatively 
low-fidelity  representations  of  a  process  to  elaborate, 
extremely  realistic,  full-motion  platforms.  Simulation- 
based  training,  therefore,  is  a  type  of  training  that 
depends  on  the  simulation  to  provide  essential  cues  to 
trigger  appropriate  behaviors.  Whereas  definitions  of 
simulation  have  focused  largely  on  the  characteristics 
of  the  learning  environment,  definitions  of  games  have 
more  often  focused  on  the  nature  of  the  learner’s  expe¬ 
riences;  for  example,  Vogel  and  her  colleagues  (2006) 
have  suggested  that  the  essential  characteristics  of  a 
computer  game  are  goals,  interactivity,  and  feedback. 
Galvao  et  al.  (2000)  include  rules  and  competition  as 
other  important  characteristics. 

The  constructs  of  game  and  simulation  are  not 
orthogonal;  in  fact,  we  view  them  as  overlapping  con¬ 
structs.  It  is  quite  possible,  for  example,  to  have  a 
simulation  that  has  no  gaming  features  (e.g.,  rules  of 
play,  competition,  scorekeeping)  or  a  game  that  is  not 
based  on  a  realistic  simulation;  however,  a  learning 
environment  may  have  both  simulation  and  gaming 
features.  Because  we  are  focusing  on  environments 
that  provide  synthetic  experience  in  this  chapter,  we 
are  most  interested  in  training  that  is  based  on  some 
type  of  simulation  of  the  environment  or  relevant  phe¬ 
nomenon.  This  does  not  mean  that  the  system  must  be 
built  on  an  accurate  physical  simulation,  but  it  does 
mean  that  crucial  aspects  of  the  actual  problem  or 
domain  are  simulated  (more  will  be  said  regarding  the 
issue  of  simulation  fidelity  in  subsequent  sections). 
Suffice  it  to  say  here  that  our  interest  is  in  SLEs  that 
promote  development  of  expertise  through  synthetic 
experience  (experience  with  simulated  or  artificial  or 
virtual  environments). 

Advances  in  technology  are  also  making  more 
sophisticated  types  of  SLEs  possible;  for  example, 
interest  is  emerging  in  using  virtual  worlds  as  training 
environments.  These  environments  typically  involve  a 
large  number  of  geographically  distributed  players 
who  all  interact  with  and  within  a  shared  sestet  of  cues 
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that  comprise  the  virtual  world.  Many  of  these  virtual 
worlds  are  persistent ;  that  is,  actions  continue  and  the 
world  changes  whether  or  not  any  given  player  is 
involved.  Interest  in  these  worlds  as  synthetic  learning 
environments  is  likely  to  increase  in  the  coming  years 
(Dickey,  2005). 

It  should  be  noted,  however,  that  games,  simula¬ 
tions,  and  so  forth  are  not  SLEs  in  and  of  themselves. 
Although  it  is  possible  to  learn  in  these  environments, 
reliable  learning  occurs  when  these  technologies  are 
used  to  present  or  practice  new  information  in  a  manner 
that  is  pedagogically  sound  (Schmidt  and  Bjork,  1992). 


JUSTIFICATION  FOR  USING  SUES 

We  should  point  out  that  there  are  very  few,  if  any, 
true  SLEs;  that  is,  painfully  few  attempts  have  been 
made  to  systematically  develop  a  learning  environment 
that  is  not  only  well  engineered  from  the  technological 
standpoint  but  also  built  to  be  consistent  with  current 
learning  science.  Rather,  SLEs  are  almost  always  built 
with  an  emphasis  on  technology,  and  pedagogy  is  often 
worked  in  as  an  afterthought.  In  fact,  in  several  areas 
the  science  is  currently  not  sufficient  to  give  appropri¬ 
ate  guidance,  even  if  it  were  sought  (Cannon-Bowers 
and  Bowers,  in  press);  however,  a  formidable  literature 
does  exist  upon  which  to  generate  hypotheses  about 
the  likely  components  of  SLEs  that  make  them  effec¬ 
tive.  This  literature  is  reviewed  briefly  below. 

Why  Should  SLEs  Work? 

Theoretical  justification  for  the  proposition  that  SLEs 
can  provide  a  viable  context  for  instruction  can  be 
found  in  what  is  known  about  how  expertise  is  devel¬ 
oped  through  experience.  Glaser  (1989)  argued  that 
beginners’  knowledge  is  spotty,  consisting  of  isolated 
definitions  and  superficial  understandings  of  central 
terms  and  concepts.  With  experience,  these  items  of 
information  become  structured  and  are  integrated  with 
past  organizations  of  knowledge.  Research  in  expertise 
also  suggests  that  experts  may  have  “stored  straightfor¬ 
ward  condition-action  rules  in  which  a  specific  pattern 
(the  condition)  will  trigger  a  stereotypic”  response  (Chi 
et  al.,  1988,  p.  xvii).  Through  the  acquisition  of  domain 
knowledge ,  it  is  argued  that  experts  build  up  a  repertoire 
of  instances,  indexing  them  in  such  a  way  that  they  are 
rapidly  accessible  when  triggered  by  environmental 
cues  (Gobet  and  Simon,  1996;  Logan,  1988). 

The  notion  that  experts  chunk  information  and  rec¬ 
ognize  meaningful  patterns  in  the  problem  space  has 
been  replicated  across  domains  (DeGroot,  1965;  Egan 
and  Schwartz,  1979;  Lesgold,  1988).  It  has  also  been 


established  that  experts  develop  a  well-organized  set 
of  schema  around  big  ideas  or  concepts.  According  to 
Bransford  et  al.  (2000,  p.  26),  “Experts  appear  to  pos¬ 
sess  an  efficient  organization  of  knowledge  with  mean¬ 
ingful  relations  among  related  elements  clustered  into 
related  units  that  are  governed  by  underlying  concepts 
and  principles.” 

Related  to  this,  learning  researchers  are  increas¬ 
ingly  making  a  distinction  between  rote  memorization 
of  knowledge  and  the  more  elusive  goal  of  learning 
for  understanding.  Bransford  et  al.  (2000)  suggested 
that  modern  views  of  learning  recognize  that  learning 
for  understanding  requires  more  than  knowledge  of  a 
series  of  disconnected  facts.  They  pointed  to  research 
that  suggests  that  expert  knowledge  is  “connected  and 
organized  around  important  concepts,”  that  it  is  “con- 
ditionalized,”  and  that  it  supports  the  ability  to  transfer 
to  other  areas  or  domains  (Bransford  et  al.,  2000,  p.  9). 

The  Potential  of  SLEs  for  Learning 

These  conclusions  have  several  implications  for  the 
design  of  SLEs.  One  of  the  most  pertinent  is  the  notion 
that  learning  occurs  best  in  a  meaningful  context;  that 
is,  effective  learning  leads  people  to  recognize  patterns 
that  are  used  as  the  basis  for  knowledge  organization, 
and  this  knowledge  is  conditionalized  (i.e.,  specific  to 
a  context),  suggesting  that  learning  environments 
should  provide  realistic  situations  and  problems  for 
learners.  New  learning,  in  turn,  can  then  be  integrated 
into  this  past  learning  and  world  knowledge.  Indeed, 
many  lines  of  inquiry  into  the  science  of  learning  con¬ 
verge  on  the  conclusion  that  experiential  learning  (i.e., 
learning  through  experience)  is  a  fundamental  human 
process  (Kolb,  1984;  Kolb  et  al.,  2001;  see  also  Chap¬ 
ter  4  in  this  volume) . 

The  concepts  of  anchored  instruction  and  situated 
learning  are  both  consistent  with  the  underlying 
assumptions  of  instruction  based  on  experiential  learn¬ 
ing  (Bransford  et  al.,  1990;  CTGV,  1992, 1997, 2000). 
These  instructional  approaches  rest  on  the  pedagogical 
principle  that  for  learning  to  be  effective  it  must  be 
anchored  in  a  meaningful  context  for  learners.  Anchor¬ 
ing  instructions  allows  learners  to  understand  how  con¬ 
cepts  are  applied  and  why  they  are  important  and  use¬ 
ful,  thus  enabling  new  learning  to  be  more  easily 
integrated  into  existing  knowledge  and  mental  models. 
According  to  the  Cognition  and  Technology  Group  at 
Vanderbilt  (CTGV,  2000),  anchored  instruction  seeks 
to  assist  learners  in  understanding  the  types  of  prob¬ 
lems  and  opportunities  that  real  experts  confront  and 
how  they  use  their  knowledge  to  solve  those  problems. 
It  also  aids  students  in  integrating  their  knowledge  by 
employing  multiple  perspectives  on  the  same  problem. 
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In  addition,  anchored  (situated)  learning  environments 
allow  learners  to  understand  how  new  information  is 
connected  to  what  they  already  know.  These  processes 
of  accommodating  ideas  to  the  external  world  and 
assimilating  experience  into  existing  knowledge  struc¬ 
tures  are  critical  driving  forces  underlying  cognitive 
development  (Kolb,  1984). 

Synthetic  learning  environments  provide  a  unique 
opportunity  to  achieve  these  objectives,  in  some  cases 
even  more  effectively  and  less  expensively  than  other 
forms  of  instruction.  According  to  Cannon-Bowers  and 
Bowers  (in  press),  SLEs  offer  a  number  of  advantages 
as  compared  to  more  traditional  forms  of  training 
(especially  in  the  workplace)  because  they: 

•  Can  be  used  as  practice  environments  for 
tasks  that  are  too  dangerous  to  be  practiced 
in  the  real  world 

•  Can  provide  increased  opportunities  for 
practice  on  tasks  that  occur  infrequently 
(e.g.,  emergency  procedures) 

•  Are  available  when  actual  equipment  cannot 
be  employed 

•  Can  contain  embedded  instructional  features 
(e.g.,  feedback)  that  enhance  the  instruc¬ 
tional  experience 

•  Can  represent  significant  cost  savings  com¬ 
pared  with  training  on  operational  equip¬ 
ment 

Much  popular  attention  has  also  recently  been 
directed  toward  employing  video  and  computer 
games  as  an  opportunity  to  improve  educational  out¬ 
comes  by  capturing  the  motivational  aspects  of  elec¬ 
tronic  gaming  for  educational  purposes  (Gee,  2003; 
Prensky,  2002).  Considerable  interest  has  been  gen¬ 
erated  in  electronic  games  for  education,  at  least  in 
part  because  of  the  obvious  enthusiasm  with  which 
many  children  and  adults  currently  play  with  them. 
Advocates  suggest  that  gaming  could  increase  trainee 
enthusiasm  for  educational  material,  which  could  in 
turn  increase  time  on  task  and  lead  ultimately  to 
improved  student  performance. 

Moreover,  the  advent  (and  availability)  of  persis¬ 
tent  virtual  worlds  for  entertainment  purposes  is  likely 
to  grow  considerably  in  the  next  few  years;  hence,  the 
technology  will  exist  and  be  ripe  for  application  to 
training  challenges.  As  noted,  popular  Web-based  vir¬ 
tual  worlds  can  attract  millions  of  users  (e.g..  Second 
Life,  World  of  Warcraft)  even  when  a  fee  is  associated 
with  playing.  It  is  not  difficult  to  imagine  using  a 
similar  approach  to  create  realistic  virtual  organiza¬ 
tions,  markets,  and  economies  that  could  provide 
employees  with  opportunities  to  explore,  acquire,  and 


test  important  job-related  skills  (e.g.,  decision  making, 
leader  behavior,  communication)  in  a  safe,  available, 
and  engaging  environment.  Of  course,  the  effective¬ 
ness  of  such  an  approach  would  hinge  largely  on  its 
ability  to  embody  sound  principles  of  instructional 
design. 

EMPIRICAL  LITERATURE 
REGARDING  SLE  EFFECTIVENESS 

The  potential  effectiveness  of  components  of  SLEs  for 
effective  teaching  has  been  demonstrated  in  many  set¬ 
tings;  for  example,  a  recent  meta-analysis  has  sup¬ 
ported  the  efficacy  of  these  environments  (Vogel  et  al., 
2006)  in  educational  settings.  In  addition  to  the 
research  cited  by  Vogel  et  al.  regarding  K-12  education 
and  special  education,  other  data  suggest  that  similar 
gains  can  be  obtained  with  college  students.  Green  and 
Bavelier  (2003),  for  example,  described  a  series  of 
studies  that  suggest  that  video  game  playing  can 
improve  visual  attention  skills  in  college  students. 
Likewise,  Emurian  (2005)  reported  that  a  Web-based 
tutor  was  effective  in  improving  college  students’  pro¬ 
gramming  skills. 

Using  an  immersive  game,  McClean  and  col¬ 
leagues  (2001)  demonstrated  improved  learning  out¬ 
comes.  These  researchers  compared  learning  outcomes 
from  a  virtual-world-based  geology  game  and  a  vir¬ 
tual-world-based  biology  game  with  outcomes  from 
both  Web-based  presentation  of  the  material  and  tra¬ 
ditional  classroom  lectures.  Lectures  produced  the 
lowest  learning  outcomes  as  measured  by  a  graded  test. 
The  Web-presented  information  increased  those  out¬ 
comes  by  a  margin  that  ranged  from  not  statistically 
significant  (geology)  to  13  to  30%  (cell  biology).  The 
games  further  increased  learning  outcomes  either  15 
to  40%  (geology)  or  30  to  63%  (cell  biology).  These 
outcomes  suggest  that  it  is  not  merely  the  visual  rep¬ 
resentation  of  material  but  also  the  active  engagement 
that  stimulates  improved  learning.  This  is  consistent 
with  findings  from  other  researchers  (Kearsley  and 
Schneiderman,  1998). 

Another  set  of  data  has  been  collected  by  Squire 
and  colleagues  (2004)  on  their  game  Supercharged!. 
In  this  study,  a  control  group  receiving  interactive  lec¬ 
tures  improved  their  understanding  by  15%  over  their 
pretest  scores,  while  those  who  played  with  the  game 
improved  their  understanding  by  28%.  These  learning 
outcomes  for  Supercharged!  are  in  line  with  those  for 
the  visual  representations  of  the  McClean  et  al.  (2001) 
study,  and  this  study  also  demonstrates  that  games  are 
superior  to  lectures.  Indeed,  among  girls,  the  control 
group  (lecture  only)  improved  on  their  pretest  scores 
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by  only  5%,  but  in  the  simulation  group  this  number 
was  23%;  thus,  at  least  in  this  one  study,  lectures  alone 
did  nothing  for  girls. 

Low-fidelity  simulations  have  also  been  demon¬ 
strated  to  be  effective  in  workplace  and  occupational 
training;  for  example,  several  studies  have  supported 
the  contention  that  management  simulations  can  be 
effective  training  environments  for  graduate  students 
and  professionals  (Gredler,  2004;  Scherpereel, 
2005).  In  addition,  pilot  training  has  relied  success¬ 
fully  on  simulators  to  train  both  technical  and  non¬ 
technical  skills  (Goeters,  2002;  Jentsch  and  Bowers, 
1998;  Roessingh,  2005).  Some  studies  also  suggest 
that  SLEs  can  help  train  clinicians  (Abell  and  Galin- 
sky,  2002;  Lane  et  al.,  2001;  Pederson,  2000),  mili¬ 
tary  personnel  (Pleban  et  al.,  2002;  Ricci  et  al., 
1996),  firemen  (Spagnolli  et  al.,  2003),  survey  inter¬ 
viewers  (Link  et  al.,  2006),  and  elderly  drivers 
(Roenker  et  al.,  2003). 

It  should  be  noted,  however,  that  SLEs  have  not 
been  found  to  be  universally  effective.  Dunn  (2003) 
failed  to  find  any  change  in  knowledge  of  workplace 
diversity  issues  among  trainees  who  used  a  computer 
game  relative  to  a  non-treatment  control  group.  Cam¬ 
eron  and  Dwyer  (2005)  also  failed  to  find  an  effect 
of  computer  games  in  reducing  field  dependence. 
Garg  and  colleagues  (2002)  did  find  a  small  advan¬ 
tage  for  virtual  environments  in  teaching  anatomy  to 
medical  students,  but  these  differences  disappeared 
after  controlling  for  spatial  ability.  The  mediating 
effects  of  spatial  ability  were  also  found  in  a  labora¬ 
tory  study  of  virtual  environments  reported  by  Waller 
(2000).  More  generally,  Ellis  and  colleagues  (2005) 
found  that  an  online  case  simulation  system  resulted 
in  poorer  performance  and  weaker  knowledge  struc¬ 
tures  among  veterinary  students  relative  to  their  tra¬ 
ditionally  trained  peers. 

FACTORS  THAT  INFLUENCE 
SLE  DESIGN 

As  noted,  few  attempts  to  develop  and  test  full-scale 
SLEs  have  been  accomplished.  This  is  particularly  true 
if  one  adopts  a  systems  view  of  instructional  design 
that  includes  not  only  technology  but  also  character¬ 
istics  of  the  learner,  instructional  features  and  strate¬ 
gies,  and  motivational  factors.  We  contend  that  all  of 
these  factors  must  be  considered  in  designing  SLEs 
(see  various  chapters  in  Parts  I  and  II  in  this  volume 
for  more  on  strategies  and  approaches).  A  brief  review 
of  variables  in  each  of  these  areas  that  we  believe  are 
most  crucial  to  sound  SLE  design  is  contained  in  the 
sections  that  follow. 


Learner  Characteristics 

Education  and  training  scholars  have  long  acknowl¬ 
edged  that  individual  differences— that  is,  character¬ 
istics  that  trainees  bring  to  the  learning  situa¬ 
tion-influence  learning  outcomes  (Jonassen  and 
Grabowski,  1993).  Among  the  variables  studied  are 
cognitive  ability  (with  higher  ability  learners  gener¬ 
ally  outperforming  lower  ones;  see  Mayer  and  Massa, 
2003),  prior  knowledge  and  experience  (with  advan¬ 
tages  for  learners  with  more  experience  in  a  domain; 
see  Shih  et  al.,  2006),  self-efficacy  (with  higher  self- 
efficacy  typically  associated  with  enhanced  learning; 
see  Gist  et  al.,  1991),  and  goal  orientation  (with  learn¬ 
ers  who  focus  on  their  mastery  of  training  content 
better  able  to  transfer  knowledge  than  learners  who 
are  more  concerned  with  performing  well  in  training; 
see  Sideridis,  2005).  It  is  likely  the  case  that  these 
characteristics  will  also  have  an  influence  on  learning 
outcomes  in  SLEs;  however,  because  the  presentation 
of  content  is  relatively  flexible,  it  may  be  possible  to 
develop  SLEs  that  can  be  tailored  to  the  individual 
needs  of  the  trainee.  This  possibility  could  help  mit¬ 
igate  the  impact  of  lower  ability  or  self-efficacy  or 
the  lack  of  prior  experience. 

Several  other  individual  differences  may  also 
impact  SLE  effectiveness;  for  example,  it  has  been 
noted  that  spatial  ability  appears  to  be  an  important 
potential  moderator  in  determining  SLE  effectiveness. 
Evidence  from  studies  by  Garg  and  colleagues  (2002) 
and  Waller  (2000)  suggest  that  a  learner’s  spatial  abil¬ 
ity  may  influence  his  or  her  ability  to  benefit  from  an 
SLE.  It  seems  reasonable  to  suggest  that  the  demands 
imposed  on  learners  in  an  interactive,  three-dimen¬ 
sional  SLE  are  different  than  those  associated  with 
more  passive  forms  of  instruction  (e.g.,  listening  to  a 
lecture  or  reading  a  book)  or  even  other  more  static 
types  of  computer-based  instruction. 

Likewise,  comfort  with  technology  may  be  an 
important  mediating  factor  in  the  effectiveness  of 
SLEs.  Past  researchers  have  discussed  knowledge-  and 
skill-based  constructs  such  as  “digital/computer  liter¬ 
acy”  (Gee,  2003)  and  “technological  fluency”  (Baker 
and  O’Neil,  2003).  In  this  regard,  it  has  been  found 
that  less  comfortable  participants  tend  to  benefit  less 
from  these  environments.  Others  have  focused  on  self- 
efficacy  as  it  relates  to  computers  in  general  (Hasan, 
2003)  or  specific  software  packages  (Johnson,  2005). 
Still  others  have  discussed  negative  emotions  associ¬ 
ated  with  computers,  such  as  computer  anxiety 
(McFarland  and  Hamilton,  2006)  or  technophobia 
(Brosnan,  1999),  also  indicating  that  these  negative 
emotions  might  interfere  with  learning  in  technologi¬ 
cal  environments. 
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Past  researchers  have  also  suggested  that  precon¬ 
ceived  attitudes  toward  or  expectations  for  training  are 
effective  predictors  of  subsequent  learning  outcomes 
(Smith-Jentsch  et  al.,  1996;  Tannenbaum  et  al.,  1991). 
With  respect  to  SLEs,  it  might  be  that  the  introduction 
of  an  SLE  (especially  a  game,  but  even  a  simulation 
or  virtual  world)  diverges  so  far  from  the  learner’s 
experience  that  it  colors  the  learner’s  expectations  for 
how  effective  it  will  be.  In  fact,  some  learners  may  not 
readily  accept  the  use  of  a  game  to  train  serious  knowl¬ 
edge  and  skill;  for  example,  it  is  clear  that  some  stu¬ 
dents  are  at  least  dubious  about  being  educated  in 
computer-based  environments  (Hunt  and  Bohlin, 
1993).  Similarly,  even  if  the  learner  has  positive  expec¬ 
tations  regarding  the  outcomes  of  this  type  of  learning, 
negative  attitudes  of  instructors  or  teachers  who  are 
part  of  the  learning  process  may  color  the  outcomes 
(Mac Arthur  and  Malouf,  1991). 

Interestingly,  the  issue  of  expectations  may  be  a 
double-edged  sword.  Specifically,  the  notion  that  train¬ 
ing  will  include  a  video  game  could  conjure  up  very 
specific  expectations  in  learners  who  have  experience 
playing  popular  entertainment  games.  If  the  educa¬ 
tional  game  fails  to  meet  trainee  expectations  (for  rea¬ 
sons  that  have  nothing  to  do  with  its  efficacy  as  a 
learning  device),  its  effectiveness  could  suffer;  see 
Tannenbaum  et  al.,  1991). 

Instructional  Features  and  Strategies 

Synthetic  learning  environments  are  complex  learning 
systems  that  depend  on  a  host  of  factors  to  be  success¬ 
ful;  in  fact,  SLEs  can  be  developed  in  countless  ways, 
and  many  variables  must  be  taken  into  account.  By  the 
same  token,  it  is  this  inherent  flexibility  that  makes 
SLEs  useful  across  a  variety  of  settings  and  for  a  host 
of  learning  objectives.  The  sections  that  follow  high¬ 
light  some  of  the  instructional  features  that  must  be 
considered  in  SLE  design. 

Authenticity /Fidelity 

One  variable  that  has  received  attention  in  the  educa¬ 
tional  design  area  is  the  authenticity  of  the  learning 
experience.  Authentic  experiences  are  those  for  which 
students  can  make  meaningful  connections  to  their 
actual  experience.  This  is  an  important  feature  of  the 
learning  environment  according  to  the  anchored- 
instruction  view.  Jonassen  (2000) ,  however,  noted  that 
authentic  instruction  does  not  necessarily  have  to  be 
developed  around  specific,  real-world  tasks;  rather, 
authenticity  can  best  be  described  as  the  extent  to  which 
the  learning  environment  causes  learners  to  engage  in 
cognitive  processes  that  are  similar  to  those  in  the  real 


world  (Honebein  et  al.,  1993;  Petraglia,  1998).  More¬ 
over,  authentic  learning  environments  provide  learners 
with  engaging,  challenging  problems  that  are  similar 
to  those  confronted  by  experts  (CTGV,  2000). 

A  related  issue  concerns  model  or  simulation  fidel¬ 
ity— that  is,  the  degree  to  which  underlying  models  or 
simulations  faithfully  represent  the  actual  phenomenon 
or  task  it  is  meant  to  represent  (Andrews  and  Bell, 
2000).  The  question  here  typically  concerns  the  ability 
to  transfer  specific  knowledge  and  skill  gained  in  train¬ 
ing  to  the  actual  operational  or  job  environment 
(Andrews  and  Bell,  2000).  When  trainees  are  learning 
how  to  apply  a  particular  skill,  then  the  training  (sim¬ 
ulated)  environment  must  respond  in  a  realistic  manner 
or  the  trainee  will  receive  incorrect  feedback  and  per¬ 
haps  experience  negative  transfer  (i.e.,  applying  skills 
incorrectly  on  the  job) .  With  respect  to  more  traditional 
educational  domains  (i.e.,  even  when  there  is  no  target 
environment  to  which  learning  is  directly  transferred) , 
the  issue  is  still  important  to  ensure  that  what  students 
are  learning  is  actually  correct. 

Model-Based  Reasoning 

The  notion  of  model-based  reasoning  has  gained  atten¬ 
tion  in  recent  years.  This  line  of  work  contends  that 
students  need  to  develop  valid  mental  representations 
of  the  physical  world  and  learn  to  create,  revise,  and 
use  these  models  to  reason  about  actual  physical  phe¬ 
nomena.  This  view  characterizes  scientific  inquiry  as 
sense  making  and  contends  that  models  and  modeling 
can  help  bridge  the  gap  between  the  concrete  and  the 
abstract  (Raghavan  et  al.,  1997).  Lehrer  and  Schauble 
(2000)  contend  that  model-based  reasoning  actually 
begins  early  in  childhood,  yet  model-based  instruction 
is  typically  not  introduced  until  much  later  in  educa¬ 
tion.  Further,  they  proposed  a  taxonomy  of  models 
(including  physical  microcosms,  representational  sys¬ 
tems,  syntactic  models,  and  hypothetical-deductive 
models)  based  on  Gentner’s  (1983)  syntactical  theory 
of  analogy. 

Positive  findings  to  support  the  use  of  model-based 
instruction  come  from  evidence  in  genetics,  evolution¬ 
ary  biology,  near-Earth  astronomy  (Cartier  and  Stew¬ 
art,  2000;  Stewart  et  al.,  2005),  and  scientific  reasoning 
(Raghavan  et  al.,  1997;  Zimmerman  et  al.,  2003). 
Taken  together,  these  studies  indicate  that  interacting 
with  models  as  part  of  a  formalized  curriculum  should 
contribute  to  the  students’  deep  understanding  of  the 
domain,  as  well  as  problem-solving  skills.  Along  these 
same  lines,  others  argue  that  computer  simulations 
afford  the  unique  opportunity  to  engage  learners  in  the 
study  of  abstract,  complex  physical  phenomena  (Dede 
et  al.,  1999)  by  making  them  more  concrete. 
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Scenario/Case  Design 

An  important  concern  in  designing  effective  SLEs  is 
the  design  of  scenarios  or  cases  that  provide  the  stim¬ 
ulus  and  context  for  instruction.  According  to  Can¬ 
non-Bowers  et  al.,  in  simulation-based  training,  the 
scenario  is  often  the  primary  device  by  which  instruc¬ 
tional  objectives  are  met.  Recently,  Cannon-Bowers, 
Salas,  and  colleagues  (Cannon-Bowers  and  Salas, 
1998;  Fowlkes  et  al.,  1998;  Salas  and  Cannon-Bow¬ 
ers,  1997)  developed  a  methodology  where  scenarios 
are  designed  by  scripting  and  sequencing  specific 
trigger  events.  Trigger  events  are  designed  to  elicit 
targeted  behavior,  thereby  allowing  trainees  to  prac¬ 
tice  and  receive  feedback  on  targeted  skills  (Salas  and 
Cannon-Bowers,  2001).  This  event -based  approach 
to  simulation-based  training  has  been  successfully 
demonstrated  in  several  settings  to  date  (Dwyer  et  al., 
1999;  Fowlkes  et  al.,  1994;  Johnston  et  al.,  1995).  In 
a  similar  vein,  Schank  et  al.  (1999)  advocated  a  strat¬ 
egy  for  developing  goal-based  scenarios  or  cases  that 
includes  guidance  for  developing  learning  goals ,  mis¬ 
sions,  cover,  role,  scenario  operations,  resources,  and 
feedback. 

One  of  the  problems  with  scenario  or  case  design 
is  that  it  is  typically  very  time  consuming  and  requires 
expertise  from  subject-matter  experts  and  instructional 
designers.  A  possible  solution  to  this  challenge  is  to 
automate  the  development  of  scenarios  by  providing 
authoring  tools  for  users  (Jentsch  et  al.,  2001).  Devel¬ 
oping  such  tools  should  be  a  priority  in  future  work. 

Collaborative! Social  Learning 

The  notion  that  collaboration  can  aid  learning  has 
gained  traction  with  instructional  researchers  in  recent 
years  (CTGV,  2000;  Clark  and  Wittrock,  2000). 
According  to  Nelson  (1999),  collaborative  learning  is 
effective  because  it  takes  advantage  of  learners’  natural 
collaborative  processes  while  fostering  exchange  in 
rich  social  contexts  and  allowing  for  multiple  perspec¬ 
tives.  Others  advocate  the  creation  of  learning  com¬ 
munities  in  classrooms  (Bielaczyc  and  Collins,  1999) 
that  emphasize  development  of  mutual  respect,  diver¬ 
sity,  and  community  growth.  Examples  of  implemen¬ 
tations  of  this  type  of  approach  include  Brown  and 
Campione’s  (1994,  1996)  “fostering  a  community  of 
learners”  approach  and  Lamport's  (1990)  work  in 
mathematics  instruction.  Indeed,  a  meta-analysis  of 
some  122  studies  of  classroom  instruction  showed  that, 
on  average,  cooperative  modes  of  learning  consider¬ 
ably  outperformed  competitive  or  solo  approaches,  and 
that  the  latter  two  were  roughly  comparable  in  terms 
of  learning  outcomes  (Johnson,  1981). 


These  findings  are  particularly  pertinent  to  the 
design  of  SLEs  because  technology  affords  the  oppor¬ 
tunity  to  develop  multiplayer  and  team  simulations 
(and  games).  In  fact,  many  argue  that  online  gaming 
is  essentially  a  social  phenomenon.  Gaming  occurs  in 
distributed  social  groups  that  resemble  communities 
of  practice  because  they  have  their  own  language,  prac¬ 
tice,  and  norms.  It  has  been  argued  that  these  commu¬ 
nities  are  excellent  potential  learning  environments 
(Squire,  2003). 

Motivational  Factors 

Many  learning  scholars  acknowledge  that  motivation 
is  paramount  to  learning  (Clark  and  Wittrock,  2000); 
in  fact,  numerous  variables  appear  to  effect  a  student’s 
motivation  to  learn,  which  in  turn  influences  learning. 
It  is  important,  then,  to  understand  how  SLE  design 
can  foster  (or  inhibit)  motivation  to  learn.  The  follow 
sections  briefly  review  some  of  the  variables  that  are 
unique  to  SLEs  in  this  regard. 

Goal  Setting! Acceptance 

In  general,  goal  setting  involves  establishing  a  standard 
or  objective  for  performance.  In  learning  systems, 
goals  help  to  focus  learners  on  the  task  and  help  them 
to  select  or  construct  strategies  for  goal  accomplish¬ 
ment;  hence,  they  serve  to  direct  attention  (Locke  and 
Latham,  1990).  Goal  commitment  (i.e.,  the  degree  to 
which  the  learner  is  committed  to  the  learning  goal) 
is  a  determinant  of  how  much  the  goal  affects  perfor¬ 
mance  (Locke  et  al.,  1981).  In  addition,  goal  setting 
has  been  linked  to  self-regulatory  processes  (Schunk 
and  Ertmer,  1999),  which  are  associated  with  effective 
learning.  Research  has  also  shown  that,  to  be  most 
effective  in  enhancing  performance,  goals  must  be  spe¬ 
cific  and  difficult  but  achievable  and  proximal  (Locke 
and  Latham,  1990). 

With  respect  to  SLEs  (and  specifically  games), 
goal  setting  is  particularly  relevant  because  SLEs  are 
often  built  to  emphasize  goal  accomplishment.  In 
many  games,  for  example,  players  must  reach  a  set  of 
interim  or  subgoals  in  pursuit  of  overriding  objectives. 
This  mechanism  may  actually  lend  itself  to  goal  setting 
in  educational  games  because  instruction  can  be  nested 
naturally  in  an  underlying  goal  hierarchy  or  structure. 
Exploiting  this  possibility  warrants  further  attention. 

Engagement/Emotional  Context 

With  the  advent  of  virtual  environments,  researchers 
have  begun  to  explore  variables  associated  with 
engagement  or  immersion  in  learning.  The  notion  of 
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immersion  is  being  hypothesized  to  be  a  psychological 
state  resulting  from  a  participant’s  intense  feelings  of 
“presence”  in  the  virtual  world  (Gerhard  et  al.,  2004). 
A  related  topic  is  the  idea  of  flow.  Csikszentmihalyi 
(1990)  described  flow  as  an  experience  where  an  indi¬ 
vidual  becomes  so  engaged  in  an  activity  that  time 
becomes  distorted,  self-consciousness  is  forgotten,  and 
external  rewards  disappear.  Instead,  people  engage  in 
complex,  goal-directed  behavior  because  it  is  inher¬ 
ently  motivating.  From  a  pedagogical  standpoint,  it  is 
not  clear  whether  or  how  this  notion  of  flow  may  affect 
the  learning  process;  however,  from  a  strictly  time-on- 
task  perspective  (i.e . ,  the  amount  of  time  engaging 
instructional  content),  it  would  follow  that  intense 
engagement  should  benefit  learning. 

Intensity  of  the  learning  experience  may  also 
impact  the  extent  of  transfer  of  learning,  with  the  hope 
being  that  game  play  drives  behavioral  change  long 
after  the  game  is  over.  Limited  evidence  of  this  possi¬ 
bility  is  encouraging;  for  example,  diabetic  children 
presented  with  a  diabetes  management  game  had  77% 
fewer  follow-up  visits  (emergency  and  urgent  care) 
than  a  control  group  (Brown  et  al.,  1997).  Playing  the 
game  was  not  enforced.  Children  were  simply  given 
the  game  and  allowed  to  take  it  home  to  play  it  as 
much  or  as  little  as  they  wished;  yet,  the  strong  rate 
of  casual  adoption  for  the  game  medium,  combined 
with  its  unique  learning  paradigm,  clearly  had  an 
impact. 

Reward  and  Social  Status 

Decades  of  research  into  motivation,  especially 
intrinsic  motivation,  generally  shows  that  externaliz¬ 
ing  the  reward  for  engaging  in  a  behavior  can  reduce 
a  person’s  intrinsic  motivation  to  engage  in  it  (Deci 
et  al.,  1999;  Ryan  and  Deci,  2000);  however, 
researchers  have  also  found  that  under  certain  condi¬ 
tions,  externalized  rewards  and  competition  can 
increase  intrinsic  motivation  (Reeve  and  Deci,  1996). 
In  particular,  an  activity  that  is  seen  as  challenging 
allows  the  user  to  gain  feelings  of  competence  and  is 
not  perceived  as  being  controlling  (i.e.,  forced  on  the 
user),  and  intrinsic  motivation  is  enhanced.  In  addi¬ 
tion,  when  the  user  places  importance  on  doing  well, 
competition  generates  affective  involvement  in  the 
activity  and  increases  personal  meaningfulness 
(Epstein  and  Harackiewicz,  1992).  It  should  be  noted 
that  in  at  least  one  study  (with  children),  a  gender 
difference  was  found  with  respect  to  public  acknowl¬ 
edgment  of  achievement.  In  that  case,  boys  were  more 
likely  to  respond  to  such  public  recognition  than  girls 
(Nemeth,  1999). 


Embodiment,  Personalization,  and  Engagement 

The  notion  of  user  embodiment  has  gained  attention 
of  late  as  the  development  of  collaborative  computer 
systems  and  virtual  worlds  has  increased.  Embodiment 
refers  to  the  mechanism  of  representation  of  the  user 
in  the  virtual  world.  According  to  Gerhard  et  al.  (2004) , 
the  notion  of  embodiment  can  be  traced  to  philosoph¬ 
ical  writings  about  the  meaning  of  the  physical  body. 
Avatars  — graphical  representations  of  the  user  in  the 
world— help  to  establish  the  user’s  identity  and  pro¬ 
vide  a  basis  for  conversation  and  social  interaction 
(Gerhard  et  al.,  2004;  Slater  et  al.,  2000).  These  fac¬ 
tors,  in  turn,  are  predicted  to  increase  the  user’s  sense 
of  engagement  in  the  simulation. 

Although  far  more  empirical  work  is  needed  to 
better  understand  the  impact  of  embodiment  on  both 
engagement  and  learning,  some  data  suggest  that  per¬ 
sonalizing  the  interaction  with  the  student  in  a  virtual 
environment  can  improve  learning  and  retention  of 
science  content  (Moreno  and  Mayer,  2004).  Baylor’s 
work  on  intelligent  agents  and  mentors  similarly  indi¬ 
cates  an  impact  of  personalization  and  presentation 
style  on  learning  outcomes  (Baylor,  2001).  Limited 
evidence  suggests,  then,  that  increasing  a  students’ 
level  of  embodiment  and  personalization  can  increase 
engagement  and  learning. 


CONCLUSIONS 

Clearly,  many  factors  must  be  taken  into  account  in 
the  development  of  an  environment  optimized  for  the 
acquisition  of  new  knowledge  and  skill.  We  have  high¬ 
lighted  but  a  few  of  these  here  as  they  relate  specifi¬ 
cally  to  synthetic  learning  environment  development. 
Gaining  an  adequate  understanding  of  these  factors 
and  the  manner  in  which  they  interact  is  obviously  a 
formidable  research  challenge.  This  challenge  is  inten¬ 
sified  by  the  fact  that  SLE  research  is  typically  expen¬ 
sive  to  conduct  because  it  typically  requires  that  a 
sufficiently  developed  SLE  is  available  for  study.  It 
would  seem,  however,  that  the  return  on  investment  is 
this  area  stands  to  be  huge,  as  the  potential  to  improve 
the  educational  process  is  so  vast.  It  is  our  hope  that 
researchers  will  find  ways  to  conduct  viable  SLE 
research— possibly  by  partnering  in  the  development 
of  viable  testbeds  and  by  sharing  facilities.  We  are  also 
hopeful  that  funding  to  support  large-scale  efforts 
becomes  available.  In  the  meantime,  we  encourage 
researchers  to  adopt  a  broad  systems  approach  to  SLE 
research  so  integration  and  generalization  of  findings 
can  be  achieved. 
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ABSTRACT 

Modeling  technologies  are  a  powerful  tool  for  affect¬ 
ing  learning.  Although  dynamic  modeling  is  not  an 
easy  task,  modeling  technologies  are  maturing.  As 
tools  become  easier  to  use,  more  people  will  build 
models,  and  this  is  likely  to  drive  learning  theory  and 
practice  in  new  directions  (Hadwin  et  al.,  2005).  This 
chapter  centers  on  one  aspect  of  modeling:  dynamic 
modeling  with  technology,  where  students  build 
dynamic  models  that  “run.”  This  chapter  particularly 
does  not  address  other  forms  of  modeling  technologies 
such  as  concept  mapping  (as  a  form  of  modeling 
domain  knowledge),  expert  systems  (modeling  of 
problems),  and  case  databases  (modeling  experiences). 
The  chapter  begins  with  the  conceptual  framework  for 
modeling  with  technology,  describes  extant  dynamic 
modeling  tools,  reviews  investigations  of  the  factors 
associated  with  modeling  technologies  in  the  class¬ 
room,  and  points  to  future  design  and  research. 


KEYWORDS 

Model:  An  artifact  representative  of  an  object  or  of  an 
internal  interpretation,  often  represented  on  a  com¬ 
puter  screen. 

Simulation:  An  executable  (runnable)  model;  com¬ 
puter  software  that  allows  a  learner  to  manipulate 
variables  and  processes  and  observe  results. 

CONCEPTUAL  FRAMEWORK  OF 
MODELING  TECHNOLOGIES 

Terms  in  Context 

Modeling  technologies  are  computer  software  that 
students  use  to  build  and  run  dynamic  models.  In  this 
chapter  model  refers  to  the  actual  artifact  represented 
on  a  computer  screen,  not  the  mental  construct  of  an 
individual  or  the  original  that  is  being  modeled.  When 
students  build  models,  there  is  an  interplay  between 
their  current  understanding  of  the  original  that  is 
modeled  (the  original  may  be  real  or  imagined)  and 
the  computer  models  they  create.  Simulations  are 
computer  software  designed  by  others  with  their  own 
views  of  the  original  to  allow  learners  to  manipulate 
variables  or  processes  and  observe  the  output  results 
(Clariana,  1989).  When  students  use  simulations, 
there  is  interaction  between  their  growing  intuition 
and  the  underlying  viewpoint  of  the  simulation,  with 
the  simulation  pressing  a  particular  viewpoint  on  the 
learner;  thus,  there  is  a  contrast  between  the  two 


approaches.  Using  someone  else’s  model  (simula¬ 
tion)  tends  to  be  peremptory,  while  building  one’s 
own  model  tends  to  be  dialectical. 

The  first  executable  models  for  learning  (i.e.,  com¬ 
puter-based  simulations)  were  created  by  highly 
skilled  programmers  at  the  advent  of  mainframe  com¬ 
puters  (Alessi,  2000;  Riley,  1990).  Today’s  modeling 
tools  run  on  personal  computers  and  do  not  require 
programming  skills.  The  nature,  methods,  and  limits 
of  modeling  with  technology  depend  on  the  interrela¬ 
tionships  between  the  person  and  the  modeling  tech¬ 
nology.  Model  building  is  always  constrained  (and 
facilitated)  by  the  technology.  The  malleability  of  the 
model  artifact  depends  on  the  effort  and  ease  required 
to  amend  or  append  the  model,  which  depends  on  the 
modeling  technology.  Thus,  the  two  most  critical  mod¬ 
eling  design  decisions  are  deciding  what  to  show  (the 
representations)  and  what  actions  the  computer  pro¬ 
gram  should  take  under  which  conditions  (Horwitz, 
1999).  If  the  model  is  intended  to  be  used  by  other 
people,  the  activities  in  which  they  can  engage  have 
to  be  defined,  too. 

The  Relationship  between 
What  Is  Modeled  and  the  Model 

The  question  of  the  nature  of  the  relationship  between 
the  model  and  what  is  modeled  (the  original)  is  as  old 
as  modeling  itself.  The  research  literature  falls  in  two 
camps.  One  camp  stresses  the  representational  charac¬ 
ter  of  models,  meaning  that  models  stand  for  reality 
(Tergan  and  Keller,  2005)  or,  as  Korzybski  (1958),  the 
founder  of  general  semantics,  stated:  “A  map  is  not 
the  territory  it  represents,  but  if  correct,  it  has  a  similar 
structure  to  the  territory,  which  accounts  for  its  use¬ 
fulness”  (p.  58).  The  other  camp  argues  for  a  footprint 
or  inscriptional  nature  of  the  relationship  (Pea,  1994), 
where  the  model  is  the  shareable,  tangible  mental 
impression  that  is  left  behind.  The  model,  the  external 
representation  or  inscription,  is  second  hand  (i.e.,  not 
based  directly  on  a  phenomenon  or  system);  in  other 
words,  a  model  is  “denatured”  (Gibbons,  2001 ,  p.  514) 
because  it  is  the  interpretation  of  the  modeler. 

The  question  of  the  nature  and  validity  of  the  rela¬ 
tionship  between  model  and  modeled  depends  on  one’s 
philosophical  standpoint  and  ontological  and  epistemo¬ 
logical  beliefs  (see  Lesh  and  Doerr,  2003,  for  an  in- 
depth  discussion).  Empiricists  or  objectivists  would 
stress  the  existence  of  an  original  and  a  way  to  capture 
the  substance  and  consider  how  well  their  models  rep¬ 
resent  the  original.  Pragmatist  and  neo-pragmatist 
would  question  structural  relationships  between  theories 
and  reality.  Pragmatists  would  measure  validity  by  how 
well  a  model  is  suited  to  solve  a  particular  problem  or 
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function  in  a  particular  context  (Strike  and  Posner, 
1985).  The  postmodern  camp  would  argue  that  the 
validity  of  the  model  is  based  on  how  well  it  aligns  with 
the  perceptions/thoughts/beliefs  of  the  modeler  without 
regard  to  any  real  original,  because  the  original  is  either 
not  detectable  (skeptic  position)  or  only  a  unique  prod¬ 
uct  of  our  construction  (radical  constructivism). 

Numerous  kinds  of  models  can  be  used  to  represent 
phenomena  in  the  world  or  the  mental  representations 
that  learners  construct  to  represent  them.  Harris  (1999) 
described  three  kinds  of  models:  theoretical  models, 
experimental  models,  and  data  models.  Giere  (1999) 
described  several  kinds  of  models,  including  represen¬ 
tational  models,  abstract  models  (mathematical  mod¬ 
els),  hypotheses,  and  theoretical  models  (abstract  mod¬ 
els  constructed  with  theoretical  principles,  such  as 
Newton’s  laws).  Lehrer  and  Schauble  (2003)  described 
a  continuum  of  model  types,  including  physical  mod¬ 
els,  representational  systems  (grounded  in  resem¬ 
blance  between  the  model  and  the  world),  syntactic 
models  (summarizing  essential  functioning  of  a  sys¬ 
tem),  and  hypothetical-deductive  models  (formal 
abstractions),  de  Jong  and  van  Joolingen  (1998)  dis¬ 
tinguished  between  conceptual  and  operational  mod¬ 
els,  where  conceptual  models  hold  principles,  con¬ 
cepts,  and  facts  that  make  up  the  system  being 
modeled,  and  operational  models  include  sequences  of 
cognitive  and  noncognitive  operations  (e.g.,  proce¬ 
dures)  that  can  be  applied. 

Different  modeling  technologies  are  built  from  and 
complement  different  views.  These  different  views  of 
the  relationship  between  what  is  modeled  and  the  model 
may  require  different  inscriptional  and  visualization  sys¬ 
tems.  The  following  section  describes  and  classifies  var¬ 
ious  modeling  tools  that  support  the  construction  of 
dynamic  runnable  models  with  visual  outputs. 


DYNAMIC  MODELING  TOOLS 

Dynamic  and  executable  modeling  tools  generally 
have  these  attributes:  (1)  variables,  rules,  and  mathe¬ 
matical  equations  embedded  in  the  software  can  be 
executed;  (2)  the  output  and  behavior  of  the  model 
change  over  time;  (3)  changes  are  calculated  and  out¬ 
puts  are  visually  displayed;  and  (4)  new  variables  can 
be  added  and  relationships  to  existing  variables 
defined,  thereby  changing  the  behavior  of  the  system 
and  the  other  variables  within  the  system.  Given  these 
general  attributes,  which  set  dynamic  modeling  tools 
apart  from,  for  example,  concept  mapping  tools,  draw¬ 
ing  tools  such  as  CAD,  flow-chart,  and  other  static 
representation  tools,  the  remaining  class  of  dynamic 
modeling  tools  can  be  still  further  classified. 


Classifications  of  Dynamic 
Modeling  Tools  with  Examples 

The  following  analytical  categories  of  dynamic  mod¬ 
eling  builds  upon  and  extends  three  learning-related 
classification  approaches:  (1)  Jonassen  and  Reeves’ 
(1996)  learning  from  computers  and  learning  with 
computers  analogically  becoming  learning  from  fin¬ 
ished  models  and  learning  by  modeling  (Jonassen  and 
Strobel,  2005);  (2)  Maier  and  Grobler’s  (2000)  mod¬ 
eling-oriented  simulations  and  gaming-oriented  simu¬ 
lations  (although  their  use  of  the  term  simulation 
remains  unclear  primarily  because  models  are  sub¬ 
sumed  under  simulation,  where  we  would  distinguish 
between  simulation  and  models);  and  (3)  Mellar  and 
Bliss’  (1994)  model  expression  to  model  exploration 
continuum.  Although  using  different  terms,  these  clas¬ 
sifications  share  as  their  main  dimension  the  degree  of 
student  ownership  and  agency  in  the  construction  of 
the  model  and  the  visibility  of  the  components,  pro¬ 
cesses,  and  relationships  within  the  model.  Using  Mel¬ 
lar  and  Bliss’  (1994)  modeling  tools  continuum  as  a 
foundation,  Schwarz  and  White  (2005)  argued  that 
modeling  tools  still  differ  on  a  variety  of  dimensions 
such  as  whether  models  focus  on  problem  solving  or 
theory  building  and  so  are  more  dependent  on  instruc¬ 
tional  context  and  the  use  of  the  models  or  modeling 
but  are  less  applicable  to  define  the  tools  themselves. 

Other  dimensions  must  also  be  included  that  pro¬ 
vide  additional  analytical  categories  to  classify 
dynamic  modeling  technologies.  To  better  illustrate  the 
various  modeling  technologies,  Table  28.1  provides  a 
summary  of  them. 

Model-Exploration  vs.  Model-Building  Tools 

•  Model  exploration.  Simulations  (as  dis¬ 
cussed  by  Cannon-Bowers  in  this  Handbook) 
are  black  boxes  that  ask  for  input  and  display 
the  output,  but  calculations  are  hidden  in  the 
system,  and  causal  relationships  between 
variables  of  the  system  can  only  be  inferred. 
Input-output  testing  is  a  form  of  model 
exploration  that  stays  on  the  visible  surface 
of  a  model  (Du  Boulay  et  al.,  1999).  At  the 
other  end  of  the  spectrum  of  model  explora¬ 
tion  tools  are  the  so-called  glass-box  models 
(Resnick  et  al.,  2000),  where  the  mechanism 
and  functions  of  the  system’s  components 
are  open  for  inspection  and  exploration. 

•  Model  building.  Unlike  purely  model-explo¬ 
ration  tools  (both  black-box  and  glass-box 
approaches),  where  learners  are  able  to 
change  preselected  variables  as  a  form  of 
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input  into  the  finished  model  and  the  soft¬ 
ware  does  not  provide  the  option  to  alter  the 
system,  with  model-building  tools  students 
are  able  to  change  aspects  of  the  systems  and 
attributes  of  variables  and  agents,  as  well  as 
design  different  subsystems  and  functional¬ 
ities  of  the  system. 

For  both  model  exploration  and  model  building,  the 
degree  of  black-box  to  glass-box  exploration  and  mod¬ 
eling  becomes  blurry.  Some  tools  provide  functions, 
subsystems,  or  layers  that  are  not  visible,  not  accessi¬ 
ble,  or  are  not  manipulative  by  the  user/modeler  of  the 
system  (e.g.,  Model-It  has  a  very  limited  mathematical 
equation  level),  whereas  in  other  tools  all  functions 
and  features  allow  manipulation  by  the  learner  (e.g., 
STELLA). 

Unit  of  Modeling 

•  Variable-based.  Tools  based  on  system 
dynamics  such  as  STELLA,  PowerSim,  Ven- 
sim,  and  Model-It  have  their  own  formalized 
language  for  qualitative  modeling  (i.e., 
stocks,  flow,  converters)  and  underlying 
mathematical  equations  for  the  quantitative 
modeling  emphasize  the  interrelationship 
and  causal  effects  of  variables  on  each  other 
(Hannon  and  Ruth,  1999;  Richmond,  1993). 
Variables  can  be  anything  from  functions  in 
mathematical  models,  such  as  elements  in 
short-term  memory  in  an  information  pro¬ 
cessing  model,  to  the  number  of  adults  in  a 
population  model. 

•  Agent! individual-based.  Agent-based  (Axel¬ 
rod,  1997;  Reynolds,  1987)  or  individual- 
based  (Grimm  and  Railsback,  2005)  mod¬ 
eling  tools  build  simulations  based  on  con¬ 
sequences  of  local  interactions  of  members 
(agents/individuals)  of  a  population.  In  sys¬ 
tem  dynamics  models,  variables  do  not 
possess  attributes  but  behave  dynamically 
over  time  based  on  the  mathematical  for¬ 
mulas  that  define  the  causal  relationships 
of  individual  variables.  But,  in  agent -based 
tools,  agents  carry  different  attributes,  and 
their  behavior  changes  according  to  a  set 
of  programmable  rules  to  react  to  exposure 
to  different  contexts  within  an  environ¬ 
ment.  The  environments  in  which  the 
agents  interact  most  often  contain  a  simu¬ 
lated  or  virtual  environment  with  spatial 
dimensions  in  which  the  agents  are  acting, 
allowing  models  to  simulate,  for  example, 


the  spread  of  a  virus  across  a  large  popu¬ 
lation  or  to  simulate  players  in  a  soccer 
game  (for  a  comparison  of  agent-based 
modeling  tools,  see  Railsback  et  al,  2006; 
see  also  Gilbert  and  Bankes,  2002;  Tobias 
and  Hofmann,  2004). 

Purpose/Domain-Specific  vs.  Generic  Tools 

•  Purpose! domain-specific .  Most  all  pur- 
pose/domain-specific  tools  are  part  of  the 
model-exploration  or  simulation  area  and 
therefore  will  not  be  addressed  in  depth  in 
this  chapter.  Exceptions  include  business 
process  modeling  tools  such  as  ARIS ,  a  real¬ 
time  business  process/flow  modeling  tool 
(Scheer,  2000) ,  and  decision  support  or  intel¬ 
ligent  support  tools  that  can  be  modeled 
(Turban  et  al.,  2005),  which  are  not  neces¬ 
sarily  restricted  to  a  particular  purpose  or 
domain.  Due  to  some  domain-specific 
embedded  functions,  their  interfaces  (i.e.,  to 
other  software  applications,  to  developers 
and  designers,  and  to  their  growing  user 
base)  are  primarily  used  in  specific  domains 
and  so  would  be  less  efficient  for  other 
domains  or  purposes. 

•  Generic.  Tools  that  allow  learners  to  create, 
explore,  and  change  models  are  generic 
tools  when  they  are  not  restricted  to  mod¬ 
eling  one  domain  area.  Generic  tools  can 
accommodate  a  wide  spectrum  of  phenom¬ 
ena  and  systems. 

Modeling  Technologies  That  Combine  Software 
and  Hardware  (Robotic-Controlled  Devices ) 

The  majority  of  computer-based  modeling  technolo¬ 
gies  are  entirely  software  based  with  screen  output 
only.  Other  modeling  technologies,  based  on  the  con¬ 
structionist  paradigm,  build  on  the  idea  that  knowl¬ 
edge  construction  happens  particularly  well  when  the 
learner  constructs  something  concrete  (Papert,  1980). 
The  LEGO  Mindstorms  (ROBOLAB)  tools  (Knud- 
sen,  1999;  Lego  Group,  2003)  are  a  combination  of 
hardware  and  software  technologies  that  allow  learn¬ 
ers  to  program  robotic  devices  such  as  electronic 
hearts  and  dinosaurs  with  simple  motor-driven  fea¬ 
tures  (described  in  an  early  childhood  context  by  Bers 
et  al.,  2002;  in  a  wider  context,  by  Resnick,  1998). 
Other  robotic  devices  with  similar  technologies  were 
built  for  data  collection  and  control  of  a  greenhouse 
(see  the  spaceplanting  project  described  by  Milrad  et 
al.,  2003).  As  Thangiah  and  Joshi  (1997)  argued  in 
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their  paper  on  programming  robots  in  an  undergrad¬ 
uate  computer  science  class,  working  with  robotics 
and  the  conditions  under  which  the  robots  are  oper¬ 
ating  especially  “challenge  the  student  to  design  pro¬ 
grams  that  have  the  capability  to  deal  with  uncertain¬ 
ties”  (p.  224)  that  otherwise  would  be  difficult  to 
simulate.  So  far,  robotic  device  systems  are  almost 
exclusively  based  on  rules  and  have  been  used  to 
teach,  for  example,  the  creation  of  programming  lan¬ 
guage  to  formalize  these  rules  into  systems  (Gilder 
et  al.,  2003). 

Stand-Alone  vs.  Collaborative  Tool  vs.  Model 
Embedded  in  a  Larger  Learning  Environment 

There  is  a  substantial  literature  on  collaborative  or 
group  modeling  (see  Rouwette  et  al.,  2002,  for  a  meta¬ 
analysis  of  77  studies).  Most  software  applications  are 
stand-alone  modeling  tools,  giving  students  only  the 
option  to  collaborate  when  they  share  the  physical 
environment  in  front  of  the  computer  screen.  More 
recently,  model  building  with  runnable  visualizations 
has  been  addressed  in  computer- supported  collabora¬ 
tive  learning  environments  (Pinkwart  et  al.,  2001;  van 
Joolingen,  2000).  The  Collide  team  (Pinkwart  et  al., 
2001)  developed  CoolModes,  a  collaborative  learning 
tool  that  contains  a  system  modeling  palette  intended 
to  extend  the  notion  of  mindtools  to  collaborative 
mindtools  (Hoppe,  2001).  A  similar  approach  can  be 
found  in  the  collaborative  system  Co-Lab  (van  Joolin¬ 
gen  et  al.,  2005)  and  ModellingSpace  (Avouris  et  al., 
2003).  In  the  case  of  Co-Lab,  the  modeling  tool  is 
embedded  in  a  larger  learning  environment  that  also 
provides  remote  experiments,  contains  task-specific 
simulations,  and  additional  content  structures  (de  Jong 
and  van  Joolingen,  1998).  In  ModellingSpace,  the  sys¬ 
tem  contains  a  wide  variety  of  different  tools  and  uti¬ 
lizes  a  collaborative  space  metaphor  (Dimitracopoulou 
and  Komis,  2004). 

Embedded  Teaching  or  Learning 
Support  Structures  within  the  Tool 

Computerized  modeling  tools  have  a  steep  learning 
curve  (Alessi,  2000;  Mayer,  2004).  Although  many 
instructional  strategies  are  available  to  facilitate  mod¬ 
eling  activities,  some  modeling  tools  already  contain 
what  can  be  classified  as  hard  support  structures 
(adapting  a  term  from  the  literature  on  scaffolding). 
Brush  and  Saye  (2002)  distinguished  between  hard  and 
soft  scaffolds.  Soft  scaffolds  are  dynamic,  situation- 
specific  aids  provided  by  a  teacher  or  peer  to  support 
learning;  they  are  often  tailored  responses  to  individual 
difficulties  of  a  particular  learner.  Hard  scaffolds  are 


the  static  supports  designed  in  anticipation  of  typical 
obstacles  faced  by  learners.  Vmodel,  for  example,  con¬ 
tains  feedback  mechanisms  that  identify  when  struc¬ 
tural  elements  of  the  model  are  inconsistent  or  causal 
relationships  are  flawed.  Similarly,  STELLA  includes 
visual  feedback  to  indicate  violations  of  unit  consis¬ 
tency  or  tautologies.  The  story-telling  function  of 
STELLA  can  be  thought  of  as  an  embedded  support 
of  how  to  model  when  learning  to  model  via  modeling 
demonstrations.  The  thought  processes  including  the 
conceptual  and  procedural  knowledge  in  the  expert’s 
model  are  made  visible,  providing  the  learner  with  a 
task  overview  that  serves  as  an  advanced  organizer  for 
future  performance.  Additionally,  learners  can  anno¬ 
tate  their  own  models  to  provide  a  self-paced  walk¬ 
through  of  the  system  they  created. 

Different  Modeling  Languages 
or  Building  Blocks 

Typically,  a  modeling  technology  representational  sys¬ 
tem  is  textual,  graphical,  or  mixed  (e.g.,  STELLA 
stock- and-flow  diagrams  with  text  equations).  The 
graphical  representation  tends  to  be  more  intuitive, 
while  the  textual  representation  (e.g.,  typing  an  equa¬ 
tion  to  define  a  relationship)  tends  to  be  more  precise 
and  controlled.  Further,  the  overall  representation  may 
be  more  or  less  an  analog  of  the  original  (e.g.,  concrete 
vs.  abstract,  where  the  screen  shows  a  drawing  of  a 
lake,  river,  and  clouds  vs.  boxes  with  lake,  river,  or 
cloud  labels),  with  the  assumption  that  the  least 
abstract  is  easier  to  apprehend. 

Current  modeling  software  typically  has  a  graphi¬ 
cal  interface  and  describes  relations  in  the  model  either 
quantitatively  (i.e .,  with  an  equation;  high  precision ) 
or  qualitatively  (i.e.,  with  a  set  of  predetermined  visual 
notations;  intuitive).  This  is  important  because  differ¬ 
ent  representation  approaches  have  been  shown  to 
induce  very  different  reasoning  approaches  (Lohner  et 
al.,  2005). 

Real  systems  often  can  be  described  using  a  quan¬ 
titative,  axiomatic  basis— for  example  ,  by  using  equa¬ 
tions;  however,  “most  of  our  mental  models  and  verbal 
theories  ...  do  not  talk  about  numbers  and  numerical 
values,  but  rather  of  properties  which  are  categorical 
or,  at  best,  ordinal”  (Troitzsch,  2004,  para.  7).  If  so, 
then  an  important  part  of  model  building  is  translating 
qualitative  mental  models  into  quantitative  dynamic 
models.  The  representation  scheme  of  a  dynamic  mod¬ 
eling  system  is  constrained  by  the  requirements  of  the 
underlying  software  code  to  be  able  to  run  the 
model  — the  machine-understandable  ontology  of 
modeling  elements  — and  these  constraints  can  make 
the  modeling  system  more  difficult  to  use. 
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Programming  Languages 

Computer  programming  is  probably  the  most  abstract 
and  perhaps  most  flexible  approach  for  dynamic  mod¬ 
eling  with  technology.  The  screen  is  a  blank  slate,  and 
the  programmer  can  make  almost  anything  happen. 
Beyond  knowing  about  the  phenomenon  or  system 
being  modeled,  however,  good  modeling  through  com¬ 
puter  programming  requires  considerable  knowledge 
and  skill  of  the  programming  language  formalisms  and 
system  and  of  human-computer  interactions,  making 
this  approach  less  accessible  to  a  lay  audience.  Current 
modeling  programs  are  easier  to  use  than  text-based 
programming,  but  there  is  always  a  trade-off  between 
ease  of  use  and  depth  and  variety  of  functionality. 

Rule-Based  Languages 

A  more  accessible  text-based  computer  language  pro¬ 
gramming  approach  for  modeling  is  rule-based  pro¬ 
gramming.  Logo  (Papert,  1980)  and  StarLogo,  for 
example,  are  rule-based  programming  languages 
where  input  by  a  user  is  interpreted  using  a  set  of  rules 
and  it  is  possible  to  use  so-called  primitives,  self- 
defined  or  predefined  functions.  Through  if-then-else 
conditions,  rule -based  models  can  be  fairly  sophisti¬ 
cated;  although  mostly  used  with  agent-based  model¬ 
ing  tools,  they  are  perceived  as  not  being  sufficient  to 
represent  very  complex  problems  (Repenning  and 
Ioannidou,  2004).  To  remedy  the  shortcomings,  tools 
are  conceptualized  in  which  the  behavior  (a  response 
attribute  or  set  of  attributes)  adapts  to  the  context,  and 
the  tools  are  programmed  to  allow  emerging  modifi¬ 
cation  of  the  rules  based  on  self-measured  performance 
to  achieve  certain  goals,  which  become  models  that 
learn  (e.g.,  when  an  avatar  playing  soccer  in  a  virtual 
game  adapts  its  shooting  techniques  based  on  the  result 
of  prior  success  of  scoring  a  goal  or  passing  the  ball) . 

Summary 

Existing  tools  can  be  mapped  on  these  and  other 
dimensions;  however,  new  developments  in  other  areas 
such  as  social  computing  and  scientific  modeling  soft¬ 
ware  present  challenges  to  our  classification  and  to  our 
model  of  what  modeling  technologies  are.  To  sketch 
one  of  the  challenges,  one  could  argue  that  massively 
multiplayer  online  role-playing  games  (MMORPGs), 
such  as  Second  Life  with  its  1 .5  million  players,  could 
be  classified  as  black-box,  agent-based  modeling  tools 
in  which  the  dynamic  relationships  between  agents  are 
partially  visible  (and  can  never  be  made  fully  visible 
because  the  agents  are  human  beings)  but  personally 
controlled  agents  (avatars)  are  a  mix  of  player-direct- 
manipulated  and  rule-based  executed  actions  that  are 


based  on  the  affordances  of  the  context  in  which  the 
software  application  is  supporting  the  automatization 
and  mediation  of  the  actions,  and  provides  an  historical 
record  of  the  actions.  Because  this  classification 
approach  and  examples  were  derived  from  the  learning 
literature  ,  they  express  some  of  the  dimensions  related 
to  the  complexity  of  using  dynamic  modeling  in  the 
classroom.  What  are  some  specific  factors  that  have 
been  studied? 


DYNAMIC  MODELING 
IN  THE  CLASSROOM 

The  nature  and  complexity  of  dynamic  modeling  with 
technology  present  unique  problems  that  can  be  ame¬ 
liorated  or  exacerbated  by  the  modeling  technology. 
Hmelo-Silver  and  Azevedo  (2006)  noted  that  the  com¬ 
plexity  of  modeling  includes:  (1)  the  amount  of 
domain  content  to  be  modeled,  (2)  the  granularity  of 
the  model  depiction  required  by  the  modeling  task,  (3) 
the  level  of  abstractness  of  the  model  representation, 
(4)  the  affordances  available  to  the  students,  and  (5) 
how  difficult  it  is  for  students  to  perceive  and  to  under¬ 
stand  the  model’s  output.  These  issues  of  complexity 
likely  interact  with  students’  individual  differences, 
especially  their  beliefs  about  the  nature  of  knowledge. 
The  following  sections  look  at  these  issues  surround¬ 
ing  modeling  complexity  as  each  relates  to  modeling 
technologies. 

Complexity  and  Ill-Structuredness 
of  the  Modeling  Task 

Amount  of  Domain  Content 

How  much  does  the  modeling  technology  limit  the 
model’s  size?  One  aspect  of  modeling  is  determining 
where  to  start  and  end,  but  determining  the  model’s 
boundary  is  a  difficult  task  (Sterman,  2000).  How  do 
students  know  when  the  model  they  are  building  is 
done?  Is  the  model  done  when  the  screen  is  full  of 
stuff?  How  big  is  the  screen,  and  how  big  should  it  be? 
Building  small  models  seeks  a  pragmatic  middle 
ground.  Alessi  (2000)  advocated  modeling  small  sub¬ 
sets  of  an  original— for  example,  two  or  three  variables 
at  a  time.  As  in  traditional  instruction,  the  instructional 
designer  or  instructor  selects  the  content  subset  based 
on  topic,  task,  student  maturity,  and  so  on.  These 
smaller  models  can  reflect,  support,  and  refine  students’ 
mental  models  of  that  task  and  subset  and  later  may  be 
combined  into  larger  models  (Carney  et  al.,  2002)  but 
only  if  the  modeling  technology  supports  this. 
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Granularity  of  the  Model 

Granularity  is  a  form  of  abstraction  that  affects  com¬ 
plexity.  Level  of  aggregation  or  granularity  refers  to 
the  level  of  resolution  or  scale  of  a  model— for  exam¬ 
ple,  representing  heat  as  vibrating  molecules  in  motion 
vs.  as  a  number  on  a  thermometer  (Spector  et  al., 
2001).  Grain  size  is  a  critical  factor  in  dynamic  mod¬ 
eling,  so  a  modeling  technology  should  allow  repre¬ 
sentation  at  different  levels  of  scale  and  allow  the  user 
to  easily  combine  several  models  into  one  larger 
model;  however,  granularity  is  domain  dependent,  so 
the  modeling  technology  must  be  purpose-built  for 
such  model  building  and  representation  (rather  than 
generic). 

Level  of  Abstractness  of 
the  Model  Representation 

Dynamic  modeling  software  employs  structured  rep¬ 
resentational  formalisms  (Carney,  2002)  that  include 
the  way  objects  and  the  relationships  between  objects 
are  displayed  and  the  various  ways  output  is  depicted 
when  the  model  is  run.  Lohner  et  al.  (2003)  listed  four 
functions  of  a  structured  representational  system:  (1) 
support  or  extend  working  memory,  thereby  reducing 
cognitive  load  by  simultaneously  exhibiting  all  of  the 
entities  in  the  same  space;  (b)  provide  a  layout  of  the 
problem  space  to  include  the  relations  between  only 
the  most  necessary  subset  of  entities;  (3)  determine 
cognitive  behavior,  as  the  syntax  and  semantics  of  the 
representations  (what  we  call  grammar)  constrain 
some  aspects  and  stimulate  other  aspects  of  cognitive 
behavior  (see  Smith  and  Olkun’s,  2005,  concept  of 
spatial  priming );  and  (4)  provide  a  means  for  commu¬ 
nication  with  self  and  with  others. 

Modeling  technology  formalisms  strongly  influ¬ 
ence  what  is  learned  when  building  models.  Vmodel 
(a  visual  representation  language  based  on  concept 
maps)  was  designed  to  lower  the  typical  instructional 
overhead  necessary  to  use  dynamic  modeling  tools, 
referred  to  as  a  low  floor  approach.  The  underlying 
theory  of  Vmodel,  qualitative  process  theory  (Forbus, 
1984),  holds  that  process  is  central  for  explaining  cau¬ 
sality.  In  a  descriptive  study  with  seventh  graders,  Car¬ 
ney  (2002)  considered  one  aspect  of  the  grammar  of 
the  Vmodel  tool  called  the  process  entity.  Findings 
show  that  students  conflated  (mixed  up)  process  with 
other  elements  of  the  model— for  example,  labeling 
processes  as  things.  Also,  processes  were  seldom  used, 
and  most  processes  that  were  used  had  no  influence 
on  anything  else  in  the  model  (i.e.,  they  were  dead 
ends).  In  this  case  the  visual  representation  system  may 
have  led  students  to  view  process  entities  as  represen¬ 


tation  elements  to  show  structure  rather  than  function 
of  the  original  (Louca  et  al.,  2003).  In  other  words, 
every  moveable  thing  on  the  screen  became  an  object. 

Lohner  and  colleagues  (2003)  considered  the  influ¬ 
ence  of  graphical  vs.  text  modeling  representations 
(SimQuest)  on  the  modeling  process  and  product  by 
having  students  make  a  model  and  then  using  it  to 
generate  output  data  to  be  compared  to  data  from  a 
preexisting  hidden  expert  model.  The  authors  antici¬ 
pated  a  holistic  vs.  analytic  outcome;  that  is,  the 
graphic  group  would  produce  better  models  (i.e.,  as 
measured  by  congruence  to  the  hidden  original) 
because  they  could  trial  more  relations  in  the  time 
given,  whereas  the  text  group  could  trial  fewer  rela¬ 
tions  but  would  acquire  a  deeper  understanding  of 
those  relations.  As  expected,  the  graphical  groups  ran 
more  models  with  more  total  unique  relations  and  with 
more  average  relations  per  model.  The  text  group  spent 
a  lot  of  time  trying  to  create  equations.  The  graphical 
group  obtained  significantly  greater  modeling  scores 
compared  to  the  text  group.  Analyses  of  the  spoken 
interactions  revealed  that  the  graphical  and  textual 
groups  had  similar  spoken  reasoning,  and  the  model 
output  data  from  both  groups  were  used  only  mini¬ 
mally  for  testing  the  model.  The  authors  recommended 
that  these  different  representations  should  support  dif¬ 
ferent  phases  in  the  modeling  process.  Initially,  a 
graphical  approach  is  better  for  early  generalized  mod¬ 
eling,  and  then  a  text -based  approach  can  be  used  for 
refinement  of  the  general  models  — a  transition  from 
qualitative  to  quantitative  modeling  of  this  quantitative 
original. 

In  a  follow-up  investigation,  Lohner  and  col¬ 
leagues  (2005)  considered  the  influence  of  graphical 
vs.  text  modeling  representations  (SimQuest)  on  rea¬ 
soning  during  modeling.  As  in  the  previous  study,  the 
students  built  a  model  that  would  generate  output  data 
to  be  compared  with  data  from  a  preexisting  expert 
model.  As  in  the  earlier  study,  students  in  the  graphical 
condition  produced  better  models  than  those  in  the 
textual  condition.  Process  analysis  revealed  a  consid¬ 
erable  amount  of  off-task  behavior  by  some  groups  (up 
to  50%  of  the  time),  contrary  to  some  claims  regarding 
the  motivational  affects  of  modeling,  although  most 
groups  did  stay  on  task  more  than  90%  of  the  time. 
Students  in  the  graphical  modeling  treatment  spent 
significantly  more  time  making  evaluative  comments 
about  their  model,  and  to  a  lesser  degree  talking  about, 
planning,  and  carrying  out  the  next  iteration  of  their 
model.  Generally,  in  both  treatments,  the  inquiry  pro¬ 
cess  did  not  follow  a  systematic  temporal  order.  Also, 
in  both  treatments  there  seemed  to  be  a  strong  division 
or  even  chasm  between  building  the  model  and  using 
the  model  (i.e.,  to  test  the  model),  but  this  was  most 
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pronounced  in  the  graphical  treatment,  where  students 
repeated  an  orientation-hypothesis-build  cycle  with¬ 
out  testing  each  model  iteration.  For  both  text  and 
graphical  treatments,  students  did  most  experiments 
(i.e.,  using  the  model  to  test  the  model)  without 
expressing  a  hypothesis,  and  when  a  hypothesis  was 
used  it  appeared  out  of  the  blue,  especially  with  the 
text  groups. 

Louca  and  colleagues  (2003)  used  a  contextual 
inquiry  qualitative  analysis  approach  to  determine  how 
fifth  graders  communicated  ideas  while  modeling  rel¬ 
ative  motion  (which  is  a  quantitative,  axiomatic  orig¬ 
inal)  using  two  computer  modeling  tools:  Micro  Worlds 
and  Stagecast  Creator.  Micro  Worlds  models  are  built 
by  writing  a  formal  text-based  script  for  each  action 
(e.g.,  left,  forward  10,  wait  1),  while  Stagecast  Creator 
models  are  built  using  a  mouse-only,  click-and-drag 
approach  to  develop  visual  scripts  of  each  action.  Note 
that  the  students  used  the  software  for  ten  50-minute 
sessions  over  2  weeks.  The  Stagecast  Creator  dyads, 
relative  to  the  Micro  Worlds  dyads,  spent  more  time  in 
programming  and  program  planning  (both  visual  activ¬ 
ities)  and  less  time  debugging  but  the  same  amount  of 
time  in  testing  and  running  the  model.  Students  using 
the  Stagecast  Creator  visual  design  approach  focused 
more  on  holistically  depicting  relative  motion  through 
trial-and-error  iteration,  referred  to  as  explanatory  ad 
hoc  modeling  by  Horwitz  (1999);  students  using 
Micro  Worlds  were  forced  to  examine  the  model  prim¬ 
itives  and  functionally  code  each  object  (e.g.,  elegant 
programming  using  an  axiomatic  base)  (Horwitz, 
1999). 

More  remarkable  is  that  these  students  used  the 
modeling  system’s  grammar  to  explain  their  ideas  to 
each  other,  especially  with  Stagecast  Creator.  This  sug¬ 
gests  that  learners  internalize  more  than  just  domain 
concepts  (Louca  et  al.,  2003)  but  are  also  developing 
mental  patterns  of  the  modeling  system’s  grammar 
(i.e.,  its  ontology,  terminology,  and  syntax).  Internal¬ 
izing  the  modeling  system  grammar  may  have  pro¬ 
found  long-term  effects  (Carney,  2002).  Given  enough 
exposure  to  a  modeling  system,  acquiring  the  grammar 
of  the  software  is  a  likely  consequence  that  is  a  mea¬ 
surable  and  in  some  cases  worthwhile  outcome.  But, 
this  also  means  that  the  instructional  designer  must 
evaluate  the  modeling  software’s  grammar  to  deter¬ 
mining  if  it  is  worth  learning. 

Interface  of  the  Modeling  Technology 

Dynamic  modeling  software  allows  modelers  to  type, 
drag,  click,  or,  most  likely,  all  of  these.  Is  the  screen 
designed  so  these  actions  are  obvious  or  not?  Beyond 
interaction,  dynamic  modeling  systems  support  other 


kinds  of  engagement  and  actions;  for  example,  the 
screen  layout  can  invite  using  an  optimum  number  of 
components  on  the  screen  or  the  positioning  of  certain 
things  in  certain  areas  of  the  screen  (see  the  previous 
discussion  on  graphical  vs.  textual  layout). 

A  screen  affordance  also  includes  guiding,  con¬ 
straining,  and  determining  physical  action  and  cogni¬ 
tive  behavior  of  the  individual  when  using  the  model¬ 
ing  system  (Zhang,  1997)  and  includes  the  cognitive 
residue  that  results  from  using  it  (Salomon,  1993). 
Each  dynamic  modeling  tool  provides  different  screen 
layouts  and  interactivity,  so  this  begs  the  question  of 
how  much  does  a  modeling  affordance  matter?  Is  there 
an  important  mental  difference  when  using  one  layout 
vs.  another,  or  are  interactivity  and  layout  simply  a 
matter  of  ease  of  use,  convenience,  personal  choice, 
and  familiarity? 

Does  a  mind-body  connection  bind  screen  affor¬ 
dance  and  mental  representation?  Smith  and  Olkun 
(2005)  considered  the  affects  of  interactivity  on  mental 
rotation  ability.  Students  observed  objects  rotating  as 
a  function  of  interactively  dragging  around  it  vs.  just 
clicking  once  and  observing  an  animation  of  rotation. 
The  three  investigations  included  a  group  of  9-year- 
old  students,  a  second  group  of  9-year-olds,  and  a 
group  of  20-year-olds.  A  computer-based  post-test 
measured  the  accuracy  and  speed  of  the  students’  men¬ 
tal  rotation  ability.  In  all  three  investigations,  the  inter¬ 
active  treatment  group’s  mental  rotation  ability  was 
significantly  more  accurate  than  the  animation  group’s. 
In  terms  of  speed,  the  9-year-olds  in  the  interactive 
treatment  group  were  generally  faster  than  those  in  the 
animation  treatment  group;  however,  this  pattern  was 
reversed  for  the  20-year-olds,  for  whom  the  interactive 
group  was  significantly  slower  on  the  mental  rotation 
post-test  than  the  animation  group.  The  authors  sug¬ 
gested  that  the  9-year-old  participants  were  visually 
primed  by  the  lesson  interactivity  for  the  mental  rota¬ 
tion  post-test  while  the  20-year-old  participants  were 
disrupted  by  the  interactive  experience.  Apparently,  the 
lesson  interactivity  conflicted  with  the  older  students’ 
already  well-developed  mental  rotation  approach  (e.g., 
analytical  pattern  matching). 

Extending  this  point,  any  learner  may  have  a 
learned  bias  for  one  or  another  interaction  approach, 
and  a  mismatch  will  likely  affect  those  individuals  neg¬ 
atively.  A  possible  interactivity  effect  (Evans  and  Gib¬ 
bons,  2006)  should  be  exploited  in  modeling  system 
software.  Next-generation  input  devices  (e.g.,  the  Wii 
hand-held  remotes)  and  sensor  bars  that  detect  three- 
dimensional  hand  motion,  as  well  as  click- type  inter¬ 
actions  that  provide  tactile  (rumble)  and  audio  feedback 
through  the  input  device,  will  be  learned  and  then  sub¬ 
sumed  by  the  next  generation  of  learners  through  game 
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play  and  so  may  become  the  expected  way  of  interact¬ 
ing  with  modeling  technologies. 

Output  of  the  Model 

When  a  dynamic  model  is  run,  the  output  usually 
includes  some  combination  of  text,  numbers,  graphics, 
pictures  or  others  visuals,  animations,  videos,  and 
audio.  The  combinations  of  multiple  and  different  out¬ 
put  representations  mixed  in  with  the  model’s  repre¬ 
sentation  are  intended  to  complement  each  other;  how¬ 
ever,  learners  typically  are  not  very  good  at  attending 
to  appropriate  aspects  on  the  screen,  resulting  in  frag¬ 
mentary  and  disjointed  knowledge  structures  (Bode- 
mer  et  al.,  2004).  Further,  some  types  of  output  repre¬ 
sentations  are  more  difficult  than  others  for  a  novice 
to  understand  (e.g.,  vectors  indicating  force  and  direc¬ 
tion,  dot  patterns  indicating  speed,  graphs).  So, 
increasing  the  amount  and  complexity  of  the  output 
information  on  the  screen  increases  cognitive  load 
which,  at  some  point,  negatively  affects  understanding. 
What  is  the  optimal  model  output  format? 

Bodemer  and  colleagues  (2004)  conducted  two 
investigations  that  manipulated  the  cognitive  load  of 
visuals,  specifically  by  separating  the  pictorial  illus¬ 
tration  from  its  labels  (split  representation),  by  directly 
labeling  the  illustration  of  a  bicycle  pump,  or  by  having 
the  learner  interactively  drag  and  drop  the  labels  onto 
the  illustration.  Cognitive  load  was  categorized  as 
intrinsic  (the  complexity  of  the  specific  domain  and 
related  to  previous  familiarity),  extrinsic  (the  inter¬ 
face),  and  germane  (focus  only  on  most  important 
components).  The  experimental  treatments  were 
designed  to  both  decrease  extraneous  cognitive  load 
and  increase  germane  cognitive  load.  Their  findings 
suggest  that  an  integrated  screen  is  better  than  a  split 
screen  (experiment  1),  which  suggests  that  the  model 
representation  and  model  output  should  be  spatially 
contiguous  (Mayer,  1997).  Further,  in  both  experi¬ 
ments  1  and  2,  drag-and-drop  interactivity  was  better 
than  the  other  pictorial  conventions  used. 

Evans  and  Gibbons  (2006)  replicated  Bodemer  et 
al.’s  (2004)  investigation  on  the  affects  of  interactivity 
on  learning  and  coined  the  term  interactivity  effect  to 
refer  to  the  learning  benefits  gained  from  interacting 
with  visual  output  vs.  just  observing  the  output,  in  this 
case  on  a  transfer  application  test  but  not  on  the  process 
description  retention  post-test.  Rather  than  an  interac¬ 
tivity  effect,  it  can  be  argued  that  the  drag-and-drop 
interactivity  was  more  effective  than  the  non-interac¬ 
tive  treatment  in  this  case  because  it  forced  the  learners 
to  slow  down  and  pay  attention  to  each  individual 
component  of  the  visual,  relative  to  processing  the 
visual  as  a  whole  at  a  glance. 


If  there  is  an  interactivity  effect,  how  can  the 
learner  interact  with  the  model  output?  Typical  output 
of  a  dynamic  model  consists  of  constant  updating  of 
numerical  values  in  graphs  showing  quantity  change 
over  time,  or  values  increasing  and  decreasing  on 
model  components,  and  so  on.  Placing  charts  and 
graphs  contiguous  to  their  most  related  model  compo¬ 
nents  makes  sense.  The  most  recent  version  of 
STELLA  allows  the  learner  to  drag  and  drop  a  stock 
onto  a  graph  to  immediately  see  the  output,  a  clear 
application  of  the  interactivity  effect.  Also,  the  use  of 
highlighting  using  color,  size,  and  form  to  associate 
model  components  to  model  output  may  achieve  a 
suitable  level  of  attention  to  each  of  the  output  values. 

The  Role  of  Individual  Differences 

Does  individual  difference  influence  modeling?  If  so, 
what  role  does  the  modeling  technology  play?  Because 
of  the  cognitive  load  related  to  the  complexity  of 
dynamic  modeling,  higher  ability  students  may  be  able 
to  build  better  models  and  gain  more  from  modeling 
than  lower  ability  students  (Gijlers  and  de  Jong,  2005). 
Further  research  is  needed  in  this  area  of  individual 
differences. 

Naivete  or  lack  of  knowledge  and  experience  with 
the  domain  content  is  important  in  modeling  with  tech¬ 
nology.  Overcoming  commonly  held  but  erroneous 
preconceptions  is  an  issue  that  cuts  across  the  research 
on  using  models  for  learning,  especially  in  science 
education.  An  investigation  by  Tao  and  Gunstone 
(1999)  considered  the  affects  of  10  weeks  of  extensive 
use  of  models  (simulations)  in  physics  that  intention¬ 
ally  used  conceptual  conflict— that  is,  scientific  expla¬ 
nations  that  contradicted  typical  student  preconcep¬ 
tions.  They  reported  that  students  vacillated  between 
their  preconceptions  and  the  scientific  conceptions 
embodied  in  the  simulations  from  one  context  to 
another  during  instruction.  They  maintained  both  men¬ 
tal  conceptions  at  the  same  time  (of  two  minds  or 
vacillating)  and  relied  on  one  or  the  other  moment  to 
moment,  depending  on  context  cues. 

Learner  preconceptions  pose  an  even  greater  prob¬ 
lem  when  building  models.  Unguided  model  building 
is  likely  to  entrench  naive  conceptions  even  more  than 
using  models,  as  students  tend  to  build  their  naivete  into 
their  models  (Briggs,  1990).  For  this  reason,  the  amount 
and  type  of  guidance  to  provide  during  modeling  are 
important.  Some  modeling  technologies  provide  scaf¬ 
folds,  although  the  best  guidance  probably  comes  from 
the  learning  strategy,  peers,  and  the  instructor. 

Possibly  a  critical  factor  in  the  instructional  effec¬ 
tiveness  of  using  and  building  models  involves  the 
student’s  belief  about  the  nature,  source,  and  extent  of 
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knowledge.  Windschitl  and  Andre  (1998)  examined 
the  affects  of  epistemological  complexity  when  learn¬ 
ing  from  a  simulation  of  the  human  circulatory  system 
under  either  confirmatory  (guided  discovery)  or 
exploratory  (free  discovery)  conditions.  Among  other 
findings,  the  treatment  condition  interacted  signifi¬ 
cantly  with  epistemological  beliefs.  Individuals  with 
more  advanced  epistemological  beliefs  learned  rela¬ 
tively  more  in  the  exploratory  treatment  (as  measured 
by  a  multiple -choice  test),  while  individuals  with  less 
developmentally  advanced  beliefs  learned  more  in  the 
confirmatory  treatment.  Although  a  significant  body  of 
research  has  found  that  guided  discovery  is  superior 
to  free  discovery  when  learning  from  using  models 
(Bruner,  1961;  Mayer,  2004),  students’  beliefs  about 
the  nature  of  knowledge  may  be  a  critical  but  over¬ 
looked  variable  that  interacts  with  the  amount  of  guid¬ 
ance,  especially  when  building  models.  Also,  it  seems 
likely  that  extensive  experience  with  dynamic  model¬ 
ing  advances  a  learner’s  epistemological  complexity. 

Scaffolding  and  Guidance 
during  Model  Building 

Classroom  activities  that  surround,  direct,  and  support 
modeling  activity  are  a  critical  part  of  the  success  of 
modeling;  for  example,  guidance  can  include  guiding 
questions  and  clear  assignments,  online  tools  (Bera 
and  Liu,  2006),  and,  in  particular,  help  in  generating 
and  testing  hypothesis  (Bodemer  et  al.,  2004).  Also, 
overarching  design  principles  and  model-building 
planning  skills  can  be  learned  independently  of  the 
modeling  software  programming  formalism  (Fay  and 
Mayer,  1994).  These  issues,  however,  are  beyond  the 
scope  of  this  chapter  (see  Chapter  36  in  this  volume). 
Instead,  our  interest  centers  on  guidance  that  can  be 
provided  directly  or  indirectly  by  the  modeling  tech¬ 
nology. 

As  mentioned  earlier,  unguided  model  building 
may  entrench  naive  conceptions,  as  students  will  tend 
to  build  their  naive  ideas  into  the  model.  An  active 
coaching  support  system  may  provide  one  solution. 
Ureel  and  Carney  (2003)  determined  the  frequently 
asked  questions  that  arise  and  the  common  mistakes 
that  students  make  when  they  use  Vmodel,  and  they 
included  these  in  an  active  coaching  support  system. 
The  system  uses  a  rule-based  system  with  model  trac¬ 
ing  or  qualitative  simulation  to  detect  problems,  and 
when  a  problem  is  detected— for  example,  when  a 
model  element  is  used  incorrectly,  a  more  appropriate 
component  is  available,  or  if  there  is  a  vague  construc¬ 
tion-then  an  onscreen  avatar  looks  puzzled  or  frowns 
and  the  offending  component  of  the  model  is  high¬ 
lighted.  The  student  can  click  on  either  for  help.  To 


justify  this  level  of  model-building  guidance,  it  is  crit¬ 
ical  to  know  how  much  time  and  effort  are  required 
(cost  vs.  benefit)  to  build  unique  coaching  solutions 
for  the  universe  of  models  that  might  be  built. 

For  most  students,  understanding  the  model  repre¬ 
sentation  and  the  model’s  output  may  require  consid¬ 
erable  support,  depending  on  the  student’s  familiarity 
with  each  form;  for  example.  Smith  et  al.  (1992)  uti¬ 
lized  a  progression  of  simulations  (i.e..  Modeling  with 
Dots,  Archimedes,  and  Sink  the  Raft)  to  teach  sixth 
graders  the  principles  of  weight  and  density.  The 
instruction  began  with  interactions  with  real  objects  of 
various  weights  and  densities  and  then  progressed  to 
using  a  simulation  depicting  density  as  dots  in  same 
size  boxes,  then  to  a  simulation  of  floating  using  liq¬ 
uids  and  objects  using  the  same  dot-box  depiction  of 
density,  and  then  to  an  application  simulation  of  den¬ 
sity.  The  authors  noted  that  these  sixth  graders  devel¬ 
oped  concepts  that  have  been  shown  to  be  quite  diffi¬ 
cult  for  seventh  graders  and  are  usually  not  attempted 
until  ninth  grade. 

Simulations  (using  models)  have  the  same  issues 
regarding  output  complexity  as  does  building  models, 
but  with  a  lower  cognitive  overhead.  Perhaps  using 
simulations  can  serve  as  a  precursor  to  building  mod¬ 
els.  In  STELLA,  a  display  page  screen  called  the  inter¬ 
face  level  can  be  used  as  a  simulation  by  including 
output  graphs  and  variable  control  knobs,  while  the 
map/model  level  remains  hidden  behind  the  interface 
screen.  The  student  can  learn  the  output  formalisms 
and  important  variables  from  the  simulation  before 
trying  to  build  a  related  model . 

It  is  reasonable  to  have  students  build  a  progression 
of  models  beginning  with  foundational  ideas  and  pro¬ 
gressing  to  more  advanced  applications  and  interac¬ 
tions  of  these  principles;  however,  the  progression  of 
models  must  be  carefully  planned  and  integrated  and 
the  modeling  technology  must  be  able  to  represent  the 
various  levels  of  abstraction  required  (both  qualitative 
and  quantitative),  such  as  the  SimQuest  inquiry  mod¬ 
eler  (van  Joolingen,  2004). 


DESIGN  AND  RESEARCH  ISSUES 

Dynamic  modeling  has  positive  and  negative  aspects. 
Dynamic  modeling  is  time  consuming,  complex  at 
many  levels,  difficult,  may  be  especially  dependent  on 
students’  beliefs  about  knowledge,  is  unfamiliar  to 
most  teachers,  is  unlike  most  classroom  cultures,  and 
may  prematurely  entrench  inadequate  ideas.  On  the 
other  hand,  dynamic  modeling  is  unique  and  unlike  all 
other  instructional  approaches,  potentially  encourag¬ 
ing  authentic  learning.  Dynamic  modeling  certainly 
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provides  students  (and  teachers)  with  a  new  way  to 
look  at  things,  encouraging  unique  insight  while  estab¬ 
lishing  new  mental  competencies.  The  instructional 
effectiveness  of  building  models  is  not  yet  established, 
and  it  is  important  to  note  that  simulations  (using  mod¬ 
els)  are  not  very  instructionally  effective  (de  Jong  and 
van  Joolingen,  1998). 

Students  often  have  difficulty  with:  (1)  understand¬ 
ing  the  effects  of  nonlinear  relationships,  (2)  keeping 
the  entire  system  original  in  mind  when  working  on  a 
component  of  the  original,  (3)  appreciating  (holisti¬ 
cally)  the  full  range  of  control  and  influence  possible 
within  a  complex  system,  and  (4)  transferring  princi¬ 
ples  learned  in  one  context  to  a  different  situation 
(Milrad  et  al.,  2002;  van  Eck  and  Dempsey,  2002). 
Add  to  this  list  the  tendency  to  overgeneralize  from  a 
few  examples  (Feldman,  2003)  or,  put  differently,  the 
tendency  to  generalize  from  just  a  few  examples 
(Tenenbaum,  1999);  an  inability  to  prioritize  the  func¬ 
tionally  important  components  from  the  universe  of 
possible  model  components,  as  evidenced  by  using 
fewer  components  and  more  irrelevant  components 
(Spector  et  al.,  2001);  an  inability  to  associate  deci¬ 
sions  with  outcomes  when  the  outcome  is  delayed;  and 
perceptual  myopia,  where  stubborn  preconceptions 
veil  or  distort  observations  of  the  actual  model  output 
(Gunstone  and  Watts,  1985).  Good  model  building 
forces  learners  to  experience  these  issues  but  can  serve 
as  a  mindtool  to  support  areas  of  weakness. 

Good  modeling  with  technology,  then,  is  about  a 
distinctive  kind  of  conceptual  change  in  the  modelers 
that  requires  attention  to  different  kinds  and  levels  of 
detail.  Young  or  novice  modelers  often  maintain  a  sim¬ 
ple  copy  epistemology  (Barowy  and  Roberts,  1999), 
holding  the  view  that  models  are  little  more  than  mini 
copies  of  the  original.  A  model  can  show  surface  fea¬ 
tures  (descriptive),  it  can  show  function  and  behavior 
(i.e.,  relations  and  process;  explanatory),  and  it  can 
represent  conceptual  deep  structure  (i.e.,  structural 
knowledge;  explanatory).  Dynamic  modeling  requires 
managing  the  interconnections  between  a  variety  of 
structural  and  functional  parts,  and  it  especially 
depends  on  functional  relations  (Seel,  2003),  but 
young  students  and  novices  will  tend  to  represent 
visual  structural  features  and  confuse  these  with  deeper 
functional  relations  (Hmelo  et  al.,  2000). 

Building  and  using  models  likely  affects  a  broad 
range  of  learning  outcomes  not  ordinarily  measured 
by  traditional  measures  of  domain  knowledge;  for 
example,  de  Jong  et  al.  (1999),  in  an  investigation  of 
several  guided  discovery  approaches  with  a  simulation, 
used  three  measures  of  learning:  definitional  knowl¬ 
edge  of  facts  and  concepts,  speeded  test  of  intuitive 
knowledge  where  both  correctness  and  answer  time 


were  measured,  and  a  concept  sorting  task  measure  of 
structural  knowledge.  No  difference  was  found  in  def¬ 
initional  knowledge  and  structural  knowledge  mea¬ 
sures  for  the  three  treatments,  nor  a  difference  in  intu¬ 
itive  knowledge  correctness  for  the  three  treatments; 
however,  a  significant  difference  was  found  for  the 
“most  guided”  of  the  three  for  speed,  as  well  as  sig¬ 
nificant  improvement  in  both  correctness  and  speed 
from  pretest  to  post-test.  Future  modeling  research 
must  focus  on  determining  optimal  modeling 
approaches  for  different  learning  outcomes. 

Many  science  educators  see  modeling  as  funda¬ 
mental  to  science  and  of  great  utility  to  scientists  (For- 
bus  et  al.,  2001;  Jackson,  2006).  Schwarz  and  White 
(2005)  argued  that  learning  about  the  nature  and  utility 
of  scientific  models  and  engaging  students  in  the  pro¬ 
cess  of  creating  and  testing  models  (called  model- 
based  inquiry  or  meta-modeling)  should  be  a  central 
focus  of  all  science  education.  To  support  this,  they 
developed  a  tool  called  Model-Enhanced  Thinker 
Tools.  In  their  pilot  investigation  in  four  seventh-grade 
classrooms  in  San  Francisco  over  a  three-month 
period,  students  gained  physics  knowledge,  inquiry 
skills,  scientific  beliefs,  and  modeling  knowledge.  This 
was  especially  true  for  the  high-ability  students. 

Place-based  computing  may  become  an  important 
niche  in  dynamic  modeling.  Colella  (2000)  considered 
an  approach  called  immersive  dynamic  modeling, 
where  tenth-grade  students  acted  as  dynamic  entities 
within  a  model  of  disease  transmission  by  using  wear¬ 
able  computers  called  thinking  tags  during  a  3-week- 
long  project.  The  students  showed  a  persistent  willing¬ 
ness  to  suspend  their  disbelief  and  behave  as  though 
the  simulation  activity  was  real  and  exhibited  a 
remarkable  level  of  ownership  for  the  proposed  exper¬ 
iments.  Such  place-based  dynamic  modeling  may 
apply  to  some  content  areas  that  are  not  amenable  to 
traditional  screen-based  modeling  approaches  due  to 
requirements  of  actual  social  interaction. 

It  is  increasingly  common  to  use  powerful  models 
in  everyday  work  settings.  This  begs  some  important 
questions:  What  is  the  optimal  way  to  learn  an  under¬ 
lying  model  used  in  the  workplace?  Are  weather-pre¬ 
dicting  models  objects  to  think  with  (Papert,  1980)? 
At  what  level  does  the  weatherman  need  to  understand 
the  underlying  model?— “Trying  to  learn  meteorology 
from  such  a  model  would  be  like  taking  chess  lessons 
from  Deep  Blue”  (Horwitz,  1999,  p.  181).  The  role  of 
big  models  in  learning  is  an  open  question. 

As  noted,  the  amount  and  kind  of  guidance  pro¬ 
vided  during  modeling  are  critically  important 
research  areas;  however,  further  developments  in  the 
design  of  modeling  software  will  occur.  Strobel  and 
Gottdenker  (2002),  for  example,  argued  for  tools  that 
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bridge  static  and  dynamic  modeling  so  variables, 
attributes,  and  relationships  can  be  transformed  with  a 
click  from,  say,  a  concept  map  into  to  a  dynamic  sys¬ 
tem  model. 

In  our  opinion,  one  of  the  biggest  unsolved  prob¬ 
lems  in  software  development  for  distance  or  com¬ 
puter-supported  collaborative  learning  is  how  to  design 
pointing  devices  and  features  that  allow  the  user  to 
easily  show  another  user  aspects  on  the  screen  at  a 
distance.  Workplace  and  application  sharing  is  still  in 
its  infancy. 

Even  though  modeling  with  technology  has  been 
around  for  awhile  ,  the  recent  availability  of  many  new 
easier-to-use  software  modeling  programs  (and  mod¬ 
eling  system  conferences)  indicates  a  strong  and  grow¬ 
ing  interest  in  dynamic  modeling.  The  empirical  and 
theoretical  basis  for  modeling,  however,  still  requires 
substantial  additional  work. 
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ABSTRACT 

The  learning  objects  literature  is  a  collection  of  journal 
articles,  book  chapters,  white  papers,  and  blog  entries 
that  as  a  whole  recognize  few  seminal  works,  share 
few  common  definitions  of  terms,  and  rarely  reference 
or  build  upon  one  another.  Learning  objects  research 
generally  falls  into  one  of  two  categories.  The  tradi¬ 
tional  approach  to  using  learning  objects  focuses  on 


enabling  the  just-in-time  automated  assembly  of  care¬ 
fully  structured  learning  objects  to  create  personalized 
educational  experiences.  The  permissive  approach  to 
using  learning  objects  focuses  on  making  the  reuse  and 
localization  of  all  resources,  regardless  of  their  struc¬ 
ture,  as  effective  and  efficient  as  possible.  The  field  is 
subject  to  a  large  number  of  criticisms.  Nascent  work 
in  open  educational  resources  points  to  the  likely  future 
of  the  field. 


*  The  references  used  in  this  chapter,  together  with  many  that  were  collected  but  not  used,  are  available  online  at  http://www.citeulike.org/user/open- 
content;  all  websites  referenced  in  this  chapter  were  available  February  1,  2007. 
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KEYWORDS 

Learning  object:  A  digital  resource  that  can  be  reused 
to  mediate  learning. 

Open  educational  resource:  A  learning  object  that  can 
be  freely  used,  reused,  adapted,  and  shared. 


INTRODUCTION 

The  learning  objects  literature  is  a  collection  of  journal 
articles,  book  chapters,  white  papers,  and  blog  entries 
that  as  a  whole  recognize  few  seminal  works,  share 
few  common  definitions  of  terms,  and  rarely  reference 
or  build  upon  one  another.  This  lack  of  structure  in  the 
research  makes  it  difficult  to  write  an  overview  of  the 
learning  objects  research  literature  .  Rather  than  telling 
a  compelling  story  that  moves  clearly  and  purposefully 
in  a  handful  of  directions,  the  best  an  author  can  do  is 
cluster  otherwise  disconnected  pieces  by  their  under¬ 
lying  philosophies .  This  places  an  unusual  amount  of 
responsibility  on  the  author.  I  have  chosen  the  cluster¬ 
ing  of  topics  I  felt  most  important,  but  I  understand 
that  others  will  have  different  views  of  what  matters 
most. 


HISTORICAL  OVERVIEW 

The  idea  of  reusing  digital  educational  resources  is 
almost  as  old  as  the  computer  itself.  As  early  as  the 
1960s,  researchers  were  describing  how  “curricular 
units  can  be  made  smaller  and  combined,  like  stan¬ 
dardized  Meccano  [mechanical  building  set]  parts, into 
a  great  variety  of  particular  programs  custom-made  for 
each  learner”  (Gibbons  et  al.,  2002,  p.  28).  Although 
the  general  notion  of  a  learning  object  is  at  least  40 
years  old  now,  it  was  Ted  Nelson  (who  coined  the  term 
hypertext )  who  developed  the  conceptual  foundations 
of  learning  objects  and  modern  content  reuse  in  the 
description  of  his  Xanadu  (Nelson,  1982)  and  OSMIC 
(Nelson,  1996)  systems  beginning  in  the  early  1960s. 
Nelson’s  work  conceived  and  grapples  with  most  of 
the  major  issues  still  facing  learning  object  designers 
and  reusers  today. 

The  Xanadu  design,  which  describes  Nelson’s 
ideal  hypertext  system,  calls  for  all  content  to  be 
archived  in  a  fixed,  uneditable  manner.  Whenever  a 
user  desires  to  make  changes  to  a  piece  of  content 
previously  stored  in  the  system,  those  changes  are 
stored  separately,  and  users  have  ongoing  access  to 
both  versions  of  the  document.  The  modern  Connex¬ 
ions  system  developed  at  Rice  University  uses  a  similar 
system  (see  http://cnx.org/). 


Because  a  specific  version  or  historical  view  of  a 
specific  document  is  guaranteed  to  exist  in  a  specific 
location  in  perpetuity,  it  is  possible  to  reuse  portions 
of  documents  in  Xanadu  by  reference;  for  example, 
suppose  an  author  wants  to  quote  a  portion  of  an  exist¬ 
ing  document  in  a  new  document.  Instead  of  cutting 
and  pasting  the  text  into  the  document,  the  author  could 
reference  the  specific  starting  and  stopping  locations 
in  the  existing  document,  and  the  content  from  that 
existing  document  will  be  rendered  dynamically  in  the 
new  document  whenever  the  new  document  is  ren¬ 
dered.  This  functionality  is  currently  available  as  the 
open-source  Xanadu  Transquoter  (see  http: //transliter¬ 
ature  .org/transquoter/) . 

Issues  of  granularity  and  context  that  plague  cur¬ 
rent  designers  and  reusers  of  learning  objects  are  com¬ 
pletely  and  elegantly  sidestepped.  Rather  than  requir¬ 
ing  authors  to  design  and  build  content  with  future 
reuse  in  mind,  breaking  their  content  into  chunks,  for 
example,  as  in  the  Xanadu  approach,  authors  simply 
create  and  publish  their  content  as  they  see  fit.  Other 
authors  who  desire  to  reuse  portions  of  the  content 
later  on  simply  indicate  the  section  of  the  existing 
document  they  wish  to  reuse,  and  this  section  is  ren¬ 
dered  dynamically  within  the  new  document  later. 
Also,  issues  of  context  of  learning  objects  are  also 
completely  avoided,  as  readers  of  the  new  document 
can  always  navigate  back  to  the  original  document 
from  which  the  snippet  came  to  better  understand  the 
context  of  the  learning  object.  (This  functionality  is 
currently  approximated  in  the  Purple  system;  see 
http://www.eekim.com/software/purple/purple.html.) 

Nelson  created  a  vocabulary  and  catalog  of  con¬ 
cepts  and  approaches  relating  to  what  would  come  to 
be  called  learning  objects.  Terms  such  as  primedia 
(describing  the  primitive  or  primordial  media  bits  that 
are  reused  within  the  system)  and  transclusion 
(describing  the  way  in  which  primedia  from  one  doc¬ 
ument  are  dynamically  included  in  another)  hold  huge 
conceptual  value  for  anyone  desiring  to  learn  more 
about  the  idea  of  reusable  digital  content.  Unfortu¬ 
nately,  these  terms  and  concepts  are  all  but  absent  from 
the  current  learning  objects  literature  because  the 
father  of  the  modem  concept  grounded  his  thinking  in 
another  literature. 

With  the  emergence  of  the  World  Wide  Web  in  the 
early  1990s,  the  idea  of  reusable  materials  came  to  the 
forefront  once  more.  In  1994,  Wayne  Hodgins  coined 
these  things  learning  objects  (Hodgins,  2002).  Perhaps 
because  Nelson’s  Xanadu  project  had  stalled  and  been 
largely  forgotten  by  the  time  Hodgins  was  writing, 
Hodgins  and  other  early  writers  (see,  for  example, 
Downes,  2000)  located  the  conceptual  roots  of  the 
learning  objects  approach  in  the  reuse  literature  of 
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object-oriented  programming.  This  led  researchers  to 
say  things  like  the  following  (Femandez-Manjon  and 
Sancho,  2002,  p.  6): 

The  idea  behind  learning  objects  is  clearly  grounded  in 
the  object-oriented  paradigm:  independent  pieces  of 
instruction  that  may  be  reused  in  multiple  learning  con¬ 
texts  and  that  fulfil  [s/cl  the  principles  of  encapsulation, 
abstraction  and  inheritance. 

These  statements  place  the  development  of  learn¬ 
ing  objects  within  a  computer  science  paradigm,  ask¬ 
ing  instructional  designers  to  speak  about  things  in 
terms  of  encapsulation,  abstraction,  inheritance,  and 
polymorphism  (Morris,  2005).  The  popular  connection 
of  learning  objects  to  software  engineering  has  created 
a  noticeably  technical  emphasis  in  the  research.  Had 
the  learning  objects  notion  been  connected  to  Nelson’s 
work  instead  of  object-oriented  programming,  the 
research  may  well  have  been  more  focused  on  new 
media,  creative  writing,  technical  writing,  and  other 
fields  more  closely  related  to  instructional  design  than 
computer  science.  This  review  of  research  is  largely 
structured  around  this  tension  between  differing  views 
of  the  conceptual  lineage  and  future  trajectory  of  learn¬ 
ing  objects  research. 

COMPETING  DEFINITIONS 
AND  RELATED  TERMS 

The  learning  objects  notion  is  confusing  in  part 
because  there  are  dozens  of  definitions  of  the  term 
learning  object  (LO) ,  as  well  as  several  phrases  refer¬ 
ring  to  the  same  notion  of  reusable  digital  educational 
resources.  The  most  frequently  cited  definition  of 
learning  objects,  and  the  most  all-inclusive,  is  that  put 
forth  by  the  Institute  of  Electrical  and  Electronics 
Engineers’  Learning  Technology  Standards  Commit¬ 
tee  (IEEE,  2005):  “Learning  Objects  are  defined  here 
as  any  entity,  digital  or  non-digital,  which  can  be  used, 
re-used  or  referenced  during  technology  supported 
learning.  ...Examples  of  Learning  Objects  include: 

•  Multimedia  content 

•  Instructional  content 

•  Learning  objectives 

•  Instructional  software  and  software  tools 

•  Persons,  organizations,  or  events  referenced 
during  technology  supported  learning” 

The  reaction  against  this  extremely  broad  definition 
has  been  very  strong.  Wiley  struggled  to  constrain  the 
definition  somewhat  with  “any  digital  resource  that  can 
be  reused  to  support  learning”  (Wiley,  2000a,  p.  23), 


but  even  this  is  still  very  broad.  More  colorfully,  people 
have  reacted  by  writing  pieces  such  as  My  Left  Big  Toe 
Is  a  Learning  Object  (Levine,  2004)  and  Urinal  as  a 
Learning  Object  (Leinonen,  2005). 

The  problems  created  by  so  broad  a  definition  are 
compounded  by  the  sheer  number  of  more  specific 
definitions  that  appear  in  the  literature.  Indeed,  almost 
every  article  written  about  learning  objects  provides 
its  own  unique  definition  of  the  term;  for  example,  in 
reviewing  papers  from  the  2004  ICALT  conference, 
Rossano  et  al.  (2005)  found  four  separate  definitions 
used  within  that  single  conference. 

Whereas  there  are  literally  dozens  of  published 
definitions  of  learning  object ,  a  number  of  slightly 
different  terms  have  similar  meanings.  Merrill  (1998, 
p.  2)  prefers  the  term  knowledge  object  and  defines  it 
as  something  similar  to  a  database  schema:  “A  knowl¬ 
edge  object  consists  of  a  set  of  fields  (containers)  for 
the  components  of  knowledge  required  to  implement 
a  variety  of  instructional  strategies.”  Gibbons  et  al. 
(2002,  p.  27)  prefer  the  term  instructional  object, 
describing  it  as  any  element  "that  can  be  independently 
drawn  into  a  momentary  assembly  in  order  to  create 
an  instructional  event.”  Other  commonly  used  terms 
that  describe  similar  concepts  include  the  Department 
of  Defense  Advanced  Distributed  Learning  Initiative’s 
sharable  content  objects  (ADL,  2004),  Hannafin  et 
al.’s  (2002)  resources,  and  Downes’  different  use  of 
the  term  resources  (Downes,  2004).  Friesen  (2004) 
decried  the  whole  state  of  affairs  regarding  learning 
objects  definitions  and  brought  the  discussion  back  to 
the  ground  by  reminding  us  that,  “innovations  must  be 
presented  in  terms  that  are  meaningful  for  teaching 
practice.” 


GUIDING  METAPHORS 

Learning  objects  researchers  have  used  a  variety  of 
metaphors  to  describe  learning  objects  and  their  appro¬ 
priate  use.  The  most  common  metaphors  — including 
LEGOs,  molecules,  and  bricks  and  mortar— provide 
an  extremely  interesting  view  into  individuals’  under¬ 
lying  beliefs  about  teaching  and  learning.  The  LEGO 
metaphor  characterizes  learning  objects  as  small 
chunks  of  content  which,  through  their  adherence  to 
standards,  are  each  able  to  be  combined  with  every 
other  in  a  straightforward  manner.  This  was  the  first 
popular  metaphor,  stressing  ease  of  reuse,  and  was 
conceived  by  Hodgins  (2002). 

The  molecular  metaphor  characterizes  learning 
objects  as  small  chunks  of  content  which,  according 
to  their  semantic  and  structural  makeup,  have  stronger 
affinities  for  binding  with  some  learning  objects  and 
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weaker  affinities  for  binding  with  others.  This  meta¬ 
phor  stresses  the  role  of  learning  objects’  contexts, 
emphasizing  that  not  every  object  can  fruitfully  be 
combined  with  every  other.  This  metaphor  was  also 
popular,  showing  up  in  Wiley  (1999)  and  Norman 
(2004).  Mejias  (2003)  applied  for  a  patent  relating  to 
what  he  called  learning  molecules. 

The  brick- and-mortar  metaphor  characterizes  learn¬ 
ing  objects  as  small  chunks  of  content  which,  being  a 
variety  of  shapes  and  sizes,  are  difficult  to  assemble  in 
a  meaningful  way  without  some  kind  of  contextual  glue 
to  hold  them  together  and  give  the  aggregation  mean¬ 
ing.  This  metaphor  stresses  that  learning  objects  are 
“bricks  held  together  and  made  meaningful  by  a  con¬ 
textual  mortar”  and  receives  treatment  in  Wiley’s 
(2005)  discussion  of  learning  object  metaphors. 

An  important  thing  to  note  from  this  progression  of 
metaphors  is  not  that  each  one  places  more  emphasis 
on  the  important  role  of  context  in  the  meaning  making 
and  learning  process.  The  most  important  thing  to 
notice  is  that  each  metaphor  assumes  that  a  learning 
object  is  a  closed,  uneditable  unit— but  unlike  the  pri- 
media  of  Nelson’s  Xanadu,  there  is  no  mechanism  for 
creating  alternate  versions  of  the  objects.  The  prevailing 
metaphors  make  the  assumption  that  learning  objects 
can  be  aggregated  but  not  adapted.  This  frequently 
unspoken  assumption  is  discussed  further  below. 

DEGREES  OF  SPECIFICATION 

Unsurprisingly,  a  wide  variety  of  opinion  exists  regard¬ 
ing  how  learning  objects  should  be  structured  inter¬ 
nally,  marked  up  for  search,  and  reused  in  the  context 
of  learning.  These  differences  of  opinion  are  best  char¬ 
acterized  as  lying  along  a  continuum  of  specification. 
Some  instructional  design  approaches  demand  adher¬ 
ence  to  exacting  standards  for  the  way  learning  objects 
are  structured  and  tagged  with  metadata  (e.g.,  the  Shar- 
able  Content  Object  Reference  Model,  or  SCORM), 
while  others  completely  reject  the  notion  that  such 
decisions  can  be  made  ahead  of  time  and  forced  upon 
the  instructional  design  community.  Both  intellectual 
camps  have  important  arguments  to  make  in  favor  of 
their  approach.  Both  approaches  suffer  from  signifi¬ 
cant  drawbacks  as  well. 

Highly  Specified  Approaches 
to  Using  Learning  Objects 

Merrill’s  (1999)  instructional  transaction  theory  pro¬ 
vides  a  significant  amount  of  instructional  functionality 
to  users  of  conforming  systems.  At  a  high  level,  a  sys¬ 
tem  implementing  instructional  transaction  theory  is  a 


simulation  environment  with  embedded  facilities  to 
both  prompt  learners  to  practice  specific  tasks  and  give 
personalized,  intelligent  feedback  based  on  learner  per¬ 
formance  in  the  environment.  All  system  content  is 
represented  as  knowledge  objects  in  a  database,  and 
system  software  operates  on  these  knowledge  objects 
to  render  the  simulation,  cue  and  respond  to  user  actions 
in  the  simulation,  and  create  and  deliver  feedback. 

A  system  implementing  instructional  transactions 
is  able  to  make  such  sophisticated  use  of  knowledge 
objects  because  the  format  and  structure  of  these  data 
have  been  very  precisely  specified  ahead  of  time,  and 
system  algorithms  can  depend  on  finding  content  struc¬ 
tured  and  marked  up  in  specified  ways.  Sophisticated, 
automated  reuse  is  the  strength  of  instructional  trans¬ 
action  theory  and  other  approaches  that  specify  the 
manner  in  which  learning  objects  should  be  structured 
and  marked  up.  For  additional  highly  specified 
approaches  to  using  learning  objects,  see  O’Keeffe  et 
al.  (2006),  Duitama  et  al.  (2005),  and  Colucci  et  al. 
(2005).  The  weakness  of  these  approaches  is  that  no 
learning  objects  occur  “naturally”  in  the  highly  struc¬ 
tured  manner  they  specified— each  existing  chunk  of 
content  must  be  (paradoxically)  specially  prepared 
before  it  can  be  reused. 

Less  Specified  Approaches 
to  Using  Learning  Objects 

On  the  other  hand,  Wiley  and  colleagues’  (2004)  02 
model  of  using  learning  objects  requires  much  less 
from  the  learning  objects  in  terms  of  their  structure. 
The  02  model  centers  on  a  sequence  of  increasingly 
difficult  problems  to  be  solved  by  learners.  Instructional 
designers  locate  learning  objects  to  be  used  by  students 
and  present  them  (perhaps  with  recommendations  for 
the  sequence  in  which  learners  should  engage  the  mate¬ 
rials)  in  the  context  of  the  problem  whose  solution  they 
support.  Extending  the  brick-and -mortar  metaphor, 
learning  objects  are  the  bricks  of  this  design  model, 
and  the  problem  statement  (and  optional  learning  object 
sequencing  information)  is  the  contextual  mortar  that 
gives  the  individual  learning  objects  meaning. 

The  publicly  accessible  Internet  contains  literally 
thousands  of  terabytes  of  digital  materials  that  can  be 
reused  to  support  learning  without  reformatting  by 
models  such  as  02.  The  structure  of  this  material  varies 
greatly  — some  is  video,  some  is  audio,  some  is  PDF 
(portable  document  format),  some  is  HTML  (hypertext 
markup  language),  and  so  on— but,  because  humans 
carry  out  the  aggregation  of  these  resources,  the  het¬ 
erogeneous  nature  of  the  resources  is  not  problematic. 
These  resources  may  be  amenable  to  semiautomated 
reuse  via  what  Spector  (1999)  called  weak  intelligent 
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design  support  systems— systems  intended  to  supple¬ 
ment  what  a  human  does  (whereas  strong  systems  are 
intended  to  replace  what  a  human  does). 

Immediate  reuse  is  the  strength  of  instructional 
design  approaches  that  can  use  any  kind  digital 
resource.  The  weakness  of  these  approaches  is  that 
only  the  most  rudimentary,  unsophisticated  reuse  of 
materials  can  be  automated.  Reuse  in  the  context  of 
less  specified  approaches  almost  always  requires  the 
involvement  of  human  instructional  designers  (Wiley, 
2000b). 

Highly  Specified  Approaches  to  Cataloging 
and  Finding  Learning  Objects 

Several  technical  specifications  exist  that  describe  the 
types  of  metadata  that  should  be  collected  to  enable 
the  discovery  of  learning  objects.  As  with  the  confu¬ 
sion  regarding  learning  objects  themselves,  a  large 
number  of  specifications  or  standards  specify  learning 
objects  metadata.  Metadata  are  data  about  data.  A 
metadata  record  captures  title,  author,  publication  date, 
and  other  information  to  help  people  find  learning 
objects ,  much  as  the  cards  in  a  card  catalog  help  library 
patrons  find  books. 

The  five  most  important  metadata  standards  or 
specifications  for  researchers  of  learning  objects  to 
know  are  the  IEEE  Learning  Object  Metadata  standard 
(IEEE,  2005),  the  IMS  Learning  Resource  Meta-Data 
Specification  (IMS,  2006),  the  ARIADNE  Educational 
Metadata  Recommendation  (ARIADNE,  1998),  the 
Dublin  Core  metadata  standard  (DCMI,  2006),  and  the 
SCORM  metadata  specification  (ADL,  2004).  Briefly, 
the  Dublin  Core,  IMS,  and  ARIADNE  projects  began 
their  lives  working  independently,  developing  separate 
specifications  for  what  metadata  should  be  captured 
and  how  the  data  should  be  expressed.  Eventually,  the 
IMS  and  ARIADNE  projects  found  out  about  each 
other  (the  IMS  project  was  running  in  the  United  States 
and  the  ARIADNE  project  was  running  in  the  Euro¬ 
pean  Union),  and  it  was  agreed  that  they  should  har¬ 
monize  their  efforts  under  the  auspices  of  a  true  inter¬ 
national  standards  organization  and  guarantee  that 
their  work  would  interoperate.  The  IEEE  Learning 
Object  Metadata  (LOM)  Standard  is  the  context  in 
which  they  chose  to  carry  out  this  work.  Dublin  Core 
later  agreed  to  participate  under  similar  terms.  The 
ADL’s  SCORM  work  inherits  the  benefits  of  these 
interoperability  agreements  because  SCORM  is  abest- 
of-breed  package  of  existing  specifications. 

Practical  efforts  to  create  interoperability  between 
these  varying  specifications  and  standards  have  been 
quite  successful.  Najjar  and  colleagues  (2003) 
described  how  they  transformed  ARIADNE  metadata 


into  LOM  metadata  using  XSLT.  The  authors  are  quick 
to  point  out  that  the  work  was  not  trivial.  IMS  (2006) 
has  also  released  a  best  practice  guide  for  using  XSLT 
to  transform  IMS  Learning  Resource  Meta-Data  into 
IEEE  LOM. 

Given  its  status  as  an  internationally  recognized, 
accredited  technical  standard,  the  IEEE  Learning 
Object  Metadata  standard  is  emerging  as  the  primary 
standard.  Perhaps  more  than  any  other  metadata  spec¬ 
ification  or  standard,  LOM  belongs  in  a  section  on 
highly  specified  approaches.  The  LOM  includes  doz¬ 
ens  of  elements,  many  with  their  own  controlled  vocab¬ 
ularies.  As  an  example,  when  describing  the  learning 
resource  type  of  a  learning  object,  creators  of  metadata 
must  choose  one  of  the  following  values:  exercise, 
simulation,  questionnaire,  diagram,  figure,  graph, 
index,  slide,  table,  narrative  text,  exam,  experiment, 
problem  statement,  self-assessment,  or  lecture.  In  addi¬ 
tion  to  using  the  list  of  elements  and  their  possible 
values  correctly,  implementers  of  the  LOM  will  also 
have  to  conform  to  the  lengthy  54-page  XML  binding. 

In  addition  to  formal  specifications  and  standards , 
several  published  research  studies  also  describe 
highly  specified  approaches  to  cataloging  learning 
objects  using  novel  ontologies  (see,  for  example,  Qin 
and  Hernandez,  2006).  As  with  highly  specified 
approaches  to  using  learning  objects,  highly  specified 
approaches  to  cataloging  learning  objects  come  with 
the  strength  of  enabling  sophisticated,  automated  uses 
of  the  metadata  to  support  the  location  and  use  of 
learning  objects.  The  weakness  with  following  such 
a  highly  structured  approach  is  the  significant  degree 
of  time  and  technical  expertise  required  to  conform 
with  the  specified  standards. 

Less  Specified  Approaches  to  Cataloging 
and  Finding  Learning  Objects 

In  the  mid-2000s,  several  Web-based  services  provided 
their  users  with  a  new  way  to  catalog  information.  Both 
del.icio.us,  a  social  bookmarking  service,  and  flickr,  a 
photo- sharing  site,  implemented  functionality  that  has 
come  to  be  called  tagging.  Tagging  differs  from  the 
creation  of  traditional  metadata  in  two  significant 
ways.  First,  when  librarians  or  educational  content 
producers  create  metadata  for  learning  objects,  they 
create  the  metadata  to  support  an  unknown  future  user 
in  finding  resources.  Conversely,  when  people  tag 
bookmarks  or  photos  in  del.icio.us  or  flickr,  they  are 
creating  metadata  to  help  themselves  find  the  materials 
sometime  in  the  future.  Second,  when  librarians  or 
educational  content  producers  create  metadata  they  use 
terms  from  previously  specified  vocabulary  lists,  such 
as  those  provided  by  the  Library  of  Congress  or  the 
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IEEE  LOM  standard.  Conversely,  when  people  tag 
bookmarks  or  photos  in  del.icio.us  or  flickr,  they  use 
any  terms  they  like— whatever  they  think  will  best  help 
them  find  the  bookmark  or  photo  later. 

As  opposed  to  the  more  complicated  Open 
Archives  Initiative  metadata  harvesting  protocol 
employed  by  more  structured  approaches,  services 
such  as  del.icio.us  and  flickr  expose  their  metadata  by 
means  of  Really  Simple  Syndication,  or  RSS.  The  idea 
of  thinking  about  learning  objects  as  syndicated 
resources  rather  than  packaged  resources  was  first 
described  by  Downes  (2000)  and  then  made  explicit 
by  a  number  of  authors,  most  notably  Downes  (2002) 
and  Lamb  and  Levine  (2004). 

The  economics  of  the  highly  and  less  specified 
approaches  are  very  different.  In  the  first  case,  one 
works  through  lengthy,  complicated  standards  to  sup¬ 
port  the  activities  of  an  unknown  future  user.  In  the 
second  case,  people  tag  resources  with  whatever  terms 
they  think  appropriate  to  support  their  own  future  uses. 
The  combination  of  a  simpler  approach  to  creating 
metadata  with  clear  personal  incentives  to  create  meta¬ 
data  has  made  tagging  extremely  popular.  Although 
only  large  organizations  or  projects  with  trained  staff 
can  afford  to  create  Library  of  Congress  or  LOM  meta¬ 
data,  hundreds  of  thousands  of  people  around  the  world 
have  applied  tens  of  millions  of  tags  to  millions  of 
learning  objects  available  online  (understanding  learn¬ 
ing  objects  to  be  digital  resources  that  can  be  reused  to 
mediate  learning) .  As  of  late  2006,  most  popular  online 
services  provided  users  with  the  ability  to  tag. 

The  strength  of  less  specified  approaches  to  cata¬ 
loging  learning  objects  is  that  the  economics  are  such 
that  hundreds  of  thousands  of  people  now  voluntarily 
create  metadata.  The  weakness  of  the  less  specified 
approach  is  that  the  potential  for  problems  related  to 
polysemy  in  which  a  single  tag  may  have  many  mean¬ 
ings  fc.g.,  Web  may  refer  to  a  spider’s  Web  or  the  World 
Wide  Web)  and  synonymy,  in  which  a  single  concept 
may  be  tagged  with  different  words  (metadata  vs. 
meta-data).  Both  problems  can  make  it  difficult  for 
users  to  locate  learning  objects  even  when  they  have 
been  tagged. 

Middle-Ground  Approaches  to  Cataloging 
or  Using  Learning  Objects 

Some  researchers  have  tried  to  walk  a  middle  ground 
by  blending  some  of  the  benefits  of  high  degrees  of 
specification  with  some  of  the  benefits  of  lesser 
degrees  of  specification.  Wang  and  Hsu  (2006)  devel¬ 
oped  an  ontology-based  system  for  cataloging  materi¬ 
als  that  is  not  used  to  support  an  automated  instruc¬ 
tional  design  system.  Their  system  supports  human 


users  in  the  discovery  of  learning  objects,  which 
humans  then  combine  into  courses  by  hand.  They  have 
reported  an  80%  mean  savings  in  time  to  produce  a 
new  course  across  30  users  using  either  their  blended 
system  or  traditional  approaches. 

Verbert  and  colleagues  (2006)  presented  another 
approach  they  refer  to  as  the  abstract  learning  object 
content  model  (ALOCoM).  ALOCoM  includes  an 
ontology  that  differentiates  between  content  frag¬ 
ments,  content  objects,  and  learning  objects.  Content 
fragments  are  combined  into  content  objects,  which  in 
turn  are  combined  into  learning  objects.  Content  object 
types  and  their  structures  are  defined  with  reference  to 
IBM’s  Darwin  Information  Typing  Architecture 
(DITA),  a  system  for  creating  and  managing  reusable 
technical  documentation.  Verbert  et  al.  (2006)  defined 
types  of  learning  objects  only  after  analysis  of  what 
instructional  content  is  actually  already  available  to 
mediate  the  primary  problem  with  highly  specified 
approaches  — the  extensive  retooling  necessary  for 
content  to  work  in  these  systems.  They  therefore  chose 
Slide  as  the  first  type  of  learning  object  to  work  with, 
and  they  built  support  for  OpenOffice  (Impress)  and 
MS  Office  (PowerPoint)  presentations  into  their  sys¬ 
tem. 

The  system  includes  a  disaggregator,  into  which 
users  can  upload  presentations.  The  disaggregator 
pulls  the  slides  apart  into  their  constituent  pieces  (e.g., 
bullets  of  text,  paragraphs  of  text,  images)  and  maps 
these  into  the  ALOCoM  ontology,  automatically  cre¬ 
ating  metadata  and  storing  metadata  and  content  for 
future  use.  When  building  a  new  presentation,  users 
search  the  repository  from  with  PowerPoint  itself  (by 
means  of  a  plug-in)  for  satisfactory  bullets  of  text, 
paragraphs,  images,  or  entire  slides,  and  they  can  pull 
these  directly  into  the  presentations  they  are  building. 

Criticisms  of  Learning  Objects 

Researchers  have  critiqued  the  entire  learning  objects 
way  of  thinking  from  a  number  of  perspectives .  Frie- 
sen  (2004),  in  a  well-known  paper  titled  “Three 
Objections  to  Learning  Objects  and  e-Leaming  Stan¬ 
dards,”  clearly  articulated  some  of  the  most  popular 
criticisms.  First,  the  community  of  interest  seems 
incapable  of  reaching  agreement  on  a  common  set  of 
terms,  a  criticism  echoed  by  Parrish  (2004)  and  others. 
Second,  specifications  and  standards  related  to  learn¬ 
ing  objects  are  almost  completely  technical,  focusing 
on  things  such  as  XML  and  controlled  vocabularies. 
They  fail  to  engage  pedagogy  directly  and  miss  the 
opportunity  to  move  the  practice  of  teaching  and  learn¬ 
ing  forward.  Third,  the  disproportionately  large  influ¬ 
ence  of  large  corporations  and  the  American  military 
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on  specifications  and  standards  makes  them  all  but  irrel¬ 
evant  for  public  and  higher  education.  Friesen  (2004) 
summarized  by  suggesting  that  “objects  and  infrastruc¬ 
tures  for  learning  cannot  simultaneously  be  both  ped- 
agogically  neutral  and  pedagogically  valuable.” 

Tompsett  (2005)  provided  a  mathematical,  graph 
theoretic  criticism  of  the  purported  capability  of  auto¬ 
mated  systems  to  assemble  learning  objects  into 
instruction.  Tompsett  began  by  considering  learning 
objects  as  nodes  in  a  graph,  where  “mutually  consis¬ 
tent”  learning  objects  are  connected  by  edges  in  the 
graph.  Fie  next  modeled  the  problem  of  choosing  learn¬ 
ing  objects  for  automated  assembly  as  the  search  for 
a  k-clique,  in  which  at  least  k  nodes  in  the  network  are 
completely  connected.  Finally,  he  modeled  the  prob¬ 
lem  of  sequencing  these  learning  objects  as  a  traveling 
salesman  problem,  in  which  a  path  must  be  found  that 
traverses  the  collection  of  learning  objects  without  vis¬ 
iting  any  learning  object  twice.  Tompsett  described 
these  problems  as  being  mathematically  complex: 
“Each  is  almost  trivial  to  solve  on  a  small  scale,  but 
becomes  unsolvable,  within  any  practical  terms,  as  the 
scale  increases”  (p.  443). 

Wiley  and  colleagues  (2004)  identified  a  number 
of  problems  with  learning  objects  regarding  context. 
Instructional  designers  work  to  make  learning  objects 
as  free  from  surrounding  context  as  possible  to 
increase  their  potential  for  reuse:  however,  in  the  cur¬ 
rent  intellectual  climate  that  emphasizes  things  such 
as  social  context,  various  aspects  of  settings,  and  sit¬ 
uatedness,  the  move  to  decontexualize  educational 
materials  is  troubling.  Jonassen  and  Churchill  (2004) 
viewed  the  entire  learning  object  approach  as  support¬ 
ing  outdated  instructivist  ways  of  thinking  about  teach¬ 
ing  and  learning,  and  although  Bannan-Ritland  et  al. 
(2002)  agreed,  they  saw  exciting  opportunities  for  con¬ 
structivists  who  are  willing  to  work  with  learning 
objects.  Wiley  and  colleagues  summarized  the  context- 
related  issue  as  the  reusability  paradox,  in  which  the 
more  reusable  learning  objects  are,  the  less  instruc- 
tionally  effective  they  are,  and  vice  versa. 

A  final  criticism  relates  to  the  anticipated  emer¬ 
gence  of  an  educational  object  economy  in  which  indi¬ 
viduals  and  corporations  can  buy  and  sell  access  to 
learning  objects  via  micropayment  systems  — systems 
capable  of  selling  digital  goods  for  arbitrarily  small 
amounts  of  money  (e.g.,  access  to  an  online  news  story 
for  a  penny  or  less).  As  Liber  (2005)  noted,  the  idea 
of  micropayment  systems  has  been  around  for  a  very 
long  time  (all  the  way  back  to  Nelson’s  work  in  the 
1960s!),  but  no  viable  micropayment  has  yet  been 
implemented.  Wiley  and  colleagues  (2004)  posited  that 
the  fear  of  pirated  copies  of  learning  objects  being 
traded  in  Napster-like  networks  has  prevented  publish¬ 


ers  from  creating  these  systems.  Wilhelm  and  Wilde 
(2005)  reported  that  the  burden  of  clearing  copyright 
for  learning  objects  to  be  used  in  the  course  production 
process  can  block  the  process  altogether. 


OPEN  EDUCATIONAL  RESOURCES 

Earlier  in  this  chapter,  I  commented  that  the  prevailing 
learning  object  metaphors  make  the  assumption  that 
learning  objects  can  be  aggregated  but  not  adapted. 
Wilhelm  and  Wilde  (2005,  p.  69)  made  explicit  the  all- 
pervasive,  fundamental  barrier  to  repurposing  or  adapt¬ 
ing  learning  objects:  "While  we  contemplated  modi¬ 
fying  some  learning  material  to  construct  part  of  our 
course,  this  task  generally  required  obtaining  permis¬ 
sion  from  website  owners.” 

Acquiring  copyright-related  permissions  from  a 
rights  holder  entails  two  kinds  of  costs.  The  first  kind 
of  cost  is  the  license  cost  paid  in  exchange  for  the 
rights  to  reuse  a  learning  object.  The  second  kind  of 
cost  is  the  hidden  transaction  costs  associated  with 
determining  who  holds  the  rights  to  a  specific  learning 
object  (which  can  be  very  time  consuming),  contacting 
the  rights  holder,  and  negotiating  a  contract  under 
which  you  can  acquire  the  right  to  reuse  the  learning 
object.  The  sum  of  the  transaction  and  license  costs 
can  literally  bury  an  instructional  development  process 
primarily  dependent  on  learning  objects,  particularly 
when  large  numbers  objects  are  being  used.  For  this 
reason,  most  people  assume  that  for  all  practical  pur¬ 
poses  learning  objects  can  only  be  aggregated  and  not 
adapted  to  fit  their  specific  contexts  or  meet  the  needs 
of  their  specific  learners. 

In  the  spring  of  2002,  the  William  and  Flora 
Hewlett  Foundation  sponsored  a  forum  at  UNESCO 
related  to  this  topic.  The  report  of  that  meeting  intro¬ 
duced  the  term  open  educational  resource  to  the 
world  (UNESCO,  2002): 

Open  Educational  Resources  are  defined  as  “technology- 
enabled,  open  provision  of  educational  resources  for  con¬ 
sultation,  use  and  adaptation  by  a  community  of  users 
for  non-commercial  purposes.”  They  are  typically  made 
freely  available  over  the  Web  or  the  Internet.  ...Open 
Educational  Resources  include  learning  objects  such  as 
lecture  material,  references  and  readings,  simulations, 
experiments  and  demonstrations,  as  well  as  syllabi,  cur¬ 
ricula  and  teachers'  guides. 

Individuals  who  wish  to  reuse  open  educational 
resources  bear  neither  the  license  costs  nor  the  trans¬ 
action  costs  associated  with  materials  trapped  within 
traditional  copyrights  due  to  the  way  in  which  these 
materials  are  licensed.  Open  educational  resources  are 
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licensed  with  open-source-style  licenses,  such  as  the 
Creative  Commons  licenses  (http://creativecom- 
mons.org/)  or  the  GNU  Free  Documentation  License 
(http  ://www.gnu  .org/copyleft/fdl  .html) . 

Although  the  idea  of  open  educational  resources 
sounds  idealistic,  according  to  Wiley  (2006)  over  2500 
university  courses  (which  are  composed  of  individu¬ 
ally  addressable  learning  objects)  are  currently  avail¬ 
able  as  open  educational  resources,  including  over 
1700  courses  from  U.S .  universities,  450  from  Chinese 
universities,  350  from  Japanese  universities,  and  175 
courses  from  French  universities.  As  if  to  showcase 
the  adaptability  of  the  materials  in  these  courses,  many 
of  them  have  already  been  translated  into  Spanish, 
Portuguese,  Chinese,  and  Thai.  The  Connexions 
project  at  Rice  currently  hosts  3590  open  learning 
objects;  Textbook  Revolution  (http://textbookrevolu- 
tion.org/)  contains  links  to  260  freely  available  ,  copy¬ 
right-clean  textbooks. 

Each  of  these  millions  of  learning  objects— every¬ 
thing  from  modules  to  textbooks  to  courses  — is 
licensed  in  such  as  way  that  reusers  can  both  aggregate 
and  adapt  the  materials  with  neither  license  or  trans¬ 
action  costs. 


CONCLUDING  REMARKS 

A  review  of  the  learning  objects  literature  reveals  a 
largely  disconnected  group  of  researchers  united  by  an 
interest  in  reusing  educational  materials  but  little  else. 
The  research  area  is  a  conglomerate  of  competing 
terms,  competing  metaphors,  competing  technical  stan¬ 
dards,  and  competing  ontologies.  As  implied  by  these 
many  bifurcations,  the  research  area  itself  is  an  uncom¬ 
fortable  amalgam  of  two  competing  philosophies. 

First  are  the  traditional  learning  objects  researchers 
whose  goal  is  to  automate  the  just-in-time  assembly 
of  learning  objects  into  personalized  educational  expe¬ 
riences.  They  rely  on  learning  objects  adhering  to  spe¬ 
cific  structural  and  content  standards  to  leverage  the 
power  of  intelligent  systems  to  provide  the  learner 
exactly  what  she  needs. 

Second  are  the  permissive  learning  objects 
researchers  whose  goal  is  to  make  the  reuse  and  local¬ 
ization  of  all  resources,  regardless  of  their  structure  or 
adherence  to  other  standards,  as  effective  and  efficient 
as  possible  .  They  assume  that  humans  will  be  involved 
in  the  process  of  localizing  learning  objects,  and  they 
rely  on  learners  to  engage  in  selecting  what  they  want. 

Although  the  arguments  of  both  parties  have  their 
strengths,  neither  has  yet  contributed  the  promised 
application  or  process  that  will  revolutionize  education 
and  training.  In  such  a  young  and  leaderless  field,  great 


opportunities  remain  for  researchers  who  can  unify  the 
field  around  practical,  working  responses  to  historic 
and  emerging  problems  in  education,  training,  infor¬ 
mal,  and  lifelong  learning. 

Hybrid  approaches  that  find  creative  ways  to  syn¬ 
thesize  the  seemingly  contradictory  agendas,  prereq¬ 
uisites,  and  strengths  of  the  traditional  and  permissive 
approaches  to  learning  objects  research  are  likely 
where  the  future  of  the  field  lies.  Nascent  work  in  open 
educational  resources  has  revealed  one  assumption 
many  seem  to  have  looked  past— namely,  that  when 
properly  licensed  learning  objects  can  be  adapted  and 
adjusted  in  addition  to  being  aggregated  and  aligned. 
As  visionary  people  help  the  field  identify  and  remove 
additional  unspoken  barriers  we  should  eventually  be 
able  to  bridge  the  gap  between  philosophical  differ¬ 
ences  to  better  serve  learners  everywhere. 
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ABSTRACT 

The  objective  of  this  chapter  is  to  create  an  understand¬ 
ing  of  the  importance  of  open  source  software  and  open 
standards  (OSS/OS)  for  e-learning  research.  Open 
source  is  a  fundamental  new  way  to  develop  software, 
and  open  standards  are  needed  to  make  software  com¬ 
ponents  work  together.  It  is  argued  that  OSS  and  OS  can 
improve  the  convergence  of  knowledge  in  the  e-leaming 


field,  improve  the  general  quality  and  interoperability  of 
e-learning  applications,  and  improve  collaboration 
between  researchers  and  users.  All  of  these  are  beneficial 
and  necessary  requirements  for  e-learning  research. 
After  a  general  introduction  into  basic  OSS  and  OS 
concepts,  the  following  questions  will  be  answered:  (1) 
How  does  OSS/OS  facilitate  the  technological  activities 
of  the  researchers  in  terms  of  methodology,  collabora¬ 
tion,  and  dissemination  of  results?  (2)  How  does 
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OSS/OS  facilitate  the  development  of  technological 
knowledge  in  the  field?  (3)  How  does  OSS/OS  facilitate 
the  development  of  technological  artifacts  in  the  field? 
The  development  and  use  of  the  open  standard  IMS 
Learning  Design  (a  formal  design  language  for  online 
courses)  and  the  open  source  applications  that  are  devel¬ 
oped  to  run  and  present  IMS  Learning  Design  courses 
are  used  as  an  example  to  demonstrate  the  use  of 
OSS/OS  in  e-learning  research.  In  the  concluding  sec¬ 
tion,  we  provide  some  practical  information  for 
researchers  on  how  to  get  involved  in  OSS  and  OS  and 
how  to  use  them  in  e-learning  research. 


KEYWORDS 

IMS  Learning  Design:  A  formal  instructional  design 
language  that  is  used  to  specify  the  design  of  a 
teaching  and  learning  process  in  a  machine  inter¬ 
pretable  way. 

Open  source  software  (OSS):  Software  that  has  one  of 
the  OSI  Open  Source  Licenses  attached.  These 
licenses  state  that  the  source  code  of  a  program 
should  always  be  available  to  everyone  and  that 
everyone  can  change  the  source  code. 

Open  standard  (OS):  Commonly  agreed  upon  and  pub¬ 
lished  specifications  of  the  conventions  used  in  a 
community  to  ensure  the  quality  and  interoperabil¬ 
ity  of  products  and  services. 

Technology-oriented  research:  Research  that  aims  to 
develop  new  technological  knowledge,  methods, 
and  artifacts  to  change  the  world  as  it  exists  with 
the  final  aim  of  improving  the  way  we  live. 

Theory -oriented  research:  Directed  at  the  development 
of  theories  about  the  world  as  it  exists,  with  the 
final  aim  of  predicting  or  understanding  events. 


INTRODUCTION 

The  objective  of  this  chapter  is  to  create  an  understand¬ 
ing  of  the  importance  of  open  source  software  and 
open  standards  (OSS/OS)  for  e-learning  research. 
Open  source  is  a  fundamental  new  way  to  develop 
software,  and  open  standards  are  required  to  make 
software  components  work  together.  Both  stimulate 
exchange,  collaboration,  interoperability,  and  conver¬ 
gence  of  knowledge  and  these  are  beneficial  require¬ 
ments  for  future  e-leaming  research. 

e-Learning  can  be  defined  as  the  use  of  information 
and  communication  technologies  (ICTs)  to  facilitate 
and  enhance  learning  and  teaching.  e-Learning 
research  is  aimed  at  the  development  of  new  technol¬ 


ogies  to  improve  learning,  training,  and  teaching  in 
various  ways: 

•  By  making  them  more  accessible  to  every¬ 
one  at  any  place  and  at  any  time 

•  By  making  them  more  effective  by  facilitat¬ 
ing  the  implementation  of  advanced  peda¬ 
gogical  and  organizational  approaches 

•  By  making  them  more  efficient  by  providing 
advanced  (partly  automated)  support  mech¬ 
anisms  for  learners  and  teachers  to  perform 
their  various  tasks 

•  By  making  them  more  attractive  to  users  by 
providing  adapted  tasks  and  resources 

e-Learning  research  is  technology  oriented  instead 
of  theory  oriented.  Technology-oriented  research,  also 
called  technology  development  or  engineering,  differs 
in  fundamental  ways  from  theory-oriented  research. 
These  research  approaches  differ:  ( 1 )  in  how  the  prob¬ 
lems  are  addressed,  (2)  in  the  ways  in  which  research 
activities  are  performed,  (3)  in  the  notation  and  com¬ 
munication  means  that  are  used,  and  (4)  in  the  results 
that  are  delivered  (see,  for  example.  Gibbons,  2000; 
Hannay  and  McGinn,  1980;  McGinn,  1978;  Rogers, 
1995;  Simon,  1969;  Vincenti,  1990).  Mitcham  (1994, 
p.  116)  stated  that:  “Virtually  all  historians  ...  use  the 
word  ‘technology’  to  refer  to  both  ancient  and  modern, 
primitive  and  advanced  making  activities,  or  knowl¬ 
edge  of  how  to  make  and  use  artifacts,  or  the  artifacts 
themselves.”  A  distinction  can  be  made  among  (1)  the 
technological  activities  of  the  researchers  (methods  for 
making  an  artifact),  (2)  the  technological  knowledge 
that  is  a  result  of  these  making  activities  (models  and 
specifications),  and  (3)  the  technological  artifacts  that 
are  the  results  of  these  activities.  These  distinctions 
have  been  used  to  structure  this  chapter  when  we  dis¬ 
cuss  the  use  of  OSS/OS  in  e-learning  research.  The 
following  questions  are  answered: 

•  How  does  OSS/OS  facilitate  the  technolog¬ 
ical  activities  of  the  researchers  in  terms  of 
methodology,  collaboration  and  dissemina¬ 
tion  of  results? 

•  How  does  OSS/OS  facilitate  the  develop¬ 
ment  of  technological  knowledge  in  the 
field? 

•  How  does  OSS/OS  facilitate  the  develop¬ 
ment  of  technological  artifacts  in  the  field? 

Before  going  into  these  questions,  the  concepts  of  open 
source  software  (OSS)  and  open  standards  (OS)  are 
discussed  more  in  general  with  an  emphasis  on  the  use 
of  OSS/OS  as  means  to  perform  research  on  e-learning. 
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OPEN  SOURCE 
What  Is  Open  Source? 

Software  is  written  in  a  computer  language  before  it 
is  compiled  into  binary  code  that  computers  can  run. 
The  human  readable  text  originally  written  by  the  pro¬ 
grammers  in  a  computer  language  is  the  source  code 
of  a  program.  The  source  code,  the  derived  binary 
code,  and  the  documentation  are  protected  by  intellec¬ 
tual  property  rights  (IPRs).  Only  the  owner  of  the  IPR 
is  entitled  to  change  the  code  or  the  documentation, 
and  only  the  owner  of  the  copyright  is  entitled  to  copy 
and  distribute  these. 

This  closed  source  software  approach  has  been 
under  attack  by  the  free  software  and  open  source 
movement.  The  Free  Software  Foundation  (FSF), 
founded  by  Richard  Stallman  in  1984,  is  the  organi¬ 
zation  behind  MIT’s  GNU  project.  One  of  the  contri¬ 
butions  of  the  FSF  is  the  development  of  the  General 
Public  License  (GPL,  2006)  to  protect  the  IPRs  of 
contributors  and  prevent  unwanted  commercialization 
of  the  software.  The  FSF  (FSF-DEF,  2006)  defines  free 
software  as  "the  freedom  of  all  users  to  run,  copy, 
distribute,  study,  change  and  improve  software.  Source 
code  is  seen  as  a  kind  of  scientific  knowledge  that 
should  be  published  to  facilitate  innovation.”  It  is 
worth  noting  that  “free  software”  as  used  here  does 
not  coincide  with  the  notion  of  software  that  is  avail¬ 
able  without  cost  to  the  user.  Some  software  that  is 
freely  available  is  not  free  software,  and  some  free 
software  may  involve  nominal  costs  to  users. 

The  term  free  software  has  developed  some  nega¬ 
tive  connotations,  especially  in  industry.  This  is  the 
reason  why  a  group  of  people,  including  Eric  Steven 
Raymond,  started  in  1997  to  promote  the  use  of  free 
software  by  stressing  the  technical  superiority  and  low 
cost  instead  of  its  rather  anti-business  and  ideological 
aspects.  They  used  the  term  open  source  software 
instead  of  free  software  and  founded  the  Open  Source 
Initiative  (OSI)  in  1998  to  provide  a  definition  of  OSS 
and  a  set  of  criteria  for  open  source  licenses  (OSI- 
licenses,  2006).  The  GPL  license  is  considered  to  be 
a  valid  OSI  license,  among  many  other  licenses  that 
are  less  restrictive  for  use  in  the  commercial  world. 

Open  Source  Development  Model 

Many  OSS  development  projects  are  now  underway. 
SourceForge,  for  example,  supports  more  then  a 
100,000  projects,  most  for  general  use  but  many  spe¬ 
cifically  developed  for  e-leaming.  One  of  the  charac¬ 
teristics  of  OSS  is  that  it  is  developed  in  a  different 
way  than  commercial  software.  Raymond  (1998, 
2001)  compared  two  development  models:  the  cathe¬ 


dral  (as  a  metaphor  for  traditional  software  develop¬ 
ment)  and  the  bazaar  (as  a  metaphor  for  OSS  devel¬ 
opment)  to  ground  the  idea  of  higher  quality  and  lower 
costs.  Characteristics  of  the  bazaar  model  for  OSS 
development  tend  to  include:  (1)  globally  distributed 
communities  of  developers  collaborating  primarily 
through  the  Internet,  (2)  developers  working  in  paral¬ 
lel,  (3)  developers  exploiting  the  power  of  peer  review 
for  debugging  and  requirements  analysis,  (4)  rapid, 
incremental  release  schedules,  and  (5)  projects  with 
pools  of  experienced  and  esteemed  professional  devel¬ 
opers  (Feller  and  Fitzgerald,  2002).  OSS  communities 
have  developed  some  strong  cultural  norms  that  govern 
the  mainly  self-organized  development  system 
(Bergquist  and  Ljungberg,  2001;  Jorgensen,  2001). 

The  success  of  the  OSS  development  model  has 
invoked  many  questions,  especially  in  economics  and 
organizational  theory.  Madey  et  al.  (2002,  p.  1807) 
formulated  it  this  way: 

The  OSS  movement  is  a  phenomenon  that  challenges 
many  traditional  theories  in  economics,  software  engi¬ 
neering,  business  strategy,  and  IT  management.  Thou¬ 
sands  of  software  programmers  are  spending  tremendous 
amounts  of  time  and  effort  writing  and  debugging  soft¬ 
ware,  most  often  with  no  direct  monetary  compensation. 

Empirical  studies  have  been  performed  on  the  size  and 
distribution  of  development  teams  (Crowston  and 
Howison,  2005),  comparisons  of  the  organization  of 
various  OSS  projects  (Dempsey  et  al.,  1999;  Gallivan, 
2001;  Mockus  et  al.,  2002),  and  the  organization  of 
social  relationships  and  incentives  in  OSS  communi¬ 
ties  (Bergquist  and  Ljungberg,  2001;  Lerner  and  Tri- 
ole,  2002). 

Although  an  OSS  development  model  is  some¬ 
times  perceived  as  something  new,  the  principle  of 
sharing  software  and  codeveloping  has  been  a  common 
practice  among  academics  from  the  early  days  of  com¬ 
puter  programming.  In  the  early  1960s,  many  funda¬ 
mental  software  programs  (operating  systems,  com¬ 
puter  languages,  etc.)  were  developed  in  universities 
such  as  the  Massachusetts  Institute  of  Technology  and 
the  University  of  California  at  Berkeley,  and  in  com¬ 
pany  laboratories  such  as  AT&T  Bell  Labs  and  Xerox’s 
Palo  Alto  Research  Center.  Researchers  shared  their 
code  so  others  could  inspect,  use,  and  improve  it.  This 
mode  of  working  was  rather  similar  to  the  way  in 
which  researchers  have  always  shared  ideas  through 
publications,  reports,  notes,  and  conferences.  This 
mode  of  sharing  was  rather  informal,  and  as  a  result 
of  this  AT&T  could  begin  to  enforce  its  IPRs  on  UNIX 
in  the  beginning  of  the  1980s.  This  was  one  of  the 
triggers  for  the  development  of  the  GNU  license. 
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General  Open  Source  Applications 

Many  of  the  tools  and  services  used  in  daily  life  are 
based  on  OSS,  specifically  on  the  server  side  (e.g., 
e-mail  based  on  sendmail,  websites  based  on  Apache, 
servers  running  on  Linux).  For  the  client,  many  high- 
quality  OSS  alternatives  are  available  for  commercial 
software.  Examples  include  Thunderbird  (2006)  as  a 
mail  client,  Firefox  (2006)  as  a  browser,  GIMP 
(2006)  as  an  advanced  drawing  tool,  FreeMind 
(2006)  as  a  mind-mapping  tool.  Eclipse  (2006)  as  an 
integrated  software  development  environment,  or 
OpenOffice.org  (2006)  as  an  office  suite.  Most  of 
these  tools  are  interoperable  with  commercial  soft¬ 
ware  through  the  use  of  import  and  export  filters  to 
different  formats. 

Open  Source  in  Learning, 

Education,  and  Training 

Open  source  software  is  used  in  a  variety  of  ways  in 
learning,  education,  and  training  contexts.  Many  types 
of  computer  use  involve  some  kind  of  informal  learn¬ 
ing,  such  as  performing  a  search  with  Google,  using 
Wikipedia,  making  and  using  podcasts,  writing  and 
reading  blogs  and  wikis,  and  so  on.  The  use  of  OSS 
in  schools  is  being  explored  and  applied  in  many 
regions  in  the  world  where  cost  savings  and  stimulat¬ 
ing  local  industries  are  important  issues.  In  these  cases, 
the  Linux  operating  system  is  often  used  as  a  base  for 
an  educational  software  package  that  contains  a  selec¬ 
tion  of  general  and  specific  open  source  applications 
that  can  be  used  in  the  schools  (see,  for  example, 
Edubuntu,  2006).  In  the  Spanish  region  of  Extrema¬ 
dura,  a  Debian-based  version  of  Linux  (LiNex,  2006), 
is  deployed  on  some  70,000  desktop  PCs  and  400 
servers  in  the  educational  sector.  In  Norway,  around 
200  schools  use  Skolelinux  (2006). 

In  addition  to  these  products,  many  projects  are 
delivering  specific  educational  applications.  The  area 
of  e-learning  offers  several  open  source  learning  man¬ 
agement  systems  (LMSs),  including  Moodle  (2006), 
Sakai  (2006),  .Learn  (2006),Bodington  (2006),  ATutor 
(2006),  and  Dokeos  (2006),  among  others.  The  devel¬ 
opment  of  these  LMSs  is  at  the  moment  challenged 
by  the  emergence  of  new  generations  of  technologies, 
such  as  the  use  of: 

•  Web  services  for  e-leaming  (Alonso  et  al„ 
2004;  Vossen  and  Westerkamp,  2003) 

•  Semantic  Web  principles  (Anderson  and 
Whitelock,  2004;  Berners-Lee  et  al.,  2001) 

•  Adaptive  learning  principles  (Berlanga  and 
Garcia,  2005;  Brusilovsky,  2001;  De  Bra  et 
al.,  2004) 


•  Learning-process-oriented  systems  (Dalziel, 
2003;  LAMS,  2006;  Paquette  et  al.,  2006) 

•  Social  software  (ELGG,  2006) 

•  Shared,  self-created  multimedia  files  (e.g., 
podcasts)  (LionShare,  2006) 

•  Mobile  technologies  in  learning  (Jones  et  al., 
2005) 

BECTA  (2005)  has  performed  a  study  of  the  use  of 
OSS  in  U.K.  schools.  The  cases  presented  in  the  study 
show  that  OSS  can  be  used  as  a  server  operating  sys¬ 
tem,  as  a  desktop  operating  system,  and  for  applica¬ 
tions  used  in  the  classroom  or  for  administration.  The 
study  also  mentions  the  reasons  why  schools  moved 
to  OSS: 

•  They  liked  its  transparency  and  flexibility, 
which  made  it  possible  to  alter  the  software 
according  to  their  needs. 

•  There  was  an  educational  value  to  providing 
pupils  with  a  broader  experience  of  operat¬ 
ing  systems  and  software. 

•  It  was  a  way  to  achieve  value  for  the  money 
spent  and  to  extend  the  ICT  network  and 
facilities. 

•  They  had  access  to  appropriate  knowledge, 
skills,  and  experience  to  support  an  OSS 
implementation. 

•  Most  stakeholders  (pupils ,  teachers ,  parents) 
also  appreciated  the  use  of  OSS. 

The  disadvantages  identified  included  a  lack  of  cur¬ 
riculum-specific  courseware,  compatibility  problems 
with  some  commercial  software,  and  a  lack  of  famil¬ 
iarity  among  teachers  and  pupils.  The  study  has  also 
found  that  the  total  costs  for  the  use  of  OSS  were 
lower  compared  to  the  use  of  proprietary  software, 
but  this  is  dependent  largely  on  the  way  OSS  is  used 
and  supported. 

OPEN  STANDARDS 
What  Are  Open  Standards? 

Open  standards  are  of  enormous  importance  in  our 
society  to  ensure  that  products  and  services  are  of 
sufficient  quality  and  can  work  together— that  is  to 
say,  they  are  interoperable.  The  term  open  standard 
has  many  interpretations.  According  to  Krechmer 
(2005),  this  is  due  to  the  fact  that  creators,  imple¬ 
mented,  and  users  of  OS  each  have  a  different  set  of 
requirements  and  as  a  result  a  different  perspective 
on  OS;  for  example,  the  creators,  as  represented  by 
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the  standards  organizations,  will  focus  on  the  open¬ 
ness  of  the  process  of  standard  development,  specif¬ 
ically  a  due  process  with  open  meetings  and  decisions 
made  by  consensus.  The  implementers  will  focus 
more  on  the  free  use  of  the  standards  and  the  com¬ 
patibility  with  previous  implementations.  The  end 
users  will  focus  on  such  aspects  as  the  number  of 
implementations  from  different  vendors  and  the  com¬ 
patibility  with  currently  used  systems.  End  users  are 
often  interested  in  de  facto  standards  instead  of  OS  — 
for  example,  when  they  say  that  they  want  to  stan¬ 
dardize  on  Blackboard. 

The  definition  that  I  will  use  for  OS  (in  e-learn¬ 
ing)  is  as  follows:  “Open  standards  (in  e-learning)  are 
commonly  agreed  upon  and  published  specifications 
of  the  conventions  used  in  a  community  to  ensure  the 
quality  and/or  interoperability  of  (e-learning)  prod¬ 
ucts  and  services.”  Several  remarks  can  be  made 
about  this  definition.  First  of  all,  the  definition  con¬ 
tains  the  word  specification ,  which  means  that  an 
open  standard  is  conceived  as  a  document  or  set  of 
documents  and  not  as  a  specific  product  or  service. 
These  documents  contain  agreements  about  quality 
standards,  data  formats,  and  communication  proto¬ 
cols.  The  idea  is  that  e-learning  products  implement 
the  different  open  standards  to  enable  interoperabil¬ 
ity.  Many  products  also  advertise  that  they  are  com¬ 
pliant  with  certain  standards,  but  this  is  difficult  to 
test  because  of  the  lack  of  formal  conformance  pro¬ 
cedures  to  test  whether  a  product  is  truly  compliant 
with  a  specific  standard.  The  European  TELCERT 
(2006)  project  has  developed  a  first  set  of  tools  for 
such  conformance  testing. 

Second,  the  definition  defines  two  core  functions 
of  OS: 

•  To  ensure  the  quality  of  e-leaming  products 
and  services,  including  the  quality  of  learn¬ 
ing  objects,  the  quality  of  a  systems  design, 
the  usability  of  the  software,  and  so  on 

•  To  ensure  the  interoperability  of  e-learning 
products  and  services;  interoperability  sup¬ 
ports  the  collaboration  between  systems  but 
also  between  humans  who  develop  or  use  a 
system  (for  example,  notation  standards  and 
standard  vocabularies) 

Furthermore,  the  definition  points  to  the  fact  that 
standards  are  always  agreed  upon  and  used  within  a 
certain  community  of  interest,  such  as  a  company,  a 
consortium,  a  country,  a  specific  technology,  or  a 
(worldwide)  field  of  expertise.  A  standard  has  the 
status  of  a  recommendation  for  the  community  mem¬ 
bers.  This  leaves  open  the  possibility  that  different 


communities  and  different  standards  could  exist  in 
the  same  area,  just  as  there  are  different  power  plug 
standards  or  railway  systems  in  different  countries. 
For  metadata,  we  can  use  the  IMS/IEEE  Learning 
Object  Metadata  (LOM)  (educational  sector)  or  the 
Dublin  Core  metadata  (library  sector).  Added  to  this, 
many  countries,  professional  sectors,  and  companies 
have  defined  their  own  local  standards  which  are  not 
compatible  with  each  other  nor  with  international 
standards. 

The  ideal  of  the  standards  committees  is  to  have 
one  worldwide  accepted  standard  that  can  be  localized 
to  fit  the  needs  of  different  organizations.  This  ideal 
is,  however,  difficult  to  accomplish  in  practice,  so  we 
have  to  deal  with  the  fact  that  there  are  still  many 
different  communities,  each  with  its  own  incompatible 
standards.  In  that  case,  the  interoperability  question 
has  to  be  solved  when  the  two  communities  want  to 
collaborate. 

How  Are  Open  Standards  Developed  and 
Which  e-Learning  Standards  Are  Available? 

International  specifications  are  traditionally  developed 
by  three  standards  organizations:  the  International 
Organization  or  Standardization  (ISO,  founded  in 
1947),  the  International  Electrotechnical  Commission 
(IEC ,  founded  in  1906) ,  and  the  International  Telecom¬ 
munication  Union  (ITU,  founded  in  1865).  Most  coun¬ 
tries  and  parts  of  the  world  have  their  own  standards 
organizations  that  are  directly  associated  with  the 
international  organizations  (e.g.,  ANSI  in  the  United 
States,  CEN  in  Europe). 

Besides  these  organizations  being  country  or 
regionally  specific,  some  standards  organizations 
transcend  country  boundaries  by  adopting  an  expert 
model  approach,  such  as  the  Institute  of  Electrical 
and  Electronics  Engineers  (IEEE) .  Some  well-known 
IEEE  standards  are  the  IEEE  802.3  Ethernet  standard 
and  the  IEEE  802.11  Wireless  Networking  standard. 
The  IEEE  is  also  active  in  e-learning  through  the 
IEEE  Learning  Technologies  Standards  Committee 
(LTSC).  This  committee  is  working  on  topics  such  as 
a  digital  rights  expression  language,  computer-man¬ 
aged  instruction,  learning  objects  metadata,  and  com¬ 
petency  definitions. 

The  ISO  formed  a  joint  technical  committee  with 
the  IEC  (ISO/IEC  JTC1)  that  has  the  objective  to 
develop,  maintain,  promote,  and  facilitate  ICT  stan¬ 
dards  required  by  global  markets  meeting  business 
and  user  requirements  concerning  the  design  and 
development  of  systems  and  tools,  performance  and 
quality  of  products  and  systems,  security  of  systems 
and  information,  portability  of  application  programs, 
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interoperability  of  products  and  systems,  unified 
tools  and  environments,  and  harmonized  vocabulary 
and  user-friendly  and  ergonomically  designed  user 
interfaces . 

The  JTC1  has  several  subcommittees,  including 
SC36,  which  is  responsible  for  Information  Technol¬ 
ogy  for  Learning,  Education,  and  Training.  SC36  is 
working  on  standards  for  a  collaborative  workplace 
and  agent/agent  communication,  as  well  as  on  a 
learner- to-learner  interaction  scheme.  They  are  cur¬ 
rently  setting  up  many  more  groups. 

Standards  organizations  that  work  according  to  a 
particular  country’s  model  (e.g.,  ISO)  had  a  reputation 
in  the  1990s  of  being  too  slow  to  keep  up  with  the 
standardization  needs  of  quickly  changing  areas  such  as 
ICT  and  e-leaming.  ISO  had  a  large  project.  Open  Sys¬ 
tems  Connect,  that  tried  to  develop  a  common  computer 
networking  standard.  The  project  did  not  succeed  and 
was  stopped  in  1996,  at  which  point  it  was  taken  over 
by  an  organization  called  the  Internet  Engineering  Task 
Force  (IETF,  founded  in  1986).  This  task  force  had  a 
less  bureaucratic,  open  process  and  developed  the  basic 
protocol  suites  that  were  needed  for  the  Internet  to  oper¬ 
ate.  Later,  the  IETF  was  also  perceived  as  being  too 
slow,  and  most  industry  vendors  are  currently  working 
with  specifications  from  more  specialized  consortia 
such  as  the  World  Wide  Web  Consortium  (W3C).  This 
consortium  creates  and  maintains  standards  for  the 
World  Wide  Web  (HTTP,  URL,  Linking,  XML,  Seman¬ 
tic  Web). 

In  the  e-learning  field,  the  dominant  specialized 
consortium  is  IMS,  a  consortium  of  the  major  players, 
companies,  and  researchers  in  the  e-learning  field.  IMS 
developed  and  maintains  17  specifications  in  the  fol¬ 
lowing  fields:  metadata,  assessment,  learning  design, 
content  packaging,  sequencing,  ePortfolio,  learner 
information,  digital  repositories,  competency  defini¬ 
tion,  and  interoperability  of  learning  management  sys¬ 
tems  with  enterprise  systems. 

Most  of  the  current  e-learning  standards  concen¬ 
trate  on  the  syntax  of  the  data  format  that  should  be 
used  for  the  asynchronous  exchange  of  learning 
resources  or  learner  information.  Less  attention  has 
been  given  to  standardization  of  synchronous  com¬ 
munication  between  systems  and  standardization  of 
the  semantics  of  the  communication  process.  In  the 
field  of  semantics,  several  exceptions  can  be  found, 
including  the  work  done  by  the  International  Board 
of  Standards  for  Training,  Performance,  and  Instruc¬ 
tion  (IBSTPI;  http://www.ibstpi.org)  on  the  definition 
of  competencies  for  instructors,  instructional  design, 
training  managers,  and  evaluators.  These  standards 
have  not  yet  been  defined  in  the  technical  formats 
provided  by  IMS,  IEEE,  or  HR-XML. 


OSS/OS  AS  A  MEANS  TO  FACILITATE 
E-LEARNING  RESEARCH 

How  can  OSS/OS  facilitate  e-learning  research?  As 
stated  before,  OSS/OS  can  facilitate  e-learning 
research  that  results  in  both  new  technological  knowl¬ 
edge  and  new  technological  artifacts.  The  next  para¬ 
graphs  discuss  these  possibilities  of  using  an  OSS/OS 
approach  in  e-learning  research. 

OSS/OS  to  Facilitate  Technological 
Activity  in  e-Learning 

The  major  activity  in  e-leaming  research  is  to  develop 
new  e-learning  technologies.  This  development  pro¬ 
cess  is  facilitated  by  research  methodologies  based  on 
the  principles  of  systems  engineering  (Richey  and  Nel¬ 
son,  1996).  The  use  of  OSS/OS  can  facilitate  the  devel¬ 
opment  process  in  several  ways:  (1)  by  providing  a 
standard  notation  system  to  foster  communication  and 
collaboration,  (2)  by  facilitating  the  development  of 
the  systems  by  multiple  distributed  users  using  the 
OSS  development  model,  (3)  by  facilitating  the  eval¬ 
uation  of  the  developed  artifacts,  and  (4)  by  stimulating 
the  dissemination  of  results. 

During  this  development  there  is  a  need  to  com¬ 
municate  the  design  of  the  system  among  researchers 
and  users  within  and  outside  the  team.  For  communi¬ 
cation  purposes,  a  notation  system  is  used  to  capture 
user  requirements  and  to  notate  the  design  of  the  envis¬ 
aged  system.  When  such  a  notation  system  conforms 
to  a  widely  known  open  standard,  it  facilitates  the 
correct  understanding  of  the  design  and  thus  the  qual¬ 
ity  of  the  discussions  among  developers  and  users.  The 
best  example  of  such  a  notation  system  is  the  Unified 
Modeling  Language  (UML)  (Booch  et  al.,  1999; 
Fowler,  2000),  an  open  standard  developed  by  the 
Object  Management  Group  (OMG) .  The  use  of  UML 
is  very  common  in  ICT  research  today  but  is  still  a 
rather  new  phenomenon  in  e-learning  research, 
although  it  is  found  more  and  more  in  e-learning  pub¬ 
lications  (Zarraonandia  et  al.,  2005). 

Unified  Modeling  Language  defines  nine  types  of 
diagrams,  each  providing  a  different  view  of  a  system 
under  development.  Three  types  of  diagrams  in  e-learn¬ 
ing  publications  are  most  often  used: 

•  Cases  are  used  to  model  the  (envisaged)  user 
requirements  and  benefits  of  the  system  for 
(future)  users  (see  Figure  2  of  Asensio  et  al., 
2004). 

•  Class  Diagrams  are  used  to  model  the  core 
entities  (concepts  or  classes)  and  their  rela¬ 
tionships  in  the  problem  domain;  these  dia- 
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grams  can  also  be  used  to  express  a  domain 
ontology  or  to  design  the  data  structures  in 
an  application  (see  Figure  2  of  Koch  and 
Wirsing,  2002). 

•  Activity  Diagrams  are  used  to  model  the  pro¬ 
cesses  or  workflow  in  a  system  (see  Figure 
3  of  Derntl  and  Motschnig-Pitrik,  2003). 

The  UML  diagrams  are  shared  among  researchers  in 
publications,  and  they  support  collaboration  and  com¬ 
munication  during  the  process  of  analysis  and  design. 
They  are  also  used  in  group  communication  to  create 
a  conceptual  model  that  integrates  different  perspec¬ 
tives.  An  example  is  the  UML  that  was  developed  to 
integrate  the  classical  and  modern  views  on  assessment 
(Joosten-Ten  Brinke  et  al.,  2005). 

Based  on  the  UML  models,  OSS  can  be  developed 
implementing  OS.  OSS/OS  can  be  very  beneficial  to 
the  project,  as  new  systems  can  be  built  by  reusing 
OSS  code  or  by  adapting  the  code  of  existing  systems. 
The  use  of  OS  makes  the  inclusion  of  existing  services 
or  data  possible  .  Another  advantage  of  using  OSS/OS 
is  that  it  can  facilitate  the  evaluation  of  the  system  in 
various  ways.  The  bazaar  methods  of  OSS  can  offer 
advantages  with  regard  to  the  quality  of  the  code.  To 
paraphrase  Raymond  (2001),  release  early  and  often; 
involve  the  users;  and  remember  that  many  eyeballs 
tame  complexity  and  that,  given  enough  eyeballs,  all 
bugs  are  shallow.  Furthermore,  it  offers  advantages  in 
the  setup  of  the  experiments  themselves.  When  the 
software  is  made  available  and  a  user  base  is  develop¬ 
ing,  these  same  users  can  be  used  for  the  evaluation 
research.  This  type  of  research  can  aim  for  high  eco¬ 
logical  validity  (Gilbert  and  Troitzsch,  1999). 

Another  advantage  of  using  OSS/OS  in  e-learning 
research  is  easy  dissemination  through  channels  such 
as  SourceForge.  This  provides  a  natural  means  to 
attract  new  initiatives,  to  improve  the  software,  and  to 
use  the  software  when  it  is  at  a  certain  quality.  A  basic 
requirement  is  that  the  product  addresses  a  need  per¬ 
ceived  by  a  community  of  users. 

OSS/OS  to  Facilitate  the  Development 
of  Technological  Knowledge 

One  of  the  results  of  technological  activities  is  the 
development  of  new  technological  knowledge.  Tech¬ 
nological  knowledge  is  knowledge  that  describes  how 
an  artifact  (or  a  system)  can  be  made  and  how  this 
artifact  can  be  used.  This  knowledge  is  captured  in: 

•  UML  diagrams  that  are  used  to  design  the 
system 

•  Code  of  the  system  that  has  been  developed 


•  Documentation  of  the  system  that  provides 
a  user  perspective  of  the  system 

•  Publications  about  the  evaluation  of  the  sys¬ 
tem 

When  using  an  OSS  approach,  this  knowledge  is  avail¬ 
able  for  all  researchers  to  test,  replicate,  or  elaborate. 
In  closed  source  systems  or  in  regular  research 
approaches,  the  access  to  the  design  and  the  code  of 
the  system  is  restricted,  so  the  knowledge  is  only  par¬ 
tially  available.  The  OSS  approach  can  lead  to  more 
convergence  of  knowledge  in  the  field  when  research¬ 
ers  adopt  the  habit  of  using  and  adapting  what  is 
already  available,  instead  of  building  everything  again 
from  scratch. 

The  use  and  development  of  OS  has  another 
advantage  in  research.  OS  can  be  considered  to  be 
consolidated,  agreed-upon  knowledge  about  the  data 
structure,  functionality,  or  semantics  of  a  system. 
Standards  commissions  provide  a  platform  to  con¬ 
verge  divergent  theories  and  models  to  the  best  pos¬ 
sible  abstraction  of  the  current  state  of  the  art;  for 
example,  the  IMS  QTI  specification  summarizes  the 
set  of  various  test  items  that  are  frequently  used  in 
education.  This  process  itself  is  a  strong  means  in  the 
field  to  come  to  an  agreement  and  summary  of  some 
aspects  in  the  field,  especially  when  researchers  are 
in  the  habit  of  contributing  to  the  standards  commit¬ 
tees,  use  the  standards  that  are  released  as  much  as 
possible,  and  test  the  specifications  to  identify  strong 
and  weak  points. 

OSS/OS  to  Facilitate  the  Development 
of  Technological  Artifacts 

Besides  the  creation  of  knowledge,  the  core  result  of 
technological  activity  is  the  development  of  the  tech¬ 
nological  artifacts.  In  e-leaming  research,  we  produce 
the  following  types  of  artifacts:  models  and  open  stan¬ 
dards,  software,  and  documentation.  In  the  previous 
paragraphs,  many  examples  of  the  OSS/OS  artifacts 
were  mentioned— for  example,  specifications  such  as 
IMS  QTI  and  IEEE  LOM  and  software  such  as  Moo- 
dle,  LionS hare,  and  Sakai. 


EXAMPLE 

An  example  of  the  development  and  use  of  the  open 
standard  IMS  Learning  Design  (IMSLD,  2003;  Koper 
and  Olivier,  2004;  Koper  and  Tattersall,  2005)  and  the 
related  OSS  will  demonstrate  the  use  of  OSS/OS  in 
e-leaming  research. 
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Figure  30.1  The  conceptual  domain  model  of  IMS  learning  design  in  UML. 


The  IMS  Learning  Design  Model 
(Technological  Knowledge) 

IMS  Learning  Design  is  an  open  standard  that  is  used 
to  specify  the  design  of  a  teaching  and  learning  process 
in  a  machine-interpretable  way.  LD  can  be  seen  as  a 
formal  instructional  design  language.  The  specification 
consists  of  a  set  of  documents  and  an  XML  Schema 
Definition  that  supports  the  coding  of  the  learning 
design  of  courses  in  XML  format.  At  the  base  of  the 
specification  is  the  Conceptual  Model  as  presented  in 
Figure  30.1.  This  model  is  specified  as  a  UML  class 
model  and  is  slightly  adapted  from  the  EML  model 
that  has  been  developed  by  studying  and  abstracting 
different  instructional  design  approaches  (Koper  and 
Manderveld,  2004).  It  was  tested,  discussed,  and 
adapted  in  different  situations  using  different  technol¬ 
ogies  (Tattersall  et  al.,  2005;  Van  Es  and  Koper,  2005), 
and  this  process  is  continuing,  although  a  first  stable 
release  of  the  open  specification  occurred  after  5  years 
of  work  (1998-2002).  The  model  itself  can  be  seen  as 
a  technological  theory  that  describes  a  large  variety  of 
instructional  design  approaches.  In  this  way,  it  serves 
as  a  convergence  mechanism  in  the  field  of  instruc¬ 
tional  design.  In  essence,  it  should  be  possible  to 
describe  and  implement  most  of  the  current  instruc¬ 
tional  design  models  (Reigeluth,  1999)  in  LD.  The  fact 
that  this  is  an  open  specification  has  invoked  many 
different  research  initiatives  in  the  area  of  ontologies 


for  learning  design,  patterns  in  learning  design,  runt¬ 
ime  adaptations,  and  the  design  and  development  of  a 
variety  of  authoring  and  runtime  tools  (Koper,  2005). 

IMS  Learning  Design  Open  Source  Software 

The  availability  of  LD  as  an  open  standard  has  stim¬ 
ulated  the  development  of  many  tools  to  support  the 
specification.  Most  of  these  tools  are  developed  as  OSS 
in  academic  settings— for  example  ,  as  part  of  doctorate 
study.  Griffiths  and  colleagues  (2005)  have  provided 
an  overview  of  available  tools  and  their  classification. 
Major  research  issues  are  the  development  of  more 
user-friendly  and  integrated  authoring  and  runtime 
tools  (Hernandez-Leo  et  al.,  2006) ,  Authoring  of  adap¬ 
tive  learning  designs  (Van  Rosmalen  et  al.,  2005),  the 
graphical  representation  of  learning  designs  (Paquette 
et  al.,  2006),  the  integration  of  assessment  in  learning 
designs  (Joosten-ten  Brinke  et  al.,  2005;  Pacurar  et  al., 
2004) ,  and  the  use  of  Semantic  Web  tools  with  learning 
design  (Amorim  et  al.,  2006;  Knight  et  al.,  2006). 


CONCLUSION  AND 
SOME  PRACTICAL  TIPS 

The  concepts  of  OSS  and  OS  have  been  elaborated  in 
this  chapter.  The  discussion  suggests  that  OSS  and  OS 
can  improve  the  convergence  of  knowledge  in  the  field. 
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improve  the  general  quality  and  interoperability  of  e- 
learning  applications,  and  improve  collaboration 
between  researchers  and  users.  How  might  one  use 
OSS  and  OS  in  research?  I  conclude  with  this  summary 
of  key  points  in  response  to  this  question: 

•  Learn  how  to  read  and  create  UML  diagrams 
to  specify  a  system  that  solves  some  real  prob¬ 
lems  in  the  fields  of  learning  and  teaching. 

•  Study  the  existing  open  standards  in  the  e- 
leaming  field,  specifically  the  ones  from  IMS 
and  IEEE  (and  ISO  as  they  become  avail¬ 
able).  It  is  necessary  to  have  some  knowledge 
about  XML  (2003),  RDF  (2003),  and  stan¬ 
dards  organizations.  Readers  should  refer  to 
the  references  provided  in  this  chapter. 

•  Download,  install,  test,  and  use  a  variety  of 
open  source  programs  and  distributions. 
Consider  the  use  of  virtual  machine  soft¬ 
ware,  such  as  VMware,  which  allows  users 
to  install  various  Linux  distributions  (e.g., 
Edubuntu)  without  affecting  other  programs 
on  their  computers. 

•  Learn  to  code  and  participate  in  an  open 
source  development  project  or  start  one  on 
your  own. 

•  Use  existing  OSS  as  much  as  possible  in 
your  work  and  be  strict  in  the  use  of  OS 
where  possible  to  encourage  convergence 
and  collaboration. 

•  Participate  in  relevant  communities;  demon¬ 
strate  your  work  in  workshops;  write  confer¬ 
ence  papers  and  journal  papers  about  your 
work;  provide  relevant  feedback  and  input 
to  standards  committees. 
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Part  IV 

Models 


This  fourth  part  of  the  Handbook  was  led  by  Jeroen 
J.G.  van  Merrienboer  and  focuses  on  issues  concerned 
with  various  types  of  and  approaches  to  learning. 
These  models  of  learning  clearly  inform  design  and 
development  and  can  be  linked  to  various  instruc¬ 
tional  strategies  covered  in  Part  II.  This  part  includes 
general  models  directed  toward  learning  in  schools, 
general  models  directed  toward  learning  outside 
schools  and  at  the  workplace,  and  models  that  focus 
on  learning  in  specific  domains  such  as  medicine, 
science,  and  reading.  This  part  of  the  Handbook  con¬ 
sists  of  11  chapters  covering  these  topics:  (1)  human 


cognitive  architecture;  (2)  outcomes-oriented  models; 
(3)  cooperative  learning  models;  (4)  cognitive  appren¬ 
ticeship  approaches;  (5)  whole-task  models;  (6) 
model-facilitated  learning;  (7)  adaptive  instructional 
systems;  (8)  problem-based  learning;  (9)  performance 
improvement  approaches;  (10)  resource-based  train¬ 
ing;  and  (11)  domain-specific  approaches.  The  final 
chapter  in  this  section  is  one  of  the  chapters  in  this 
Handbook  that  consists  of  multiple  sections  with  dif¬ 
ferent  authors  (all  authors  are  recognized  as  chapter 
coauthors  and  section  authors  are  recognized  where 
appropriate). 
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ABSTRACT 

Cognitive  load  theory  integrates  the  origins  of  human 
cognition  in  evolutionary  theory  with  the  structures 
and  functions  of  human  cognitive  architecture  to  pro¬ 
vide  effective  instructional  design  principles.  Many  of 
those  principles  are  directly  relevant  to  instructional 
technology.  This  chapter  outlines  the  evolutionary 
bases  for  human  cognitive  architecture,  examines 
those  aspects  of  human  cognition  that  are  directly  rel¬ 
evant  to  instruction,  and  discusses  the  various  instruc¬ 
tional  principles  generated  by  cognitive  load  theory 
with  specific  reference  to  their  applicability  to  instruc¬ 
tional  technology. 


KEYWORDS 

Cognitive  load  theory:  An  instructional  design  theory 
based  on  our  knowledge  of  human  cognitive  archi¬ 
tecture. 

Human  cognitive  architecture :  The  manner  in  which 
structures  and  functions  required  for  human  cog¬ 
nitive  processes  are  organized. 

Long-term  memory:  The  store  holding  all  knowledge 
acquired  during  the  processes  of  learning. 

Natural  information  processing  system:  The  proce¬ 
dures  by  which  natural  systems  such  as  human 
cognition  and  evolution  by  natural  selection  pro¬ 
cess  information. 

Working  memory:  The  structure  that  processes  infor¬ 
mation  coming  from  either  the  environment  or 
long-term  memory  and  that  transfers  learned  infor¬ 
mation  for  storage  in  long-term  memory. 


INTRODUCTION 

The  extent  to  which  any  instruction  is  effective 
depends  heavily  on  whether  it  takes  the  characteristics 
of  human  cognition  into  account.  To  determine  the 
conditions  that  maximize  learning,  we  need  to  closely 
study  human  cognition.  Once  we  have  established  the 
mechanisms  of  human  cognition,  including  why  those 
mechanisms  have  their  particular  characteristics,  we 
are  in  a  position  to  design  learning  environments  in 
accord  with  human  cognitive  architecture.  Ideal  learn¬ 
ing  environments  in  accord  with  human  cognitive 
architecture  are  not  always  in  accord  with  realistic 
learning  environments  that  mimic  the  real  world. 
Indeed,  if  human  cognition  was  organized  in  a  manner 
that  always  favored  learning  in  realistic  over  artificial 
environments,  there  would  never  be  a  need  for  special¬ 
ized  instructional  procedures  or  even  for  educational 


institutions.  Observing  and  interacting  with  the  world 
would  provide  the  best  educational  environment. 
Under  such  conditions,  all  instruction  would  have  to 
be  as  realistic  as  possible  and  would  only  be  required 
for  economic  or  safety  reasons  rather  than  for  educa¬ 
tional  reasons.  In  contrast,  it  is  suggested  in  this  chap¬ 
ter  that,  because  of  the  characteristics  of  human  cog¬ 
nition,  in  many  conditions  learning  is  best  facilitated 
by  instruction  that  does  not  accurately  model  reality. 

Cognitive  load  theory  can  be  used  to  determine 
some  of  the  characteristics  of  effective  instruction. 
This  integrated  theory  is  intended  to  provide  a  system¬ 
atically  organized  hierarchy  leading  from  evolutionary/ 
biological  reasons  for  the  characteristics  of  human 
cognitive  architecture  to  the  instructional  conse¬ 
quences  that  flow  from  that  architecture.  Only  some 
of  the  instructional  procedures  generated  by  cognitive 
load  theory  are  relevant  to  educational  technology,  and 
those  procedures  are  emphasized  in  this  chapter.  We 
begin  by  discussing  human  cognitive  architecture 
within  an  evolutionary  and  biological  framework. 


HUMAN  COGNITIVE  ARCHITECTURE 

Human  cognition  has  evolved  to  assimilate,  process, 
and  use  information  (or  knowledge,  used  synony¬ 
mously  in  this  chapter  when  dealing  with  human  cog¬ 
nition)  to  direct  human  action.  It  constitutes  an  exam¬ 
ple  of  a  natural  information  processing  system  that  is 
a  class  of  information  processing  systems  that  can  be 
found  in  nature.  As  a  natural  information  processing 
system,  human  cognition  is  hardly  likely  to  be  unique 
and,  indeed,  evolution  by  natural  selection  can  itself 
be  classed  as  a  natural  information  processing  system 
(Sweller,  2003,2004;  Sweller  and  Sweller,  2006).  The 
characteristics  of  such  systems  will  vary  depending  on 
their  functions,  but  all  natural  information  processing 
systems  share  an  identical  basic  structure  or  frame¬ 
work.  That  basic  structure  in  turn  can  be  used  to  deter¬ 
mine  how  humans  deal  with  information  and  what 
types  of  instructional  procedures,  including  technol¬ 
ogy-dependent  instruction,  are  likely  to  be  effective. 

The  essential  characteristics  of  a  natural  informa¬ 
tion  processing  system  include:  (1)  a  very  large  store 
of  information  that  allows  the  system  to  function  in 
the  varied  environments  faced  by  natural  information 
processing  systems;  (2)  procedures  for  perpetuating 
the  store  of  information  by  transferring  information 
from  one  entity  to  another;  (3)  procedures  for  changing 
the  store  by  creating  new  information  to  deal  with  a 
changing  environment;  (4)  procedures  to  ensure  that 
changes  to  the  store  do  not  destroy  its  effectiveness; 
and  (5)  procedures  to  relate  information  to  the  external 
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world.  These  core  characteristics  as  applied  to  human 
cognition  are  discussed  here  in  terms  of  live  principles 
(Sweller  and  Sweller,  2006). 

The  Information  Store  Principle 
and  Human  Long-Term  Memory 

Human  long-term  memory  provides  the  human  cogni¬ 
tive  system  with  the  large  store  of  information  indis¬ 
pensable  to  a  natural  information  system.  A  species’ 
genome  has  a  similar  function  in  biological  evolution. 
Our  knowledge  of  the  function  of  long-term  memory 
has  altered  over  time,  and,  arguably,  specifying  the  role 
of  long-term  memory  constitutes  the  primary  finding  of 
the  cognitive  science  revolution.  We  no  longer  see  long- 
term  memory  as  a  repository  of  isolated,  unrelated  facts 
that  are  occasionally  stored  and  retrieved;  instead,  it  is 
the  central  structure  of  human  cognitive  architecture  . 

Origins,  Evidence,  and  Implications 

De  Groot’s  (1946/1965)  work  on  expertise  in  the  game 
of  chess  can  be  seen  as  the  initial  work  transforming 
our  perception  of  the  role  of  long-term  memory.  De 
Groot  found  that,  if  chess  masters  are  shown  a  board 
configuration  taken  from  a  real  game  for  about  5  sec¬ 
onds,  they  can  reproduce  it  much  more  accurately  than 
weekend  players.  Chase  and  Simon  (1973)  reproduced 
this  finding  but  also  found  that  masters  and  weekend 
players  did  not  differ  in  their  ability  to  reproduce  ran¬ 
dom  board  configurations.  This  difference  between 
experts  and  novices  in  memory  for  real  configurations 
and  situations  has  been  replicated  in  a  variety  of  fields 
(Egan  and  Schwartz,  1979;  Jeffries  et  al.,  1981; 
Sweller  and  Cooper,  1985).  Furthermore,  it  is  the  only 
reliable  difference  that  has  been  obtained  differentiat¬ 
ing  novices  and  experts  in  problem-solving  skill  and 
is  the  only  difference  required  to  fully  explain  why  an 
individual  is  an  expert  in  solving  particular  classes  of 
problems.  A  chess  master  has  learned  to  recognize 
many  thousands  of  board  configurations.  When  faced 
with  a  configuration,  he  or  she  recognizes  it  and  knows 
the  best  move  to  make  given  that  configuration.  We  all 
have  that  skill  in  our  own  areas  of  expertise  and, 
indeed,  the  ability  to  recognize  situations  and  the 
appropriate  actions  that  they  require  constitutes  our 
skill  base  in  a  given  area. 

Several  implications  flow  from  these  findings. 
First,  problem-solving  skill  is  domain  specific.  A  bril¬ 
liant  mathematician  has  acquired  mathematical  prob¬ 
lem-solving  skills  that  are  unlikely  to  transfer  to,  for 
example,  financial  or  personal  relationship  skills.  The 
skills  allow  a  mathematician  to  recognize  mathemati¬ 
cal  problem  states  and  the  best  moves  associated  with 


them.  Such  skills  are  only  useful  in  a  mathematical 
context.  Second,  the  work  on  novice-expert  differ¬ 
ences  explains  why  it  takes  so  long  to  become  an 
expert  in  a  substantial  field.  It  takes  about  10  years  of 
concentrated  work  for  a  person  to  become  a  chess 
grand  master  (Simon  and  Gilmartin,  1973).  During  that 
time,  the  person  is  not  learning  complicated,  general 
problem-solving  strategies.  There  is  no  evidence  that 
leamable  or  teachable  general  problem-solving  strate¬ 
gies  exist;  rather,  the  chess  player  is  learning  to  rec¬ 
ognize  the  huge  number  of  board  configurations 
required  for  chess  expertise,  just  as  all  of  us  are 
required  to  learn  the  huge  number  of  problem  situa¬ 
tions  required  to  acquire  problem-solving  skill  in  a 
particular  area.  Third,  data  on  novice-expert  differ¬ 
ences  demonstrate  the  central  importance  of  knowl¬ 
edge  or  information  held  in  long-term  memory  to  skill 
in  any  area  (Chi  et  al.,  1982).  To  be  skillful,  we  must 
have  a  huge  stock  of  knowledge  available.  Based  on 
this  conceptualization,  long-term  memory  is  critical  to 
skilled  performance.  Knowledge  held  in  long-term 
memory  allows  us  to  function  in  the  variety  of  contexts 
in  which  we  find  ourselves  in  much  the  same  way  as 
a  genome  allows  a  species  to  function  biologically. 
Accordingly,  a  major  function  of  instruction,  including 
technology-based  instruction,  is  to  ensure  that  appro¬ 
priate  knowledge  is  held  in  long-term  memory. 

The  Borrowing  Principle 
and  Transferring  Knowledge 

Once  discovered,  information  held  by  natural  informa¬ 
tion  processing  systems  must  be  perpetuated,  and  a 
primary  function  of  education  is  to  ensure  that  knowl¬ 
edge  is  not  lost  because,  as  will  be  discussed  below, 
there  are  structural  reasons  why  the  act  of  discovery 
is  inordinately  difficult.  In  genetics,  the  store  of  DNA- 
based  information  is  perpetuated  over  long  periods  of 
time  by  sexual  and  asexual  reproduction.  Asexual 
reproduction  includes  precise  copying  from  one  gen¬ 
eration  to  the  next  and  appears  to  have  no  cognitive 
equivalent.  In  contrast,  sexual  reproduction  is  a  con¬ 
structive  procedure  designed  to  ensure  that  offspring 
differ  from  their  parents.  This  process  has  an  inevitable 
random  component,  with  the  precise  combination  of 
genetic  material  from  male  and  female  ancestors  being 
intrinsically  unpredictable. 

Psychological  mechanisms  are  required  equally  to 
preserve  information  held  in  long-term  memory  by  its 
transmission  from  individual  to  individual.  Accordingly, 
we  have  evolved  to  both  efficiently  transmit  and  receive 
information  from  other  humans  in  either  auditory  or 
visual  form.  We  imitate  what  others  do,  listen  to  what 
they  say,  and  read  what  they  write.  That  skill  permits 
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knowledge  held  in  an  individual’s  long-term  memory  to 
be  perpetuated  indefinitely.  Nevertheless,  as  is  the  case 
with  sexual  reproduction,  the  process  is  constructive  in 
nature  .  We  combine  new  information  with  information 
previously  stored  in  long-term  memory  (Bartlett,  1932). 
As  is  also  the  case  with  sexual  reproduction,  the  process 
has  a  random  component  in  that  we  cannot  predict  pre¬ 
cisely  how  the  two  sources  of  information  will  be  com¬ 
bined.  This  randomness  has  implications  for  the  narrow 
limits  of  change  principle  discussed  below. 

Origins,  Evidence,  and  Implications 

A  major  function  of  instruction  is  to  organize  efficient 
procedures  that  will  permit  knowledge  to  be  trans¬ 
ferred  to  the  long-term  memories  of  learners.  Cogni¬ 
tive  load  theory  has  generated  a  variety  of  instructional 
procedures  relevant  to  technology-based  instruction 
(see  below) .  All  are  dependent  on  the  borrowing  prin¬ 
ciple.  The  success  of  cognitive  load  theory  in  gener¬ 
ating  instructional  procedures  is,  at  least  in  part,  due 
to  its  emphasis  on  learning  from  presented  informa¬ 
tion.  That  success  provides  some  of  the  evidence  for 
the  borrowing  principle. 

In  earlier  eras,  the  suggestion  that  a  function  of 
education  is  to  transfer  knowledge  would  have  been 
considered  self-evident.  More  recently,  an  emphasis 
on  discovery  and  constructivist  procedures  encouraged 
learners  to  discover  knowledge  rather  than  to  have 
instructors  transmit  knowledge.  The  impossibility  of 
discovering  even  a  tiny  fraction  of  the  huge  amount  of 
information  required  in  the  modem  world  was  partially 
obscured  by  the  field’s  ignorance  of  the  massive  size 
of  long-term  memory.  That  ignorance  is  being  rectified 
and  increasing  numbers  of  investigators  are  reacting 
against  the  previous  orthodoxy  (Kirschner  et  al.,  2006; 
Klahr  and  Nigam,  2004;  Mayer,  2004). 

Our  increasing  knowledge  of  the  importance  of 
imitation  in  human  learning  provides  additional  evi¬ 
dence  for  the  borrowing  principle.  The  discovery  of 
mirror  neurons  that  fire  in  the  same  manner  when  we 
take  an  action,  observe  someone  else  make  the  same 
action,  or  even  listen  to  a  sentence  describing  the 
action  provides  neuropsychological  evidence  for  the 
importance  of  imitation  as  a  learning  mechanism  (Tet- 
tamanti  et  al.,  2005). 

Randomness  as  Genesis  Principle 
and  Creating  Knowledge 

The  need  to  transfer  knowledge  should  not  obscure 
the  fact  that  knowledge  must  first  be  created  to  have 
something  to  transfer.  In  addition,  circumstances 
change  and  current  knowledge  may  no  longer  be  ade¬ 


quate.  The  procedure  for  changing  the  store  of  infor¬ 
mation  held  by  natural  information  processing  sys¬ 
tems  is  standard.  All  creative  changes  are  random, 
but  only  effective  changes  are  retained,  and  ineffec¬ 
tive  changes  are  jettisoned.  In  evolution  by  natural 
selection,  this  mechanism  is  well  accepted.  Random 
mutation  is  the  initial  source  of  all  genetic  variation 
with  only  successful  mutations  resulting  in  reproduc¬ 
tion  and  continuation.  In  psychology,  and  especially 
instructional  psychology,  the  suggestion  requires  fur¬ 
ther  explanation. 

Random  generation  followed  by  effectiveness  test¬ 
ing  is  unavoidable  in  functioning,  natural  information¬ 
processing  systems.  Consider  a  person  solving  a  novel 
problem,  an  activity  intended  to  create  a  solution  that 
is  new  for  that  person.  When  deciding  on  a  problem¬ 
solving  move  or  a  series  of  moves,  the  two  basic  cat¬ 
egories  of  move  generators  are  random  generation  and 
generation  based  on  previous  knowledge.  These  two 
categories  or  a  combination  of  these  two  categories  are 
the  only  source  of  move  generation.  Previously 
acquired  knowledge  held  in  long-term  memory  can  act 
as  a  central  executive  determining  moves,  but,  of 
course,  that  knowledge  is  not  always  available.  There 
is  no  general  central  executive  that  can  determine 
moves  in  the  absence  of  relevant  knowledge.  Random 
generation,  either  mentally  or  in  the  real  world,  is  left 
as  the  only  other  alternative. 

Origins,  Evidence,  and  Implications 

The  origin  of  this  principle  is  largely  logical  rather  than 
empirical.  If  knowledge  to  determine  a  problem-solv¬ 
ing  move  is  unavailable,  random  generation  followed 
by  tests  for  effectiveness  is  the  only  other  option.  We 
can  see  the  impact  of  this  logic  in  computer  models  of 
problem  solving.  Such  models  (Sweller,  1988)  ran¬ 
domly  generate  moves  if  information  is  lacking. 

Does  random  generation  mean  that  whether  or  not 
a  novel  problem  is  solved  is  entirely  due  to  chance? 
Random  generation  can  only  function  properly  when 
coupled  with  effectiveness  testing,  and  the  combina¬ 
tion  of  random  generation  and  effectiveness  testing 
provides  the  knowledge-  or  information-creating  pro¬ 
cess  of  natural  information-processing  systems.  By 
only  accepting  effective  alterations  to  an  information 
store  and  rejecting  ineffective  alterations,  the  effective¬ 
ness  of  the  store  can  increase  incrementally  over  time. 
By  transferring  the  information  in  a  store  to  other 
entities  that  can  continue  the  process  of  random  gen¬ 
eration  followed  by  effectiveness  testing  indefinitely, 
very  complex,  sophisticated  stores  can  by  built  by  nat¬ 
ural  information-processing  systems.  Both  evolution 
by  natural  selection  and  human  cognition  provide 
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examples.  Nevertheless,  it  must  be  recognized  that 
random  generation  followed  by  effectiveness  testing  is 
a  very  slow  way  of  generating  knowledge  (Cooper  and 
Sweller,  1987;  Sweller  and  Cooper,  1985)  and  should 
only  be  used  when  the  alternative  procedure  of  knowl¬ 
edge  transfer  is  unavailable.  Knowledge  transfer  is 
vastly  more  effective.  Accordingly,  technology-based 
instruction  requiring  problem  solving  (random  gener¬ 
ation  followed  by  effectiveness  testing)  can  be 
expected  to  be  less  effective  than  instruction  providing 
demonstrations  (knowledge  transfer  via  the  borrowing 
principle) . 

The  Narrow  Limits  of  Change 
and  Human  Working  Memory 

There  are  limits  to  knowledge  creation.  In  the  initial 
instance,  knowledge  creation  occurs  using  the  random¬ 
ness  as  genesis  principle,  which  requires  random  gen¬ 
eration  followed  by  effectiveness  testing.  Knowledge 
may  be  altered  using  the  borrowing  principle  but  that 
also  involves  random  generation  when  new  informa¬ 
tion  is  combined  with  old  information.  Mechanisms 
are  required  to  ensure  that  any  alterations  to  the  store 
of  information  in  long-term  memory  maximize  the 
probability  that  a  particular  change  will  be  effective 
and  minimize  the  probability  that  a  change  will  destroy 
the  functionality  of  the  store.  These  conditions  are  met 
by  the  use  of  a  series  of  small,  incremental  changes, 
each  tested  for  effectiveness,  rather  than  a  single,  very 
large  change. 

Assume  that  the  effectiveness  is  being  tested  of 
adding  a  permutation  of  4  elements  to  the  store. 
Assume  further  that  no  knowledge  is  available  to  elim¬ 
inate  some  of  the  possible  permutations  or  select  others 
for  priority  testing.  Under  these  circumstances,  there 
are  4!  =24  possible  permutations.  In  contrast,  assume 
that  rather  than  4  elements  being  under  consideration 
there  are  10.  The  number  of  possible  permutations  is 
10!  =  3,628,800.  If  only  a  limited  number  or  perhaps 
only  one  permutation  is  actually  effective,  dealing  with 
more  than  a  very  small  number  of  elements  that  must 
be  combined  randomly  and  tested  for  effectiveness  is 
futile  because  the  presence  of  any  more  elements  rap¬ 
idly  leads  to  a  combinatorial  explosion  that  no  natural 
information  processing  system  can  possibly  handle 
unless  it  has  prior  knowledge. 

Origins,  Evidence,  and  Implications 

This  factor  alone  explains  some  of  the  major  charac¬ 
teristics  of  human  cognitive  architecture.  When  deal¬ 
ing  with  novel  information  for  which  there  is  no  or 
limited  prior  knowledge,  human  working  memory  is 


extremely  limited  in  capacity  (Miller,  1956).  Although 
the  precise  nature  of  this  limitation  has  provided  a 
source  of  research  and  discussion  (Cowan,  2005),  few 
would  dispute  the  existence  of  the  limitation.  The  con¬ 
sequences  of  this  limitation  have  profound  conse¬ 
quences  for  instructional  design.  Instructional  designs 
that  ignore  this  limitation  are  likely  to  be  ineffective. 

The  Environment  Organizing 
and  Linking  Principle 

This  principle  pertains  to  how  we  use  information  to 
function  in  our  environment.  In  contrast  to  the  limita¬ 
tions  of  working  memory,  when  dealing  with  familiar 
information  that  is  already  organized  in  long-term 
memory,  there  is  no  functional  reason  for  working 
memory  to  be  limited.  Accordingly,  huge  amounts  of 
organized  information  can  be  transferred  from  long¬ 
term  memory  to  working  memory  without  overloading 
working  memory  (Ericsson  and  Kintsch,  1995).  That 
information  can  then  be  used  to  allow  us  to  function 
in  our  complex  environment;  for  example,  if  readers 
attempt  to  reproduce  from  memory  the  immensely 
complex  set  of  squiggles  that  constitute  the  last  sen¬ 
tence,  most  will  be  able  to  do  so  effortlessly.  We  are 
able  to  do  so  because  of  the  organized,  schematic 
information  held  in  long-term  memory  that  can  be 
transferred  and  used  in  working  memory.  That  infor¬ 
mation  allows  us  to  read,  process,  organize,  and  relate 
text  to  the  external  world.  Similarly,  whereas  changes 
to  a  genome  must  be  small  and  incremental,  huge 
amounts  of  previously  organized  genetic  information 
can  be  used  simultaneously  to  produce  the  complex 
proteins  required  for  biological  survival  in  a  complex 
environment. 

Origins,  Evidence,  and  Implications 

This  principle  has  multiple  origins.  Miller’s  (1956) 
evidence  that  multiple  elements  could  be  “chunked” 
together  to  act  as  a  single  element  in  working  memory, 
schema  theorists  demonstrating  that  the  manner  in 
which  we  process  information  depends  on  previous 
knowledge  (Bartlett,  1932),  and,  more  recently,  Erics¬ 
son  and  Kintsch’s  (1995)  concept  of  long-term  work¬ 
ing  memory  all  contribute  to  the  environment  organiz¬ 
ing  and  linking  principle  .  Such  lines  of  research  dem¬ 
onstrate  how  we  use  our  large  store  of  information  to 
impose  order  and  meaning  on  our  environment.  The 
environment  organizing  and  linking  principle  provides 
the  ultimate  justification  for  human  cognition.  The  pur¬ 
pose  of  the  previous  four  principles  is  to  permit  cog¬ 
nition  to  occur  so  we  can  function  mentally  in  our 
environment. 
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COGNITIVE  LOAD  THEORY 

These  characteristics  of  human  cognitive  architecture 
have  direct  implications  for  instructional  design  and 
hence  the  design  and  purpose  of  technology-based 
instruction.  The  preferred  characteristics  of  technol¬ 
ogy-based  instruction  can  be  determined  by  using  cog¬ 
nitive  load  theory  (Clark  et  al.,  2006;  Paas  et  al.,  2003, 
2004;  Sweller,  2005a;  van  Merrienboer  and  Sweller, 
2005),  as  well  as  closely  related  theories  such  as 
Mayer’s  theory  of  multimedia  learning  (Mayer,  2005) 
or  van  Merrienboer’s  (1997)  4C/ID  model,  which  use 
this  architecture  to  determine  general  instructional 
design  principles.  As  might  be  expected,  all  theories 
that  use  this  architecture  are  compatible  and  make 
similar  or  identical  predications. 

Cognitive  load  theory  specifies  two  sources  of 
cognitive  (or  working  memory)  load  that  determine 
the  effectiveness  of  instruction.  Extraneous  cognitive 
load  is  due  to  inappropriate  instructional  designs  and 
so  must  be  reduced.  If  working  memory  resources 
are  being  fully  expended,  a  reduction  in  extraneous 
cognitive  load  is  required  to  permit  an  increase  in 
germane  cognitive  load,  a  form  of  cognitive  load  that 
can  result  in  useful  alterations  to  long-term  memory. 
These  two  sources  of  cognitive  load  determine  the 
effectiveness  of  instruction,  but  a  third  source  of  cog¬ 
nitive  load,  intrinsic  cognitive  load,  cannot  be  manip¬ 
ulated  without  compromising  understanding.  Intrin¬ 
sic  cognitive  load  (Sweller,  1994)  can  be  thought  of 
as  the  intrinsic  complexity  of  the  material  being  stud¬ 
ied.  For  learners  at  a  given  level  of  expertise,  that 
complexity  can  be  reduced  (Pollock  et  al.,  2002)  but 
only  by  reducing  learners’  understanding  of  the  sub¬ 
ject  matter.  Intrinsic  cognitive  load  can,  of  course, 
also  be  reduced  by  learning  (i.e.,  by  testing  learners 
with  more  expertise). 

These  sources  of  cognitive  load  are  additive  and 
cannot  exceed  the  available  capacity  of  working  mem¬ 
ory.  If  intrinsic  cognitive  load  is  low,  it  may  be  possible 
for  germane  cognitive  load  to  be  high  even  with  inap¬ 
propriate  instructional  techniques  causing  a  high  extra¬ 
neous  cognitive  load.  The  low  intrinsic  cognitive  load 
is  likely  to  leave  sufficient  working  memory  resources 
for  students  to  learn  even  with  a  poor  instructional 
design.  In  contrast,  if  intrinsic  cognitive  load  is  high 
due  to  high  complexity  material,  unless  the  extraneous 
cognitive  load  is  low  there  may  be  insufficient  working 
memory  capacity  to  permit  a  level  of  germane  cogni¬ 
tive  load  that  can  result  in  learning.  With  a  high  intrin¬ 
sic  cognitive  load,  it  is  essential  to  keep  the  extraneous 
cognitive  load  low  to  permit  a  sufficient  level  of  ger¬ 
mane  cognitive  load.  In  other  words,  instructional 
design  becomes  critical  with  complex  material. 


Based  on  the  cognitive  architecture  outlined  above, 
extraneous  cognitive  load  can  be  reduced  and  germane 
cognitive  load  increased  by  assisting  learners  to  transfer 
information  to  long-term  memory.  Novel  information 
coming  from  the  senses  is  not  organized  and  imposes 
a  heavy  load  on  working  memory  (the  narrow  limits  of 
change  principle) .  Information  coming  from  long-term 
memory  is  organized  and  imposes  a  minimal  load  on 
working  memory  (the  environment  organising  and  link¬ 
ing  principle) .  In  other  words ,  once  learned ,  information 
no  longer  imposes  a  working  memory  load.  How  should 
we  assist  learners  to  transfer  novel  information  to  long¬ 
term  memory?  Wherever  possible,  that  information 
should  be  in  an  organized  form  so  learners  do  not  have 
to  expend  working  memory  resources  in  imposing  an 
organizational  structure.  Almost  all  information  that 
learners  must  acquire  has  previously  been  laboriously 
organized  over  many  generations  using  the  randomness 
as  genesis  principle  outlined  above.  Nothing  is  gained 
by  requiring  learners  in  an  instructional  setting  to 
attempt  to  use  random  generation  followed  by  tests  of 
effectiveness  rather  than  the  borrowing  principle. 

Contrary  to  much  current  educational  dogma  (Kir- 
schner  et  al.,  2006;  Mayer,  2004)  and  as  indicated 
above,  our  cognitive  architecture  strongly  facilitates 
learning  through  knowledge  transfer  via  the  borrowing 
principle  rather  than  knowledge  generation  via  the  ran¬ 
domness  as  genesis  principle  .  Not  only  should  knowl¬ 
edge  be  presented  to  learners  rather  than  have  them 
engage  in  the  impossible  task  of  attempting  to  generate 
it  themselves,  but  it  should  also  be  presented  in  a 
manner  that  reduces  extraneous  cognitive  load  and 
maximizes  germane  cognitive  load.  Cognitive  load 
theory,  making  use  of  the  borrowing  principle,  has 
generated  a  range  of  techniques  intended  to  achieve 
this  purpose.  Most  of  those  techniques  are  relevant  to 
technology-based  instruction  and  will  be  discussed 
next.  Each  technique  has  been  studied  as  an  experi¬ 
mental  effect  using  randomized,  controlled  experi¬ 
ments  in  which  an  instructional  technique  generated 
by  cognitive  load  theory  is  compared  to  an  alternative, 
usually  more  traditional  technique.  These  effects  pro¬ 
vide  the  instructional  recommendations  generated  by 
cognitive  load  theory. 

Cognitive  Load  Theory  Effects 
and  Technology-Based  Instruction 

Worked  Example  Effect 

The  worked  example  effect  (Sweller  and  Cooper,  1985) 
occurs  when  novice  learners  studying  worked  solutions 
to  problems  perform  better  on  a  problem-solving  test  than 
learners  who  have  been  given  the  equivalent  problems 
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to  solve  during  training.  This  effect  flows  directly  from 
the  cognitive  architecture  described  above.  When  solv¬ 
ing  an  unfamiliar  problem,  problem  solvers  have  no 
choice  but  to  use  a  random  generation  followed  by 
effectiveness  testing  procedure  at  those  choice  points 
where  they  have  insufficient  knowledge  to  direct 
choice.  The  working  memory  load  associated  with 
using  an  effective  problem-solving  procedure  will 
interfere  with  learning  (Cooper  and  S weller,  1987; 
Paas  and  van  Merrienboer,  1994)  and  so  constitutes  an 
extraneous  cognitive  load.  In  contrast,  using  the  bor¬ 
rowing  rather  than  the  randomness  as  genesis  principle 
should  enhance  learning.  Showing  learners  how  to 
solve  problems  via  worked  examples  by  providing  an 
organized  structure  rather  than  leaving  learners  to 
devise  their  own  organization  using  their  limited  capac¬ 
ity  working  memory  should  minimize  extraneous  cog¬ 
nitive  load.  Findings  from  a  large  number  of  studies 
carried  out  by  many  researchers  in  the  1980s  and  1990s 
provided  strong  support  for  these  hypotheses. 

There  is  every  reason  to  suppose  that  the  worked 
example  effect  should  apply  directly  to  technology- 
based  instruction.  Consider  a  computer-based  simula¬ 
tion.  A  simulation  should  demonstrate  a  process  or 
procedure.  It  should  not  require  learners  to  solve  novel 
problems  even  if  problem  solving  is  part  of  the  process 
being  simulated.  The  aim  of  a  simulation  demonstrat¬ 
ing  problem  solving  should  be  identical  to  the  aim  of 
any  other  instruction  in  problem  solving,  and  that  aim 
is  to  assist  learners  in  acquiring  knowledge  in  long¬ 
term  memory  concerning  the  problem-solving  proce¬ 
dures  relevant  to  that  particular  problem.  Once  that 
knowledge  is  acquired,  learners  will  recognize  the 
problem  as  belonging  to  a  category  requiring  particular 
moves  for  solution  (Chi  et  al.,  1982).  Searching  for 
novel  solutions  using  random  generation  followed  by 
effectiveness  testing  as  part  of  a  simulation  is  no  more 
likely  to  achieve  this  aim  than  searching  for  novel 
solutions  in  any  other  instructional  context.  Using  the 
borrowing  principle,  a  good  simulation  can  eliminate 
randomness  by  precisely  indicating  the  structure  of 
what  needs  to  be  learned.  (Practicing  familiar  problem 
solutions  has  a  different  function  and  is  discussed 
below  in  the  section  on  the  expertise  reversal  effect.) 

Split-Attention  Effect 

Assume  that  the  material  being  presented  to  learners 
consists  of  two  or  more  sources  of  information.  Assume 
further,  that  the  sources  of  information  are  unintelligi¬ 
ble  in  isolation  and  can  only  be  understood  in  conjunc¬ 
tion  with  each  other.  A  geometry  worked  example  pro¬ 
vides  an  instance.  The  diagram  tells  novice  learners 
little  if  anything  of  the  problem  solution,  and  the  asso¬ 


ciated  statements  are  likely  to  be  unintelligible  without 
reference  to  the  diagram.  To  understand  the  material, 
learners  must  mentally  integrate  the  two  sources  of 
information.  Mental  integration  requires  working  mem¬ 
ory  resources  to  be  used  to  search  for  appropriate  ref¬ 
erences  between  the  multiple  sources  of  information. 
That  search  process  is  indistinguishable  from  the  ran¬ 
dom  generation  followed  by  effectiveness  testing  of 
problem  solving.  All  searches  not  based  on  knowledge 
require  the  randomness  as  genesis  principle  .  If  a  geom¬ 
etry  solution  includes  the  statement  “angle  ABC,”  then 
learners  must  randomly  choose  an  angle  and  test 
whether  it  is  angle  ABC,  with  the  process  continuing 
until  the  angle  is  found.  Little  can  be  expected  to  be 
learned  by  this  use  of  working  memory  resources.  In 
contrast,  borrowing  other  people’s  knowledge  can 
reduce  extraneous  cognitive  load.  If  the  instructions  are 
physically  integrated  so  “angle  ABC”  is  clearly  asso¬ 
ciated  with  the  appropriate  angle,  obviating  the  need  to 
search  for  the  angle,  then  extraneous  cognitive  load 
should  be  reduced  and  learning  enhanced.  That  result 
also  has  been  obtained  on  numerous  occasions  (Ayres 
and  Sweller,  2005;  Sweller  et  al.,  1990).  It  must  be 
emphasized  that  the  split-attention  effect  only  occurs 
when  multiple  sources  of  information  must  be  inte¬ 
grated  before  they  can  be  understood.  Multiple  sources 
of  information  that  can  be  understood  in  isolation 
should  not  be  physically  integrated.  Different  instruc¬ 
tional  procedures  are  required  when  multiple  sources 
of  information  can  be  understood  in  isolation  and  are 
discussed  in  the  Redundancy  Effect  section. 

Technology-based  instruction  that  ignores  the 
split-attention  effect  is  likely  to  be  less  than  effective; 
for  example,  a  simulation  demonstrating  the  functions 
of  a  mechanical  device  in  which  the  function  of  a  part 
of  the  device  can  only  be  understood  in  relation  to  the 
function  of  another  part  of  the  device  runs  the  risk  of 
split  attention.  Whenever  possible,  the  simulation 
should  be  structured  to  clearly  indicate  the  relation 
between  the  two  parts.  In  addition,  any  written  text 
should  be  formatted  in  a  manner  that  reduces  or  elim¬ 
inates  a  search  for  referents.  Learners  should  not  be 
left  to  work  out  relations  between  aspects  of  a  simu¬ 
lation.  Working  memory  resources  can  be  better 
employed. 

Modality  Effect 

This  effect  occurs  under  the  same  conditions  as  the 
split-attention  effect  in  that  both  effects  occur  under 
conditions  with  multiple  sources  of  information  that 
cannot  be  understood  in  isolation  and  so  must  be  inte¬ 
grated  (Low  and  Sweller,  2005).  The  effect  relies  on 
a  particular  characteristic  of  working  memory.  In  the 
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previous  discussion  of  human  cognition,  working 
memory  was  treated  as  a  unitary  concept;  in  fact,  it  is 
best  thought  of  as  consisting  of  multiple  channels  or 
processors  (Baddeley,  1992).  A  visual  processor  for 
dealing  with  two-  or  three-dimensional  objects  and  an 
auditory  processor  for  dealing  with  language  are  par¬ 
tially  independent.  As  a  consequence,  the  simultaneous 
use  of  both  processors  can  expand  the  effective  size 
of  working  memory,  under  some  circumstances.  One 
of  those  circumstances  consists  of  the  conditions  lead¬ 
ing  to  the  split- attention  effect— namely,  multiple 
sources  of  information  that  must  be  integrated  before 
they  can  be  understood.  Under  those  conditions,  the 
use  of  both  the  auditory  and  visual  processor  can 
expand  the  effective  size  of  working  memory  in  an 
instructionally  favorable  manner. 

Consider  a  geometry  example  again  as  discussed 
above.  To  understand  a  geometry  worked  example,  the 
learner  must  simultaneously  consider  both  the  diagram 
and  the  text  because  neither  conveys  the  required 
meaning  in  isolation.  By  presenting  the  verbal  infor¬ 
mation  in  spoken  rather  than  written  form,  working 
memory  can  be  effectively  expanded  because  informa¬ 
tion  is  shifted  from  the  overloaded  visual  processor  to 
be  shared  by  both  the  visual  (for  the  diagram)  and 
auditory  (for  the  text)  processors.  According  to  cogni¬ 
tive  load  theory,  such  an  expansion  of  effective  work¬ 
ing  memory  should  facilitate  learning.  Tindall-Ford  et 
al.  (1997)  demonstrated  the  modality  effect  when 
learners  presented  with  instructions  in  a  visual,  split- 
attention  format  learned  less  than  learners  presented 
with  the  same  material  but  with  all  of  the  verbal  infor¬ 
mation  presented  in  spoken  rather  than  written  form. 

The  modality  effect  is  directly  applicable  to  tech¬ 
nology-based  instruction.  Whereas  an  aspect  of  tech¬ 
nology-based  instruction  requires  verbal  input  to  be 
intelligible,  extraneous  cognitive  load  can  be  reduced 
by  using  spoken  rather  than  written  text.  The  use  of 
spoken  text  is  particularly  important  during  an  anima¬ 
tion.  The  expansion  of  effective  working  memory  due 
to  the  use  of  both  the  auditory  and  visual  processors 
can  permit  an  animation  to  be  viewed  while  simulta¬ 
neously  attending  to  speech  that  explains  otherwise 
unintelligible  aspects  of  the  animation.  The  use  of  writ¬ 
ten  rather  than  spoken  text  runs  the  risk  of  overloading 
the  visual  processor.  In  contrast,  appropriate  use  of  both 
visual  and  auditory  information  can  maximize  the 
potentially  powerful  effects  of  the  borrowing  principle. 

Redundancy  Effect 

The  previous  two  effects  discussed,  the  split-attention 
and  modality  effects,  apply  to  multiple  sources  of 
information,  each  of  which  is  unintelligible  in  isola¬ 


tion.  In  contrast,  the  redundancy  effect  applies  to  mul¬ 
tiple  sources  of  information  that  are  intelligible  in  iso¬ 
lation.  That  difference  in  the  logical  relation  between 
sources  of  information  results  in  quite  different  instruc¬ 
tional  consequences.  The  difference  is  important 
because  frequently,  on  the  surface,  conditions  that  lead 
to  the  split-attention  or  redundancy  effects  can  look 
identical.  It  is  only  by  considering  the  relation  between 
the  multiple  sources  of  information  that  the  appropriate 
instructional  recommendations  can  be  provided. 

In  the  current  context,  redundant  information  is 
defined  as  any  information  that  is  not  relevant  to 
learning.  Most  commonly,  redundant  information 
consists  of  the  same  information  presented  in  differ¬ 
ent  forms  or  media  such  as  presenting  the  same  verbal 
information  in  spoken  and  written  form,  but  it  can 
also  consist  of  any  unnecessary,  additional  informa¬ 
tion  such  as  decorative  pictures,  background  sound, 
or  cartoons. 

In  instructional  contexts,  redundancy  can  fre¬ 
quently  be  found  when  textual  information  repeats 
information  found  in  a  diagram.  A  diagram  indicating 
the  flow  of  blood  in  the  heart,  lungs,  and  body  along 
with  a  statement  that  “blood  flows  from  the  left  ven¬ 
tricle  to  the  aorta”  provides  an  example  of  redundancy 
(Chandler  and  Sweller,  1991).  Although  a  geometry 
diagram  and  its  associated  statements  would  appear, 
on  the  surface,  to  have  the  same  properties  and  hence 
are  governed  by  the  same  instructional  principles  as  a 
blood  flow  diagram  and  its  associated  statements,  they 
are  structurally  very  different  and  require  very  differ¬ 
ent  formatting.  A  geometry  diagram  tells  us  little  of  a 
problem  solution  and  requires  the  statements  for  an 
intelligible  solution  to  be  communicated  to  learners. 
Those  statements  must  be  integrated  with  the  diagram 
so  a  search  for  referents  is  reduced.  In  contrast,  the 
blood  flow  diagram  can  be  fully  intelligible  in  its  own 
right  and  provide  a  full  explanation.  The  appropriate 
instructional  procedure  is  not  to  integrate  the  state¬ 
ments  with  the  diagram  but  rather  to  eliminate  the  least 
effective  source  of  information,  which,  in  this  case,  is 
the  set  of  statements.  Many  experimental  examples 
demonstrate  that  the  elimination  of  redundancy  facil¬ 
itates  learning  (Chandler  and  Sweller,  1991). 

Although  the  redundancy  effect  can  be  considered 
counter-intuitive,  from  a  cognitive  load  theory  per¬ 
spective  the  reason  why  redundant  information  has 
negative  consequences  is  straightforward.  Attending  to 
unnecessary  information  and  attempting  to  integrate  it 
with  essential  information  requires  working  memory 
resources  that  consequently  are  unavailable  for  learn¬ 
ing.  Redundant  information  imposes  an  extraneous 
cognitive  load.  It  is  an  ineffective  use  of  the  borrowing 
principle. 
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There  are  many  forms  of  redundancy  other  than  the 
diagram  and  text  redundancy  described  above  (for  a 
detailed  summary,  see  Sweller,  2005b).  All  forms  of 
redundancy  potentially  apply  to  technology-based 
instruction;  for  example,  a  spoken  commentary  associ¬ 
ated  with  a  simulation  should  only  be  included  if  the 
visual  material  is  unintelligible  without  the  commentary. 
If  visual  material  is  intelligible  in  isolation,  a  spoken 
commentary  will  increase  cognitive  load  as  learners 
attempt  to  integrate  the  auditory  and  visual  information. 
When  devising  a  simulation,  care  must  be  taken  to 
ensure  that  all  information  presented  is  essential  and  not 
simply  an  alternative  way  of  presenting  the  same  infor¬ 
mation.  Equally  damaging  is  additional  information  that 
bears  little  or  no  relation  to  the  required  information.  It 
is  a  trap  to  assume  that  additional  information  will,  at 
worst,  be  neutral  in  its  effect  and  could  be  beneficial. 

Increasing  technical  sophistication  is  permitting 
increasingly  realistic  simulations.  By  definition,  the 
real  world  is  realistic ,  but  we  developed  instructional 
systems  precisely  because  many  aspects  of  the  real 
world  provide  inadequate  instruction.  That  inadequacy 
frequently  is  caused  by  redundancy.  A  realistic  simu¬ 
lation  that  provides  a  depiction  of  a  mechanical  system 
may  be  indistinguishable  from  the  real  mechanical 
system,  but  that  system  may  be  almost  useless  as  an 
instructional  tool.  When  learning  how  the  blood  flows 
in  the  heart,  lungs,  and  body,  most  of  the  structures, 
functions,  and  characteristics  of  the  body  are  irrele¬ 
vant,  which  is  why  it  took  so  long  to  discover  the 
processes  of  the  circulatory  system  using  the  random¬ 
ness  as  genesis  principle.  The  “realistic”  but  irrelevant 
features  served  to  conceal  the  critical  features.  Realis¬ 
tic  features  and  processes  should  not  be  included  in  a 
simulation  if  the  sole  reason  for  inclusion  is  realism. 
To  avoid  redundancy  and  maximize  the  effect  of  the 
borrowing  principle,  there  should  be  a  clear  instruc¬ 
tional  reason  for  including  any  information. 

Expertise  Reversal  Effect 

Two  related  points  must  be  made  concerning  the  dis¬ 
cussion  of  the  previous  effects.  First,  all  of  the  above 
effects  assume  that  learners  are  novices.  It  is  novices 
who  most  frequently  require  instruction  via  the  bor¬ 
rowing  principle.  Second,  the  previous  explanation  of 
the  redundancy  effect  implicitly  assumed  that  redun¬ 
dancy  is  purely  a  function  of  the  materials  being  used; 
in  fact,  it  is  equally  a  function  of  levels  of  expertise. 
Information  that  is  redundant  for  a  more  expert  learner 
may  be  critically  necessary  for  a  less  expert  learner.  A 
novice  may  need  to  borrow  information  from  someone 
else,  an  expert  may  not.  The  expertise  reversal  effect 
was  born  from  these  considerations. 


The  effect  occurs  when  an  instructional  procedure 
that  is  relatively  effective  for  novices  compared  to  a 
control  procedure  first  loses  its  advantage  as  levels  of 
expertise  increase  and  then  begins  to  be  worse  than  the 
control  procedure  with  further  increases  in  expertise 
(Kalyuga  et  al.,  2003).  As  an  instance,  studying  worked 
examples  is  better  than  solving  the  equivalent  problems 
for  novices,  but  with  increased  expertise  solving  prob¬ 
lems  becomes  better  than  studying  examples  (Kalyuga 
et  al.,  2001).  As  other  instances,  for  novices,  integrated 
format  or  dual-modality  instruction  is  better  than  split- 
attention  format  instruction.  With  increasing  expertise, 
rather  than  integrating,  for  example,  diagrams  and  text 
or  using  dual-modality  instruction,  it  is  better  to  elim¬ 
inate  the  text  entirely  (Kalyuga  et  al.,  1998,  2000). 

The  expertise  reversal  effect  is  a  complex  effect 
that  relies  on  redundancy.  As  expertise  increases,  pre¬ 
viously  essential  information  becomes  redundant  and 
so  imposes  an  extraneous  cognitive  load.  Studying 
worked  examples  may  be  essential  for  novices  to 
reduce  cognitive  load,  but  such  activity  becomes 
redundant  with  increasing  expertise  and  is  better 
replaced  by  practice  at  solving  problems  which,  at 
higher  levels  of  expertise,  no  longer  imposes  a  cogni¬ 
tive  load.  Similarly,  explanatory  text  may  be  essential 
for  novices  and  so  should  be  physically  integrated  with 
diagrams  or  presented  in  spoken  form  to  reduce  extra¬ 
neous  cognitive  load.  With  increasing  expertise,  the 
text  becomes  redundant  and  should  be  eliminated. 

The  expertise  reversal  effect  suggests  that  the  detail 
provided  in  technology-based  instruction  should  be 
determined  by  the  knowledge  base  of  the  learners. 
Details  that  are  essential  for  novices  may  be  redundant 
for  more  expert  learners.  Thus,  technology-based 
instruction  must  be  constructed  so  its  specifications 
change  with  changes  in  expertise.  Furthermore,  if 
instructional  materials  are  to  change  with  changes  in 
expertise,  a  method  is  required  to  rapidly  determine 
levels  of  expertise.  Kalyuga  and  Sweller  (2004)  pro¬ 
vided  a  rapid  test  of  knowledge  based  on  the  cognitive 
architecture  described  above.  During  instruction, 
learners  were  presented  a  partially  completed  problem 
and  asked  to  indicate  the  next  step  required  for  solu¬ 
tion.  The  extent  to  which  a  learner  knows  the  next  step 
to  solution  depends  on  the  knowledge  base  held  in 
long-term  memory.  That  information  can  be  used  to 
determine  subsequent  instruction.  It  should  similarly 
be  possible  to  use  this  rapid  assessment  technique  to 
determine  the  nature  of  any  subsequent  instruction. 

Guidance  Fading  Effect 

The  guidance  fading  effect  (Renkl  and  Atkinson,  2003; 
Renkl  et  al.,  2004)  is  closely  related  to  the  worked 
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example  and  expertise  reversal  effects  and  is  also  a 
compound  effect.  It  occurs  when  novices  are  initially 
presented  with  worked  examples,  but  with  increasing 
expertise  those  worked  examples  are  replaced  by  com¬ 
pletion  problems  (van  Merrienboer  et  al.,  2002)  in 
which  a  partial  solution  is  provided  and  the  learner  is 
required  to  complete  the  problem.  With  further 
increases  in  expertise,  the  completion  problems  should 
be  replaced  by  full  problems.  Again,  this  sequence  is 
predicated  on  the  assumption  that  what  constitutes  an 
extraneous  cognitive  load  depends  not  just  on  the 
nature  of  the  instruction  but  on  an  interaction  between 
the  instructional  procedures  and  learner  characteristics 
in  the  form  of  levels  of  expertise.  At  low  levels  of 
expertise,  the  learner  must  make  heavy  use  of  other 
people’s  knowledge  via  the  borrowing  principle.  With 
increasing  expertise,  the  same  information  can  be  bor¬ 
rowed  from  the  learner’s  own  long-term  memory  and 
used  for  practice  purposes. 

Technology-based  instruction  should  take  account 
of  the  guidance  fading  effect  by  initially  providing 
substantial  guidance  which  should  be  gradually  faded 
as  expertise  increases.  As  an  example,  initially,  learn¬ 
ers  should  be  shown  exactly  what  they  need  to  do  with 
minimal  action  required  on  their  part.  With  increases 
in  expertise,  determined  by  rapid  assessment  tech¬ 
niques,  learner  activity  should  be  increased  and  guid¬ 
ance  decreased.  Ultimately,  it  should  be  possible  to 
remove  all  guidance  with  the  learner  simply  practicing 
the  skill. 

Imagination  Effect 

The  imagination  effect  occurs  when  learners  who  are 
asked  to  imagine  a  procedure  or  concept  leam  more 
than  learners  who  are  asked  to  study  the  same  proce¬ 
dure  or  concept  (Cooper  et  al.,  2001;  Leahy  and 
Sweller,  2005).  Imagination  instructions  ask  learners 
to  turn  away  from  the  material  and  attempt  to  imagine 
the  relevant  procedures  or  concepts.  Imagining 
involves  running  material  through  working  memory 
which  should  assist  in  the  transfer  of  the  information 
to  long-term  memory.  The  technique  is  highly  effective 
but  only  when  used  by  learners  with  sufficient  experi¬ 
ence  in  the  domain  to  be  able  to  process  all  of  the 
necessary  information  in  working  memory  without 
assistance  from  the  instructional  material.  For  novices, 
attempts  to  run  a  procedure  through  working  memory 
are  likely  to  fail,  so  instructions  to  study  material, 
which  involves  considering  it  while  looking  at  it,  are 
superior  to  imagination  instructions.  The  switch  from 
studying  being  superior  to  imagination  being  superior 
provides  another  example  of  the  expertise  reversal 
effect.  Again,  as  was  the  case  for  the  expertise  reversal 


and  guidance  fading  effects,  for  novices  information 
is  best  borrowed  from  another  person’s  long-term 
memory,  but  as  levels  of  expertise  increase  that  infor¬ 
mation  can  be  borrowed  from  one’s  own  long-term 
memory  for  purposes  of  practice,  in  this  case  mental 
practice. 

The  imagination  effect  provides  information  on 
what  mental  activity  learners  should  be  engaged  in 
when  dealing  with  a  simulation.  Initially,  they  should 
simply  study  or  interact  with  the  simulation  materials 
because  due  to  working  memory  limitations  they  are 
unlikely  to  have  sufficient  knowledge  to  be  able  to 
effectively  imagine  the  procedures  and  concepts.  With 
increasing  expertise,  they  should  attempt  to  imagine 
the  information  covered  by  the  instruction  because  that 
procedure  seems  to  be  the  most  rapid  technique  for 
transferring  information  to  long-term  memory  and  so 
increasing  levels  of  expertise. 

Element  Interactivity  Effect 

The  expertise  reversal  effect  discussed  above  places 
an  emphasis  on  the  cognitive  load  implications  of  indi¬ 
vidual  differences  in  expertise.  Differences  in  the 
structure  of  the  information  being  considered  are 
equally  important.  None  of  the  above  effects  is  obtain¬ 
able  using  low-complexity  material  (Sweller,  1994) 
that  has  a  low  intrinsic  cognitive  load.  Recall  that  total 
cognitive  load  is  an  addition  of  extraneous,  intrinsic, 
and  germane  cognitive  load.  The  above  effects  are 
primarily  determined  by  an  excessive  extraneous  cog¬ 
nitive  load  that  reduces  germane  cognitive  load 
because  working  memory  capacity  is  exceeded.  If 
intrinsic  cognitive  load  is  low,  a  high  extraneous  cog¬ 
nitive  load  may  not  matter  a  great  deal.  There  may  be 
sufficient  working  memory  capacity  available  to 
enable  germane  cognitive  load  and  its  attendant  rapid 
learning  to  occur. 

What  determines  levels  of  intrinsic  cognitive  load? 
The  only  relevant  factor  within  a  cognitive  load  theory 
framework  is  element  interactivity,  which  is  deter¬ 
mined  by  the  number  of  interacting  elements  that  must 
be  considered  simultaneously  to  understand  the  mate¬ 
rial.  Some  information  is  low  in  element  interactivity 
in  that  the  elements  can  be  learned  one  element  at  a 
time  without  considering  any  other  elements.  Learning 
technical  terminology  provides  an  example.  One  can 
leam  the  name  of  a  component  without  learning  the 
names  of  any  other  components,  so  working  memory 
load  may  be  very  low.  Cognitive  load  effects  are  not 
likely  to  be  relevant  when  intrinsic  cognitive  load  is 
low.  In  contrast,  learning  how  components  interact  in 
a  machine  has  high  element  interactivity  because  it 
may  be  impossible  to  understand  the  function  of  one 
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component  without  simultaneously  considering  the 
function  of  all  of  the  components.  High  element  inter¬ 
activity  results  in  a  high  intrinsic  cognitive  load,  leav¬ 
ing  little  working  memory  capacity  available  for  learn¬ 
ing.  Under  these  circumstances,  levels  of  extraneous 
cognitive  load  become  critical,  and  the  cognitive  load 
effects  discussed  above  become  relevant;  thus,  cogni¬ 
tive  load  effects  become  critical  when  technology- 
based  instruction  deals  with  complex,  high  element 
interactivity  material. 

Isolated  Interacting  Elements  Effect 

When  element  interactivity  is  very  high,  it  may  be 
impossible  for  learners  to  understand  the  material 
because  it  may  be  impossible  for  them  to  simulta¬ 
neously  process  all  of  the  interacting  elements  in  work¬ 
ing  memory.  How  should  such  material  be  presented? 
From  a  cognitive  load  theory  perspective ,  the  only  way 
seems  to  be  to  initially  present  the  material  as  individ¬ 
ual  elements  ignoring  their  interactions.  This  proce¬ 
dure  will  permit  the  elements  to  be  learned  but  without 
understanding.  Once  the  individual  elements  have  been 
learned,  their  interactions  can  be  emphasized.  It  is  only 
at  that  point  that  the  material  will  be  understood 
because  it  cannot  be  understood  by  simply  considering 
individual  elements.  Empirical  work  has  demonstrated 
that  teaching  individual  elements  first,  at  the  expense 
of  understanding,  followed  by  teaching  the  interactions 
between  elements  results  in  more  effective  learning 
than  attempting  to  have  learners  understand  very  high 
element  interactivity  material  right  from  the  beginning 
of  instruction  (Pollock  et  al.,  2002). 

The  isolated,  interacting  elements  effect  may  have 
considerable  relevance  to  technology-based  instruc¬ 
tion  such  as  instructional  simulations .  Material  is  dif¬ 
ficult  to  understand  because  it  is  high  in  element 
interactivity.  Presenting  that  material  in  a  realistic 
fashion  during  a  simulation  may  be  condemning 
learners  to  attempting  to  understand  information  that 
vastly  exceeds  their  working  memory  capacity 
because  a  realistic  simulation  may  involve  a  huge 
number  of  interacting  elements.  A  less  realistic  sim¬ 
ulation  with  fewer  interacting  elements  may  be  more 
readily  understood  and  learned.  Although  a  full 
understanding  of  the  material  is  impossible  by  this 
technique,  it  is  equally  impossible  to  process  a  large 
number  of  interacting  elements  simultaneously.  It 
may  be  better  to  provide  simulations  that  result  in 
limited  understanding  initially,  followed  by  more 
complete  versions  that  permit  full  understanding.  In 
this  manner,  learners  may  be  spared  the  need  to 
attempt  to  understand  material  that  is  quite  impossible 
for  them  to  deal  with. 


DISCUSSION 

Cognitive  load  theory  provides  an  integrated  system 
dealing  with  the  evolutionary  origins  of  human  cogni¬ 
tion  leading  to  an  explanation  of  cognitive  structures 
and  processes.  In  turn,  those  structures  and  processes 
can  be  used  to  generate  instructional  principles.  The 
generation  of  applications  in  the  form  of  instructional 
principles  provides  some  warrant  for  the  validity  of  the 
original  cognitive  architecture.  This  unified  system  can 
provide  a  base  for  instructional  design  including  the 
structure  and  function  of  technology-based  instruction. 

There  are  two  aspects  of  technology-based  educa¬ 
tion  in  general  and  of  the  work  reported  in  this  chapter 
that  require  emphasis.  First,  the  use  of  technology  in 
education  should  not  be  based  merely  on  the  availabil¬ 
ity  of  technology.  There  is  a  long  history  of  new  tech¬ 
nological  applications  such  as  radio,  films,  television, 
mainframe  computers  attached  to  terminals,  stand¬ 
alone  microcomputers  and  now  the  Web  being  hailed 
as  potentially  revolutionizing  education.  Frequently, 
these  technological  advances  have  had  minimal,  long¬ 
term  educational  impact.  While  technology  changes, 
human  cognitive  architecture  does  not.  The  introduc¬ 
tion  of  educational  technology  without  reference  to  its 
cognitive  consequences  is  unlikely  to  be  effective.  The 
five  natural  information  processing  principles  outlined 
above  provide  an  initial  guide.  According  to  those 
principles,  learning  consists  of  changes  to  the  long¬ 
term  store,  the  most  efficient  method  of  bringing  about 
those  changes  is  by  borrowing  knowledge  from  knowl¬ 
edgeable  educators,  and  that  knowledge  must  be  struc¬ 
tured  in  a  manner  that  reduces  working  memory  load. 
I  am  not  aware  of  any  evidence  that  simply  using 
technology  will  necessarily  conform  to  any  of  the  five 
principles.  Instruction  that  ignores  human  cognitive 
architecture  is  likely  to  be  ineffective  whether  or  not 
it  uses  technology.  In  contrast,  technology-based 
instruction  that  is  explicitly  structured  to  conform  to 
what  we  know  of  human  cognition  has  a  much  better 
chance  of  being  effective. 

All  of  the  cognitive  load  effects  discussed  above 
were  based  on  the  assumption  that  the  aim  of  instruc¬ 
tion  is  the  acquisition  of  knowledge  in  long-term  mem¬ 
ory  and  that  the  best  way  of  achieving  this  aim  is  to 
make  use  of  the  borrowing  principle  by  providing 
direct  instructional  guidance  organized  to  reduce  extra¬ 
neous  working  memory  load. 

The  second  point  that  requires  emphasis  concerns 
how  we  determine  whether  technology-based  educa¬ 
tion  is  effective.  The  instructional  effects  described 
above  were  not  only  based  on  our  knowledge  of  human 
cognitive  architecture  but  also  were  tested  using  con¬ 
trolled,  randomized  experimental  designs.  Experiments 
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in  which  participants  are  randomly  allocated  to  two  or 
more  experimental  groups  with  different  instructional 
procedures  but  identical  test  procedures  are  essential 
when  determining  effective  instructional  procedures  in 
this  field.  Verification  procedures  in  which  instructors 
and  learners  are  presented  instruction  using  a  new  tech¬ 
nology  and  simply  asked  if  they  think  it  is  effective  or 
if  they  enjoyed  it  have  very  limited  utility. 

While  cognitive  load  theory  indicates  many  condi¬ 
tions  under  which  instruction  may  or  may  not  be  effec¬ 
tive,  those  conditions  are  closely  dependent  on  the 
cognitive  architecture  discussed  and,  specifically,  on 
the  characteristics  of  working  and  long-term  memory. 
The  theory  is  not  intended  as  a  general  theory  of  cog¬ 
nition  or  of  instruction,  and  so  many  important  vari¬ 
ables  are  not  considered  by  the  theory;  for  example, 
despite  some  discussion  about  including  motivational 
variables  in  cognitive  load  theory  and  considering  this 
issue  in  future  research,  at  present  it  has  not  been 
incorporated  into  the  theory.  That  incorporation  will, 
in  all  probability,  only  occur  if  relations  between  cog¬ 
nitive  architecture  and  motivation  can  be  established. 

In  conclusion,  advances  in  instructional  technol¬ 
ogy  have  provided  us  with  the  ability  to  use  instruc¬ 
tional  procedures  that  until  recently  were  difficult  or 
impossible  to  implement.  Frequently,  those  procedures 
have  been  recommended  purely  because  they  are  now 
possible  rather  than  because  there  was  evidence  for 
their  cognitive  effectiveness  or  even  desirability.  Cog¬ 
nitive  load  theory  is  intended  to  provide  a  guide  to  the 
types  of  techniques  that  are  likely  to  work.  It  has  been 
tested  using  controlled,  randomized  experiments  over 
many  years  by  many  investigators  in  a  wide  variety  of 
technology-based  environments.  There  is  every  reason 
to  suppose  that  the  use  of  our  knowledge  of  human 
cognition  will  continue  to  generate  technology-based 
instructional  procedures. 
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ABSTRACT 

One  of  the  most  influential  and  pervasive  theories  under¬ 
lying  instructional  design  proposes  that  (1)  there  are 
identifiably  different  types  of  learning  outcomes,  and 
(2)  the  acquisition  of  these  outcomes  requires  different 
internal  and  external  conditions  of  learning.  These  prop¬ 
ositions  underlie  an  outcome-referenced ,  conditions- 
based  paradigm  of  instructional  design  (ID).  The  out¬ 
come-referenced,  conditions-based  perspective  is  com¬ 
monplace,  if  not  universal,  in  current  instructional  psy¬ 
chology  and  instructional  design  thinking,  even  when 
the  author’s  orientation  and  values  are  not  based  on  the 
cognitive  science  that  underlies  this  perspective. 
Whether  or  not  individuals  formally  subscribe  to  or  have 
an  interest  in  these  theories  and  models,  it  is  part  of  the 
everyday  work  of  instructional  designers  and  scholars 
of  instruction  and  learning  environments.  This  chapter 
(1)  presents  the  assumptions  of  the  outcome-referenced, 
conditions-based  perspective,  (2)  describes  prevalent 
theories  and  models  within  this  tradition,  and  (3)  pro¬ 
vides  future  directions  for  research  and  development. 

KEYWORDS 

Conditions-based  instruction:  Described  by  Robert 
Gagne  (1985)  as  (1)  internal  and  external  condi¬ 
tions  of  learning;  (2)  states  possessed  by  the  learner 
such  as  prior  knowledge  are  internal  conditions, 
and  instructional  supports  that  are  designed  to  pro¬ 
mote  learning  are  external  conditions. 

Learning  hierarchy:  Description  of  successively  achiev¬ 
able  intellectual  skills,  each  stated  as  a  performance 
class  in  which  achievement  of  a  superordinate  skill 
is  in  part  dependent  on  the  internal  condition  of 
having  learned  necessary  subordinate  skills. 
Outcome-referenced  models:  Approaches  to  instruc¬ 
tional  design  in  which  (1)  consideration  of  the 
nature  of  the  learning  tasks  and  the  conditions 
required  to  support  them  are  central,  and  (2)  learn¬ 
ing  outcomes  are  categorized  to  represent  not  only 
qualitative  differences  in  the  acquired  capability 
(as  a  category  of  task  or  goal)  but  also  external 
conditions  that  support  learning  and  different 
learner  states  — the  latter  are  referred  to  as  internal 
conditions ,  which  also  facilitate  learning. 

INTRODUCTION 

One  of  the  most  influential  and  pervasive  theories 
underlying  instructional  design  proposes  that  ( 1 )  there 
are  identifiably  different  types  of  learning  outcomes, 
and  (2)  the  acquisition  of  these  outcomes  requires  dif¬ 
ferent  internal  and  external  conditions  of  learning.  In 


other  words,  this  theory  suggests  that  all  learning  is 
not  qualitatively  the  same,  that  learning  outcomes 
across  contents,  contexts,  and  learners  have  significant 
and  identifiable  similarities  in  their  cognitive  demands 
on  the  learner.  Furthermore,  each  learning  outcome 
category  significantly  differs  from  the  other  learning 
outcome  categories  in  terms  of  its  cognitive  demands 
on  the  learner.  Finally,  as  this  family  of  theories  is 
instructional  in  nature,  they  propose  that  these  distinc¬ 
tive  cognitive  processing  demands  can  be  supported 
by  equally  distinctive  instructional  methods,  strategies, 
tactics,  or  conditions. 

These  propositions  underlie  what  Wilson  and  Cole 
(1991)  term  a  conditions- of -learning  paradigm  of 
instructional  design.  Models  of  instructional  design 
that  follow  an  outcome -referenced,  conditions-based 
theory  are  predicated  upon  the  seminal  principles  of 
Robert  Gagne  (1965)  that:  (1)  learning  can  be  classi¬ 
fied  into  categories  that  require  similar  cognitive  activ¬ 
ities  for  learning  (Gagne  termed  these  internal  condi¬ 
tions  of  learning),  and,  therefore,  (2)  within  these 
categories  of  learning,  similar  instructional  supports 
are  needed  to  facilitate  learning  (Gagne  termed  these 
external  conditions  of  learning) . 

The  influence  of  an  outcome -referenced,  condi¬ 
tions-based  perspective  can  be  found  in  the  task  anal¬ 
ysis,  strategy  development,  assessment,  and  evaluation 
procedures  of  conditions-based  instructional  design; 
however,  the  point  at  which  the  outcome-referenced, 
conditions-based  perspective  has  the  greatest  influence 
and  most  unique  contribution  is  in  the  development  of 
instructional  strategies.  According  to  conditions-based 
theory,  when  designing  instructional  strategies, 
instructional  designers  must  determine  the  goals  of 
instruction,  categorize  these  goals  as  to  outcome,  and 
select  strategies  that  have  been  suggested  as  being 
effective  for  this  category  of  learning  outcome  (or 
devise  strategies  consistent  with  the  cognitive  process¬ 
ing  demands  of  the  learning  task). 

The  outcome-referenced,  conditions-based  assump¬ 
tion  is  commonplace,  if  not  universal,  in  current  instruc¬ 
tional  psychology  and  instructional  design  thinking,  even 
when  an  author’s  orientation  and  values  are  not  based 
on  the  cognitive  science  that  underlies  the  conditions 
theory.  For  example.  Nelson  (1999,  p.  242)  took  care  to 
note  that  her  prescriptions  for  collaborative  problem 
solving  “should  only  be  used  when  those  types  of  learn¬ 
ing  are  paramount.”  This  care  in  consideration  of  the 
nature  of  the  learning  task  was  remarkably  absent  before 
conditions  theory  was  developed  and  is  remarkably  con¬ 
sistent  in  its  application  now.  Whether  or  not  individuals 
formally  subscribe  to  or  have  an  interest  in  the  conditions 
theory,  it  is  part  of  the  everyday  work  of  designers  and 
scholars  of  instruction  and  learning  environments. 
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Alternatives  to  an  Outcome-Referenced, 
Conditions-Based  Perspective 

To  have  a  central  concern  with  what  is  to  be  learned  is 
not  the  only  perspective  that  may  be  taken  or  has  been 
taken  by  instructional  theorists.  Although  the  variety  of 
viewpoints  is  myriad,  brief  consideration  of  four  alter¬ 
native  views  should  help  us  put  outcome-referenced 
thinking  into  perspective.  The  four  views  to  be  dis¬ 
cussed  here  are  (1)  learner-centered,  (2)  experience- 
centered,  (3)  activity-centered,  and  (4)  content-centered. 

Learner-Centered  Instruction 

Students  of  the  history  of  educational  thought  are  often 
struck  with  the  popularity  of  learner-centered  thinking 
in  educational  reforms  since  Rousseau’s  Emile  (1773), 
which  encouraged  an  education  that  would  harmonize 
with  the  “natural  conditions  of  a  child’s  growth” 
(Ulich,  1950,  p.  219).  Pestalozzi  (1746-1827)  encour¬ 
aged  education  that  followed  the  “road  of  Nature” 
(Ulich,  1950,  p.  260)  in  which  children’s  individual 
personal  paths  were  the  essential  guide  to  both  what 
should  be  learned  and  how  it  should  be  taught.  Herbart 
(1776-1841),  although  less  romantic  than  Rousseau 
and  Pestalozzi,  was  an  important  reformer  in  the  early 
19th  century.  He  grounded  his  work  in  the  study  of 
children’s  learning  and  was  guided  in  large  measure 
by  learners’  personal  interests. 

Experience-Centered  Instruction 

John  Jacob  Comenius,  a  17th-century  educational 
reformer,  began  to  look  at  the  experience  of  learners 
(rather  than  content)  as  guiding  education.  Although 
Froebel  is  part  of  the  tradition  of  learner-centered  edu¬ 
cational  reform  of  the  18th  and  19th  centuries  along 
with  Rosseau,  Pestalozzi,  and  Herbart,  we  can  see  in 
his  work  an  emphasis  on  experience:  “The  educator 
ought  to  lead  the  child  through  such  situations  as  will 
help  him  to  relate  his  experiences  organically  with  one 
another”  (Ulich,  1950,  p.  288).  Similar  to  outcome- 
referenced  views,  an  experience-centered  conception 
of  instruction  places  more  emphasis  on  instructional 
strategy  than  does  a  more  conservative  “content-cen¬ 
tered”  approach. 

Activity-Centered  Instruction 

John  Dewey  (1859-1952)  has  variously  been  charac¬ 
terized  as  both  a  learner-centered  and  experience-cen¬ 
tered  educational  philosopher.  Dewey  provided  enor¬ 
mous  support  for  consideration  of  learner  activity  as 
a  central  concept.  Ulich  (1950,  p.  319)  noted  that 


Dewey  “wants  interest  and  activity  to  be  more  closely 
related  to  all  the  diverse  features  of  community  life, 
manual,  intellectual,  emotional,  and  social.”  Dewey 
was,  at  times,  radical  in  his  belief  about  the  centrality 
of  activity  and  experience,  to  the  point  of  rejecting 
external  goals  altogether  (Dewey,  1910,  p.  13): 

...  to  set  up  any  end  outside  of  education,  as  furnishing 
its  goal  and  standard,  is  to  deprive  the  educational  process 
of  much  of  its  meaning,  and  tends  to  make  us  rely  upon 
false  and  external  stimuli  in  dealing  with  the  child. 

Although  it  is  possible  to  think  of  learning  activities 
as  contributors  to  a  priori  learning  goals,  it  is  also 
possible  to  think  of  activity  as  either  an  end  in  itself 
or  as  leading  to  outcomes  not  previously  specified. 
With  all  due  respect  to  Dewey’s  immense  contributions 
to  educational  thought,  the  all-too-common  practice  of 
classroom  activity  for  its  own  sake  is  not  good  practice, 
in  our  view.  From  our  perspective,  even  when  it  is 
activity  centered,  good  instruction  is  congruent  with 
and  facilitative  of  the  goals  for  learning. 

Content-Centered  Instruction 

It  is  easy  to  mistake  an  outcome-referenced  approach 
with  a  content-centered  orientation.  In  comparison  to 
learner-centered,  experienced-centered,  and  activity- 
centered,  both  approaches  seem  hopelessly  conserva¬ 
tive  to  many  people:  however,  in  an  outcome -refer¬ 
enced  approach,  content  is  only  the  context.  In  most 
cases,  alternative  contents  may  serve  to  facilitate  the 
achievement  of  a  learning  goal.  In  contrast,  with  a 
content-centered  view,  the  content  is  truly  crucial.  Per¬ 
haps  the  best  example  of  a  content-centered  view  is 
the  great  books  of  the  Western  world  curriculum  pro¬ 
posed  by  Robert  Maynard  Hutchins  (1953),  Mortimer 
Adler,  and  others.  The  perennialists,  as  they  are 
described  by  Brameld  (1955),  have  a  particular  body 
of  content  in  mind,  first  and  foremost. 

Integrated  Perspectives  for  Design 

Integration  of  the  learner-centered,  experience-cen¬ 
tered,  activity-centered,  and  content-centered  perspec¬ 
tives  in  concert  with  an  outcome-referenced,  condi- 
tions-based  approach  is  not  out  of  the  question.  We 
believe,  for  example,  that  it  is  wholly  appropriate  for 
an  instructional  designer  to  determine  what  the  inten¬ 
tions  for  learning  are  and  then  determine  what  sort  of 
activities  and  experiences  would  best  facilitate  that 
learning,  based  on  the  structure  and  form  of  the  content 
to  be  learned  as  well  as  the  characteristics  of  the 
learner.  These  decisions  are  essentially  strategic 
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(Smith  and  Ragan,  2005).  Most  designers  are  some¬ 
what  eclectic  with  regard  to  these  alternative  perspec¬ 
tives.  Designers  who  first  consider  the  learning  out¬ 
come  and  who  never  lose  consideration  for  it  when 
identifying  the  instructional  supports  or  strategies  that 
facilitate  the  internal  cognitive  processes  required  for 
achieving  the  learning  outcome  are  most  aligned  with 
an  outcome-referenced,  conditions-based  perspective. 

Alternatives  within  an  Outcome-Referenced, 
Conditions-Based  Perspective 

Concern  with  learning  outcomes  takes  on  a  different 
shape  depending  on  whether  the  context  is  at  the  micro 
level,  as  in  designing  instructional  strategies  for  partic¬ 
ular  encounters,  or  is  at  the  macro  level,  as  with  curric¬ 
ulum-  and  standards-based  approaches.  This  chapter 
and  the  research  it  reports  focus  primarily  on  the  micro 
level.  Such  a  focus  concentrates  on  the  facilitation  of 
learning  of  specific  identified  learning  outcomes.  The 
macro  level,  on  the  other  hand,  engages  more  frequently 
in  policy  issues  such  as  the  widely  known  and  some¬ 
times  controversial  outcomes-based  education  move¬ 
ment.  Outcomes-based  education,  which  is  primarily 
about  the  nature  and  source  of  learning  goals  and  the 
directions  of  school  curricula,  has  little  to  do  with  how 
learning  is  facilitated,  or  about  how  instruction  should 
proceed,  or  what  conditions  are  required  for  efficient, 
effective,  and  appealing  instruction.  Generally  speaking, 
only  one  element  of  instruction  — assessment  — is 
involved  in  outcomes-based  education  at  the  macro 
level.  In  contrast,  outcome-referenced,  conditions-based 
theories  and  models  of  instruction  are  fundamentally 
concerned  with  ways  to  facilitate  learning.  We  believe 
it  is  important  to  maintain  an  awareness  of  the  implica¬ 
tions  of  these  fundamental  differences. 

This  chapter  is  not  the  only  one  in  this  handbook 
that  posits  theories  and  models  within  an  outcome- 
referenced  perspective.  Within  this  chapter  we  have 
tried  to  minimize  replication  of  models  discussed  in 
other  chapters.  We  could  have  included  a  wide  array 
of  alternative  ideas,  ranging  from  philosophies  to  the¬ 
ories  to  models  to  particular  strategies.  As  an  example, 
whole  task  models  that  focus  on  multidimensional, 
complex  cognitive  skills  as  learning  outcomes  and  the 
conditions  that  facilitate  the  attainment  of  highly  inte¬ 
grated  sets  of  various  categories  of  outcomes  fit  within 
an  outcome-referenced  perspective,  van  Merrienboer 
and  Dijkstra’s  (1997)  four-component  instructional 
design  model  is  one  example  of  such  a  model.  Prob¬ 
lem-based  learning  proceeds  from  an  outcome-refer¬ 
enced  perspective  focusing  on  the  conditions  and  strat¬ 
egies  for  problem  solving  and  achieving  related 
outcomes.  Performance  improvement  models  connect 


workplace  outcomes  and  goals  to  specific  interventions 
with  the  intent  of  closing  gaps  between  existing  and 
desired  levels  of  performance.  In  addition,  many  of  the 
instructional  strategy  chapters  in  this  handbook  pro¬ 
vide  discussions  that  link  a  specific  strategy  to  the 
learning  outcomes  for  which  they  are  most  appropriate  . 

Although  it  is  possible  to  endlessly  debate  the  cat¬ 
egories,  we  find  that  many  of  the  examples  we  provide 
have  been  described  at  many  different  levels— from 
philosophies  to  strategies.  We  think  that  all  of  these 
applications  and  interpretations  of  outcome-referenced 
thinking  warrant  consideration.  In  this  chapter,  we 
explore  in  depth  a  body  of  research-based  instructional 
theory  on  a  particular  outcome -referenced  approach, 
the  conditions-based  theory,  and  focus  our  examples 
on  models  and  theories  that  most  closely  align  them¬ 
selves  with  this  specific  perspective. 

Utility  of  an  Outcome-Referenced, 
Conditions-Based  Perspective 

The  utilization  of  outcome-referenced,  conditions- 
based  theories  and  models  of  instructional  design  is 
evident  across  the  various  settings,  domains,  and  appli¬ 
cation  fields  informed  by  instructional  theory  (Dijkstra 
et  al.,  1997).  These  include  contexts,  such  as  PreK-12 
education  (Podolskij,  1997;  Wiggins  and  McTighe, 
1998),  higher  education  (Gagne  and  Merrill,  1990; 
Miles  and  Wilson,  2004;  Terlouw,  1997),  and  business, 
military,  and  industry  training  (Clark  and  Blake,  1997; 
Pieters,  1997),  as  well  as  areas  of  application,  such  as 
instructional  multimedia  development  (Deubel,  2003), 
distance  learning  (Bourdeau  and  Bates,  1997;  Inglis, 
1989),  learning  objects  (Merrill,  1999),  expert  systems 
(Kasowitz,  1998),  and  performance  support  (Van  Tiem 
et  al.,  2001).  The  pervasiveness  of  outcome-refer¬ 
enced,  conditions-based  thinking  across  many  instruc¬ 
tional  design  theories  and  models  combined  with  the 
prevalence  of  its  utilization  across  settings  and  areas 
of  application  make  it  among  the  most  influential  per¬ 
spectives  in  the  field  of  educational  communications 
and  technology.  The  purpose  of  this  chapter  is  to  (1) 
present  the  assumptions  of  the  outcome-referenced, 
conditions-based  perspective,  (2)  describe  prevalent 
theories  and  models  within  this  tradition,  and  (3)  pro¬ 
vide  future  directions  for  research  and  development. 


THE  PROPOSITIONS  OF  A 
CONDITIONS-BASED  THEORY 

As  noted  in  the  chapter  introduction,  the  primary  prop¬ 
ositions  of  conditions-based  theory  can  be  summarized 
by  four  main  assertions:  (1)  Learning  goals  can  be 
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categorized  as  to  learning  outcome  or  knowledge  type; 
(2)  the  acquisition  of  different  outcome  categories 
requires  different  internal  conditions  (alternatively 
phrased  as  “different  internal  conditions  lead  to  differ¬ 
ent  cognitive  outcomes”);  (3)  learning  outcomes  can  be 
represented  in  a  predictable  prerequisite  relationship; 
and  (4)  different  internal  processes  are  supported  by 
identifiably  different  instructional  supports  (or  external 
conditions  for  learning).  In  this  section,  issues  relating 
to  each  of  the  primary  propositions  are  discussed. 

Learning  Outcomes  Can  Be  Categorized 

To  examine  our  first  assertion,  it  is  essential  to  under¬ 
stand  what  is  meant  by  a  learning  outcome.  The  mean¬ 
ing  we  ascribe  to  outcomes  differs  depending  on 
whether  we  perceive  these  outcomes  as  external  (as  a 
category  of  task  or  goal)  or  internal  (as  an  acquired 
capability,  perhaps  supported  by  a  unique  memory  sys¬ 
tem)  .  Theories  that  have  conveyed  internal  orientations 
include  Gagne’s  (1985)  classification  system  of  out¬ 
comes  as  acquired  capabilities  and  Landa’s  conceptu¬ 
alization  of  kinds  of  knowledge  as  psychological  phe¬ 
nomena.  In  contrast,  Reigeluth's  categorization  of 
types  of  content  implies  the  categorization  of  an  exter¬ 
nal  referent.  Merrill  (1983)  has  described  his  outcome 
categories  as  performances  and  learned  capabilities, 
representing  a  mix  of  internal  and  external  connota¬ 
tions.  Clearly,  there  is  no  consensus  even  within  the 
models  described  in  this  chapter  as  to  what  the  phrase 
learning  outcomes  denotes. 

Instructional  theorists  have  suggested  a  variety  of 
category  systems,  most  of  which  are  compatible  with 
the  declarative/procedural  classification  (Anderson, 
1990).  Gagne  added  other  categories  to  the  schema: 
attitude,  motor  skill,  and,  perhaps,  cognitive  strategies. 
Tennyson  and  Rasch  (1988)  added  a  third  class  of 
learning— contextual  knowledge.  Given  the  variation 
within  and  across  classification  systems,  there  is  also 
considerable  variation  in  the  type  of  evidence  required 
to  support  the  validity  of  each  category  system.  The 
nature  of  the  variation,  which  encompasses  type  and 
complexity  of  the  evidence,  depends  on  whether  the 
phenomena  are  viewed  as  entities  that  are  outward 
(i.e.,  they  can  be  culled  out  and  observed)  or  within 
(in  which  case  we  can  only  observe  circumstantial 
evidence  of  their  presence). 

An  examination  of  the  assertion  that  learning  out¬ 
comes  can  be  categorized  entails  both  a  philosophical 
and  a  psychological  view.  Among  the  philosophers  and 
psychologists  who  have  posited  ways  of  categorizing 
knowledge  are  Ryle  (1949)  and  Anderson  (1990), 
respectively,  who  proposed  similar  declarative  or  pro¬ 
cedural  classification  systems.  Depending  on  one’s  ori¬ 


entation  as  a  philosopher,  psychologist,  or  instructional 
theorist,  the  evidence  of  the  truth  of  the  proposition 
would  vary.  For  philosophers,  the  truth  of  the  propo¬ 
sition  would  be  an  epistemological  question,  and  the 
manner  for  determining  its  truth  would  depend  on  the 
philosophic  school  to  which  a  particular  philosopher 
subscribed.  We  do  not  pursue  this  approach  for  deter¬ 
mining  the  validity  of  our  assertion  directly.  Alterna¬ 
tively,  Reigeluth  (1983)  suggested  a  utility  criterion 
for  determining  whether  a  categorization  system  was 
appropriate.  The  psychologist  would  want  empirical 
evidence  that  the  categories  are  distinct,  which  leads 
to  our  second  proposition. 

Different  Outcome  Categories  Require 
Different  Internal  Conditions 

Most  of  the  models  within  outcome-referenced,  con- 
ditions-based  theories  propose  that  learning  categories 
are  different  in  terms  of  cognitive  processing  demands 
and  activities.  All  of  the  major  design  models 
described  in  this  chapter  appear  to  make  this  assump¬ 
tion  to  a  greater  or  lesser  degree.  Whereas  all  of  the 
models  in  this  chapter  suggest  that  general  information 
processing  occurs  in  learning,  they  also  suggest  that 
this  processing  is  significantly  and  predictably  differ¬ 
ent  for  each  of  the  categories  of  learning  identified  by 
each  model.  For  example,  Gagne  suggested  that,  in 
particular,  the  cognitive  processes  of  retrieval  of  prior 
knowledge,  encoding,  and  retrieval  and  transfer  of  new 
learning  would  differ  significantly  in  nature,  depend¬ 
ing  on  the  type  of  learning  goal.  Indeed,  several  of  the 
model  developers,  including  Gagne  (1985),  Merrill 
(1983),  Smith  and  Ragan  (2005),  and  Tennyson  and 
Rasch  (1988),  postulated  different  memory  structures 
for  different  types  of  learning  outcomes. 

A  slightly  different  statement  of  the  proposition 
allows  for  a  closer  relationship  to  the  first  proposition 
(that  outcomes  can  be  categorized):  Different  internal 
conditions  lead  to  different  cognitive  outcomes.  This 
assertion,  which  is  more  descriptive  and  less  prescrip¬ 
tive,  seems  to  be  supported  by  additional  educational 
theorists;  for  example,  both  Anderson  (1990)  and  E. 
Gagne  et  al.  (1993)  proposed  that  different  cognitive 
processes  lead  to  declarative  and  procedural  learning. 
They  also  proposed  that  these  two  types  of  learning 
have  different  memory  systems-schemata  for  declar¬ 
ative  knowledge  and  productions  for  procedural  learn¬ 
ing.  Both  theories  provided  some  empirical  evidence 
that  these  cognitive  processes  and  storage  systems 
were  indeed  unique  to  the  two  types  of  learning. 

We  must  point  out  that  even  if  conneetionists 
(Bereiter,  1991;  Hawkins  and  Blakeslee,  2004)  are 
correct  that  there  is  only  one  memory  system  (neural 
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TABLE  32.1 

Results  of  Studies  on  Hierarchies 


Author 

Date 

Learning  Task 

Results 

Nicholas 

1970 

Not  stated 

Replicated  Wiegand  (1970) 

Coleman  and  Gagne 

1970 

Exports  comparison 

Too  much  mastery  by  control  group  but  better  transfer  to  problem  solving  found 

Eustace 

1969 

Concept  “noun” 

Hypothesized  sequence  better 

Okey  and  Gagne 

1970 

Chemistry 

Learning  hierarchy  revision  better  than  original  version 

Resnick  et  al. 

1971 

Double  classification 

Successfully  predicted  outcomes 

Caruso  and  Resnick 

1971 

Replication 

Resnick  et  al.  (1971)  confirmed 

Wang  et  al. 

1971 

Math  curriculum 

Several  dependency  sequences  found 

networks)  and  only  one  basic  cognitive  process  (pat¬ 
tern  recognition,  Bereiter;  prediction,  Hawkins),  it 
does  not  necessarily  preclude  the  possibility  of  differ¬ 
ent  types  of  learning  capabilities.  It  is  possible,  for 
example,  that  generalized  activation  patterns  can  rep¬ 
resent  certain  types  of  learning. 

Outcomes  Can  Be  Represented 
in  a  Prerequisite  Relationship 

Gagne’s  work  on  learning  hierarchies  has  confirmed 
the  assumption  of  prerequisite  relationships.  In  1973, 
Gagne  described  the  idea  of  learning  hierarchies  and 
noted  that  learning  hierarchies  had  the  following  char¬ 
acteristics:  (1)  They  described  “successively  achiev¬ 
able  intellectual  skills,  each  of  which  is  stated  as  a 
performance  class”;  (2)  they  did  not  include  “verbal 
information,  cognitive  strategies,  motivational  factors, 
or  performance  sets”;  and  (3)  each  step  in  the  hierarchy 
described  “only  those  prerequisite  skills  that  must  be 
recalled  at  the  moment  of  learning”  to  supply  the  nec¬ 
essary  “internal”  component  of  the  total  learning  sit¬ 
uation  (pp.  21-22).  Gagne  also  described  several  stud¬ 
ies  on  the  validation  of  learning  hierarchies. 

A  fundamental  way  to  approach  validation  was  to 
look  at  differences  in  transfer  between  groups  that 
attained  and  groups  that  did  not  attain  hypothesized 
prerequisites.  Gagne  et  al.  (1962,  p.  9,  Table  3)  was 
cited  as  an  example  providing  positive  evidence  from 
such  an  approach.  Other  validation  studies  were 
reported,  each  investigating  in  one  way  or  another  the 
validity  of  a  particular  learning  hierarchy  — in  other 
words,  examining  the  extent  to  which  the  hierarchy 
was  a  true  description  of  prerequisite  relationships 
among  hypothesized  subtasks.  As  a  set,  these  studies 
could  be  viewed  as  evidence  of  the  validity  of  the 
concept  of  learning  hierarchies.  The  studies  are  sum¬ 
marized  in  Table  32.1. 

In  addition  to  work  done  by  Gagne  and  others 
working  directly  in  his  tradition,  research  by  individu¬ 
als  working  from  entirely  different  frames  of  reference 


has  also  solidly  confirmed  this  assumption.  Although 
early  learning  hierarchy  research  appeared  to  be  highly 
confirmatory.  White  (1973)  developed  an  important 
review  of  learning  hierarchy  research  in  the  early 
1970s.  In  his  review.  White  sought  studies  validating 
the  idea  of  learning  hierarchies.  Due  to  methodological 
weaknesses  among  the  studies,  though.  White  did  not 
find  any  studies  that  validated  a  complete  and  precise 
fit  between  a  proposed  learning  hierarchy  and  optimal 
learning  and  noted  the  weaknesses  within  the  studies. 

Research  done  subsequent  to  White’s  review, 
incorporated  his  recommendations  to  correct  method¬ 
ological  weaknesses,  and  resulted  in  a  series  of  pub¬ 
lished  studies  confirming  the  learning  hierarchy  for¬ 
mulation  (Linke,  1973;  White,  1974a,b,c).  These 
results  led  Gagne  to  conclude,  “The  basic  hypothesis 
of  learning  hierarchies  is  now  well  established,  and 
sound  practical  methods  for  testing  newly  designed 
hierarchies  exist”  (White  and  Gagne,  1974,  p.  363). 
Other  research  that  may  be  considered  within  the 
Gagne  tradition  that  appeared  to  confirm  the  learning 
hierarchy  hypothesis  includes  studies  by  Merrill  et  al. 
(1970),  Resnick  ( 1967),  Resnick  and  Wang  (1969) ,  and 
Yao  (1989).  Work  on  learning  hierarchies  outside  the 
Gagne  tradition  (i.e.,  outside  a  conditions  theory  per¬ 
spective)  includes  studies  by  Bergan  et  al.  (1982, 
1984),  Kallison  (1986),  and  Winkles  (1986). 

Different  Learning  Outcomes  Require 
Different  External  Conditions 

In  an  effort  to  find  evidence  to  support  this  basic 
assumption  of  conditions-based  theory,  we  engaged  in 
a  survey  of  research,  looking  across  a  wide  scope.  For 
the  interested  reader.  Table  32.2  presents  representa¬ 
tive  topics  and  referenced  studies  that  illustrate  the 
breadth  of  evidence  generally  supporting  this  assump¬ 
tion  of  the  outcome-referenced,  conditions-based 
model.  The  sample  of  studies  presented  in  the  table 
lend  support  — in  varying  ways  and  from  varying 
standpoints  — to  the  theory  that  different  instructional 


388 


Outcome-Referenced,  Conditions-Based  Theories  and  Models 


TABLE  32.2 

Different  Learning  Outcomes  Require  Different  External  Conditions 


Topic 

Interaction  between  usefulness  of  objectives  and  learning  outcome 

Goal  structure  and  learning  task 

Visual  presentation  mode  and  learning  task 

Evoked  cognitive  strategies  and  learning  outcomes 

Expertise  and  learning  hierarchies 
Adjunct  questions  and  type  of  learning 
Practice 

Feedback  for  different  types  of  learning 

Provided  vs.  evoked  instructional  support  for  different  types  of  learning 


Reviews  and  Studies 

Hartley  and  Davies  (1976);  Yellon  and  Schmidt  (1971) 

Johnson  and  Johnson  (1974) 

Dwyer  and  Dwyer  (1987);  Dwyer  and  Parkhurst  (1982) 

Kiewra  and  Benton  (1987);  Levin  (1986);  Peper  and  Mayer  (1978,1986); 
Pressley  et  al.  (1982);  Shrager  and  Mayer  (1989); 

Anderson  (1990);  Dunn  and  Taylor  (1990,  1994) 

Hamilton  (1985) 

Hannafin  and  Calamaio  (1987);  Hannafin  et  al.  (1986);  Reiber  (1989) 
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outcomes  may  best  be  achieved  with  differing  types 
of  instructional  support.  For  further  expansion  and  dis¬ 
cussion  of  this  basic  tenet  of  conditions  theory  the 
reader  should  refer  to  Chapter  14  in  this  Handbook 
and  Ragan  and  Smith  (2004). 

EXAMPLES  OF 

CONDITIONS-BASED  THEORIES 

Gagne  provided  the  intellectual  leadership  for  an  out¬ 
come-referenced,  conditions-based  theory  of  instruc¬ 
tion.  This  leadership  was  well  explicated  in  a  volume 
dedicated  to  Gagne’s  legacy  (Richey,  2000).  A  number 
of  scholars  have  followed  in  Gagne’s  tradition  by 
developing  more  detailed  prescriptions  of  the  external 
conditions  that  support  different  types  of  learning. 
Three  texts  edited  by  Reigeluth  (1983,  1987,  1999a), 
clearly  delineated  a  number  of  models  that  we  would 
describe  as  outcome-referenced,  conditions-based 
models  of  design.  Some  of  the  models  in  these  texts, 
such  as  the  ones  by  Scandura,  Collins,  and  Keller,  we 
would  not  describe  as  full  conditions-based  models,  as 
they  do  not  describe  the  cognitive  and  instructional 
conditions  for  more  than  one  learning  type .  Other  mod¬ 
els,  particularly  those  presented  in  Volume  II  (Reige¬ 
luth,  1999a),  that  we  did  not  consider  to  be  conditions- 
based  made  few,  if  any,  considerations  of  the  learning 
task.  We  believe  it  is  also  likely  that  the  theorists  who 
developed  these  models  would  not  be  amenable  to  the 
label  of  outcome-referenced,  conditions-based  theory 
and  the  underlying  cognitive  science. 

It  is  not  the  purpose  of  this  chapter  to  replicate  the 
thorough  discussions  of  the  conditions-based  models 
presented  by  Reigeluth;  however,  we  briefly  discuss 
and  compare  some  of  these  models,  because  it  is 
through  comparisons  that  many  of  the  major  issues 
regarding  outcome-referenced,  conditions-based  mod¬ 


els  have  been  revealed  and  exemplified.  We  reference 
research  and  evaluation  studies  that  have  examined  the 
effectiveness  of  the  conditions-based  model  or  theory 
as  a  whole  or  of  an  individual  aspect  of  the  theory.  We 
also  include  in  our  discussion  a  few  models  that  are 
not  presented  in  Reigeluth’s  texts.  Some  of  the  exam¬ 
ples  we  provide  are  arguably  not  instructional  design 
models  at  all  (such  as  the  work  of  Horn,  1976;  Resnick, 
1967;  and  West  et  al.,  1991),  but  they  all  employ, 
reflect,  or  extend  the  conditions-based  theory  propo¬ 
sitions  listed  in  the  introduction  to  this  chapter  in  one 
important  way  or  another. 

Gagne  and  Gagne,  Briggs,  and  Wager 

The  first  full  statement  of  a  conditions-based  theory  of 
instruction  appears  to  have  been  made  by  Gagne  in  the 
early  1960s;  however,  there  was  considerable  conjec¬ 
ture  within  this  paradigm  by  a  variety  of  researchers 
prior  to  Gagne.  In  addition,  Gagne  and  others  have 
developed  a  conditions-based  theory  along  a  variety 
of  lines  of  thinking  through  the  present  day.  We  direct 
the  interested  reader  to  Ragan  and  Smith  (2004)  for  a 
more  complete  discussion  of  the  evolution  of  the  con¬ 
ditions-based  theory  and  the  contributions  of  Gagne. 
As  Gagne  is  generally  identified  as  the  primary  origi¬ 
nator  of  a  conditions-based  model  of  instructional 
design,  an  understanding  of  his  evolution  of  thought 
becomes  foundational  to  understanding  the  theory  that 
extends  beyond  his  contribution.  This  theory  was  the 
basis  of  an  instructional  design  model  presented  in 
Instructional  Design:  Principles  and  Applications 
(Briggs,  1977)  and  Principles  of  Instructional  Design 
(Gagne  and  Briggs,  1979;  Gagne  et  al.,  1992). 

Research  examining  the  validity  of  Gagne’s  theory 
falls  within  one  of  two  types:  studies  that  have  exam¬ 
ined  the  validity  of  Gagne’s  instructional  theory  as  a 
cluster  of  treatment  variables  and  studies  that  have 
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examined  the  individual  propositions  of  the  theory  as 
separate  variables  (as  discussed  earlier).  A  few 
researchers  have  attempted  to  evaluate  the  overall 
value  of  specific  instruction  based  on  Gagne’s  theory 
or  portions  of  Gagne’s  theory  that  were  not  central  to 
the  conditions-based  theory.  Goldberg  (1987),  Mar¬ 
shall  (1986),  Mengel  (1986),  and  Stahl  (1979)  com¬ 
pared  existing  instruction  to  instruction  designed 
according  to  Gagne’s  principles.  These  studies  were 
across  age  groups  and  subject  matters.  Mengal  and 
Stahl  found  significant  differences  in  learning  effects 
for  the  versions  developed  according  to  Gagne’s  prin¬ 
ciples,  whereas  Goldberg  and  Marshall  found  no  sig¬ 
nificant  difference  in  treatments.  Although  we  believe 
such  gross  comparison  studies  to  be  essential  to  the 
development  of  research  in  an  area,  they  suffered  from 
some  of  the  same  threats  to  the  validity  of  conclusions 
that  have  affected  other  comparison  studies.  In  partic¬ 
ular,  it  was  unclear  whether  the  traditional  versions 
were  completely  devoid  of  features  of  Gagne’s  princi¬ 
ples  and  whether  the  Gagnetian  versions  were  fully 
consistent  with  his  principles. 

Merrill’s  Component  Display  Theory 
and  Instructional  Transaction  Theory 

We  consider  Merrill’s  component  display  theory 
(CDT)  (Merrill,  1983)  and  instructional  transaction 
theory  (ITT)  (Merrill,  1999),  which  are  extensions  of 
Gagne’s  theory,  to  be  outcome -referenced,  conditions- 
based  theories  of  instructional  design,  because  they 
prescribe  instructional  conditions  based  on  the  types 
of  desired  learning  outcomes.  In  CDT,  Merrill  classi¬ 
fied  learning  objectives  (or  capabilities)  along  two 
dimensions:  performance  level  (remember,  use,  or 
find)  and  content  type  (facts,  concepts,  principles,  or 
procedures).  Across  the  dimensions,  his  theory 
addressed  12  distinct  categories  of  objectives.  Instead 
of  having  an  inclusive  declarative  knowledge  category 
as  Gagne  did,  though,  CDT  individually  characterized 
remembering  facts,  concept  definitions,  rule  state¬ 
ments,  and  procedural  steps.  Also,  instead  of  having  a 
single  category  for  cognitive  strategies  as  Gagne  did, 
in  CDT  Merrill  proposed  find  operations  for  each  of 
the  content  types  through  his  intersection  of  the  two 
dimensions  (e.g.,  find  a  fact,  find  a  concept,  find  a  rule, 
and  find  a  procedure).  In  ITT,  Merrill  presented  13 
types  of  learning  with  associated  instructional  strate¬ 
gies,  which  he  identified  as  “transactions,”  grouped 
into  three  major  categories:  (1)  component  transac¬ 
tions,  which  involved  identifying,  executing,  and  inter¬ 
preting;  (2)  abstraction  transactions,  which  entailed 
judging,  classifying,  generalizing,  deciding,  and  trans¬ 
ferring;  and  (3)  association  transactions,  which 


involved  propagating,  analogizing,  substituting, 
designing,  and  discovering  (Merrill  et  al.,  1992). 

Merrill  provided  a  rationale  for  his  categorization 
scheme  for  CDT  based  on  “some  assumptions  about  the 
nature  of  subject  matter”  (Merrill,  1983,  p.  298).  The 
rationale  for  content  type  was  based  on  five  operations 
that  he  proposed  could  be  conducted  on  subject  matter: 
(1)  identification  (facts),  (2)  inclusion  (concepts),  (3) 
intersection  (concepts),  (4)  ordering  (procedures),  and 
(5)  causal  operations  (principles).  He  derived  his  per¬ 
formance  levels  from  assumptions  regarding  differences 
in  four  memory  structures:  associative,  episodic,  image, 
and  algorithmic.  Merrill  did  not  explicitly  address  the 
internal  processes  that  accompanied  the  acquisition  of 
each  of  these  categories  of  learning  types. 

Merrill  described  instructional  conditions  as  pre¬ 
sentation  forms  in  CDT  and  classified  these  forms  as 
primary  and  secondary.  Primary  presentation  forms 
had  two  dimensions:  content  (generality  or  instance) 
and  approach  (expository  or  inquisitory).  Secondary 
presentation  forms  were  types  of  elaborations  that  may 
have  extended  the  primary  presentations:  context,  pre¬ 
requisite,  mnemonic,  mathemagenic  help,  representa¬ 
tion  or  alternative  representation,  and  feedback.  Mer¬ 
rill’s  theory  then  further  described  for  each  category 
of  capability  “a  unique  combination  of  primary  and 
secondary  presentation  forms  that  will  most  effectively 
promote  acquisition  of  that  type  of  objective”  (Merrill, 
1983,  p.  283). 

Researchers  have  examined  CDT  in  two  ways:  eval¬ 
uation  in  comparison  to  traditional  approaches  (Keller 
and  Reigeluth,  1982;  Robinson,  1984;  Stein,  1982;  Von 
Hurst,  1984)  and  examination  of  individual  strategy 
variations  within  CDT  (Chao,  1983;  Keller,  1986; 
Sasayama,  1985).  Many  of  the  weaknesses  of  Merrill’s 
theory  were  similar  to  those  of  Gagne’s  theory,  such 
as  the  lack  of  an  explicit  and  empirically  validated 
connection  between  internal  processes  and  external 
events.  Merrill’s  theory,  however,  provided  even  less 
conjecture  on  internal  processes.  It  was  also  less  com¬ 
plete,  as  his  theory  (1)  addressed  only  the  cognitive 
domain,  (2)  did  not  fully  delineate  the  instructional 
conditions  for  the  find  (cognitive  strategies)  category, 
and  (3)  did  not  have  a  category  for  complex  learning 
reflected  in  what  has  often  been  called  problem  solv¬ 
ing.  On  the  other  hand,  a  strength  of  CDT  has  been  its 
evolution  to  tit  the  demands  of  designing  intelligent 
CAI  systems,  as  noted  by  Wilson  (1987). 

Reigeluth:  Elaboration  Theory 

Reigeluth  and  his  associates  (Reigeluth,  1999b;  Reige¬ 
luth  and  Darwazeh,  1982;  Reigeluth  and  Rogers,  1980; 
Reigeluth  and  Stein,  1983;  Reigeluth  et  al.,  1978) 
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developed  the  elaboration  theory  as  a  guide  for  devel¬ 
oping  macro  strategies  for  large  segments  of  instruction, 
such  as  courses  and  units.  The  elaboration  theory  is 
conditions  based  in  nature  ,  as  it  describes  “a)  three  mod¬ 
els  of  instruction;  and  b)  a  system  for  prescribing  those 
models  on  the  basis  of  the  goals  for  a  whole  course  of 
instruction”  (Reigeluth  and  Stein,  1983,  p.  340).  The 
theory  specifies  a  general  model  of  selecting,  sequenc¬ 
ing,  synthesizing,  and  summarizing  content  in  a  simple 
to  more  complex  structure.  The  major  features  of  the 
general  model  include:  (1)  an  epitome  at  the  beginning 
of  the  instruction,  (2)  levels  of  elaboration  of  this  epit¬ 
ome,  (3)  learning-prerequisite  sequences  within  the  lev¬ 
els  of  elaboration,  (4)  a  learner-control  format,  and  (5) 
use  of  analogies,  summarizes,  and  synthesizers. 

The  conditions-based  nature  of  the  model  stemmed 
from  Reigeluth's  specification  of  three  differing  struc¬ 
tures— conceptual,  procedural,  and  theoretical— which 
were  selected  based  on  the  goals  of  the  course.  Reige¬ 
luth  further  suggested  that  conceptual  structures  were 
of  three  types:  parts,  kinds,  and  matrices  (combinations 
of  two  or  more  conceptual  structures).  He  described  two 
kinds  of  procedural  structures:  procedural  order  and  pro¬ 
cedural  decision.  Finally,  he  subdivided  theoretical 
structures  into  two  types:  those  that  described  natural 
phenomena  (descriptive  structures)  and  those  that 
affected  a  desired  outcome  (prescriptive  structures). 

The  nature  of  the  epitome,  sequence,  summarizes, 
prerequisites,  synthesizers,  and  content  of  elaborations 
varied  depending  on  the  type  of  knowledge  structure 
chosen,  which  was  based  on  the  goals  of  the  course; 
for  example  ,  if  the  knowledge  structure  was  concep¬ 
tual,  the  epitome  would  contain  a  presentation  of  the 
most  fundamental  concepts  for  the  entire  course.  If  the 
structure  was  procedural,  the  epitome  would  present 
the  procedure  that  was  most  fundamental  or  repre¬ 
sented  the  shortest  path.  Reigeluth  recommended 
using  Merrill's  CDT  as  the  guideline  for  designing  at 
the  micro  or  lesson  level  within  each  elaboration  cycle. 

Increasingly,  Reigeluth  (1992)  has  placed  more 
emphasis  on  the  importance  of  using  a  simplifying- 
conditions  method  of  sequencing  instruction  than  on 
the  sequencing  and  structuring  of  instruction  based  on 
one  of  the  major  knowledge  structures.  The  simplify- 
ing-conditions  method  suggested  that  designers  “work 
with  experts  to  identify  a  simple  case  that  is  as  repre¬ 
sentative  as  possible  of  the  task  as  a  whole”  (p.  81). 
This  task  would  serve  as  the  epitome  of  the  course, 
with  succeeding  levels  of  elaboration  abating  the  sim¬ 
plifying  conditions  so  that  the  task  became  increas¬ 
ingly  complex.  The  theory  still  retains  some  of  its 
conditions-based  orientation,  though,  as  Reigeluth  has 
suggested,  different  simplifying-conditions  structures 
must  be  developed  for  each  of  the  kinds  of  knowledge 


structures  he  described  (Reigeluth  and  Curtis,  1987; 
Reigeluth  and  Rogers,  1980).  In  recent  years,  Reige¬ 
luth's  (1999b)  discussions  of  elaboration  theory  have 
emphasized  it  as  a  holistic,  learner-centered  approach 
in  an  effort  to  distance  it  from  analytic  approaches 
centering  on  learning  tasks  or  content. 

As  with  the  previous  models,  some  research  has 
evaluated  the  effectiveness  of  instruction  based  on  the 
principles  of  elaboration  theory  in  comparison  to 
instruction  based  on  other  models.  Examples  of  this 
type  of  research  have  included  work  done  by  Beukhof 
(1986)  and  Wedman  and  Smith  (1989).  Given  that 
Reigeluth  (1979)  proposed  that  the  elaboration  theory 
was  a  macro- strategy  theory  effective  for  the  design 
of  units  and  courses  and  recommended  CDT  as  a 
micro-design  strategy  for  lessons,  we  are  not  surprised 
that  researchers  have  not  uniformly  found  positive 
effects  of  the  elaboration  theory  designs  on  shorter 
instruction.  Researchers  have  also  examined  design 
questions  regarding  individual  variables  within  elabo¬ 
ration  theory,  such  as  synthesizers  (McLean,  1983;  Van 
Patten,  1984),  summarizes  (Tilden,  1985),  non-exam¬ 
ples  in  learning  procedures  (Marcone  and  Reigeluth, 
1988),  and  sequencing  (English  and  Reigeluth,  1994). 

Elaboration  theory  is  a  macro-strategy  design  the¬ 
ory  that  was  much  needed  in  the  field  of  instructional 
design  as  an  alternative  to  designing  instruction  seri¬ 
ally  based  on  enabling  objectives.  Throughout  the  evo¬ 
lution  of  elaboration  theory,  Reigeluth  has  proposed 
design  principles  that  have  maintained  a  conditions- 
based  orientation.  Until  Reigeluth’s  work,  many 
designers  had  assumed  that  instruction  should  proceed 
from  one  enabling  objective  to  another  from  the  begin¬ 
ning  of  a  course  to  the  end,  due  to  heavy  emphasis  on 
learning  hierarchy  analysis.  Reigeluth  suggested  a  the¬ 
oretically  sound  alternative  for  designing  large  seg¬ 
ments  of  instruction. 

It  is  unfortunate  that  researchers  in  the  field  have 
not  found  it  pragmatically  possible  to  evaluate  elabo¬ 
ration  theory  in  comparison  to  alternatives  with 
course-level  instruction.  In  light  of  advances  in  cogni¬ 
tive  theory,  Wilson  and  Cole  (1991)  suggested  a  num¬ 
ber  of  recommendations  for  revising  the  elaboration 
theory  to  which  Reigeluth  (1992)  responded.  We  rec¬ 
ommend  these  articles  for  further  reading. 

Smith  and  Ragan 

Rather  than  developing  a  new  conditions-based  model. 
Smith  and  Ragan  (2005)  sought  to  exemplify  and  elab¬ 
orate  Gagne’s  theory.  To  address  what  they  perceived 
to  be  limitations  in  most  conditions-based  models,  they 
focused  on  the  cognitive  process  necessary  for  the 
acquisition  of  each  of  the  different  learning  capabilities. 
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With  regard  to  the  external  conditions  of  learning. 
Smith  and  Ragan  suggested  that  the  events  of  instruc¬ 
tion  as  Gagne  portrayed  them  insufficiently  considered 
learner- generated  and  learner- initiated  learning;  there¬ 
fore,  Smith  and  Ragan  restated  the  events  in  terms  of 
being  either  learner  supplied  or  instruction  supported. 
Instruction,  which  predominated  in  learner-supplied  or 
generative  activities,  characterized  learning  environ¬ 
ments  (Jonassen  and  Land,  2000)  and  new  paradigms 
of  instruction  (Reigeluth,  1999a).  In  the  case  of 
learner-initiated  events,  instruction  facilitated  or 
prompted  the  learner  to  provide  the  cognitive  process¬ 
ing  necessary  for  an  instructional  event.  As  instruction 
provided  increasing  cognitive  support  for  an  instruc¬ 
tional  event,  the  event  was  considered  to  be  more  sup- 
plan  five  (or  mathemagenic)  in  character. 

Based  on  such  findings  that  indicated  that  learners 
perform  better  on  comprehension  and  recall  tests  if 
they  generated  associations  for  themselves  rather  than 
having  the  associations  supplied.  Smith  and  Ragan 
(2005)  suggested  that  instructional  strategies  should 
be  as  generative  as  possible.  Smith  and  Ragan 
acknowledged,  however,  that  it  was  sometimes  neces¬ 
sary  to  provide  more  external  support  “for  learners  to 
achieve  learning  in  the  time  possible,  with  a  limited 
and  acceptable  amount  of  frustration,  anxiety,  and  dan¬ 
ger”  (pp.  145-146).  The  approach  that  Smith  and 
Ragan  recommended  for  instructional  design  was  one 
of  problem-solving,  in  which  designers  determined  the 
amount  of  cognitive  support  required  for  the  events  of 
instruction  based  on  careful  consideration  of  context, 
learner,  and  learning  task.  Smith  and  Ragan  also  pro¬ 
posed  a  model  for  determining  the  optimal  balance  of 
generative  strategies  and  supplantive  strategies  based 
on  context,  learner,  and  task  variables.  Many  methods 
associated  with  constructivism,  including  guided  dis¬ 
covery,  coaching,  and  cognitive  apprenticeship,  were 
examples  of  learner-centered  events  involving  external 
facilitation.  As  proposed  by  Dick  (1997),  these  meth¬ 
ods  could  be  interpreted  as  micro  theories  that 
impacted  the  instructional  strategy  component  of  an 
outcome-referenced,  conditions-based  perspective. 

Smith  (1992)  cited  theoretical  and  empirical  sup¬ 
port  for  some  of  the  learner-task-context-strategy 
relationships  proposed  in  the  comparison  of  generative 
and  supplantive  strategies  (COGSS)  model,  which 
formed  the  basis  of  the  suggested  balance  between 
instruction-supplied  and  learner- generated  events.  In 
Smith's  presentation,  she  proposed  an  agenda  for  val¬ 
idation  of  the  model.  Smith  and  Ragan  (2005)  stated 
that  the  relative  advantages  afforded  by  generative  or 
supplantive  forms  had  not  been  thoroughly  and  empir¬ 
ically  investigated;  yet,  “bodies  of  theory  and  research 
suggest  that  neither  approach  is  universally  superior 


but  that  many  factors  may  influence  the  efficacy  of  one 
instructional  approach  over  the  other”  (p.  143). 

Smith  and  Ragan  (2005)  referenced  several  areas  of 
research  related  to  generative  and  supplantive  strategies, 
such  as  research  on  generative  vs.  mathemagenic  teach¬ 
ing  methods  (Jonassen,  1985;  Osborne  and  Wittrock, 
1985;  Wittrock,  1974),  learner  control  in  computer- 
based  instruction  (Hannafin,  1984;  Steinberg,  1977; 
Tennyson,  1984),  discovery  vs.  expository  learning 
(Herman,  1969;  Ray,  1961),  and  cognitive  capacity  and 
allocation  of  mental  resources  (Britton  et  al.,  1978;  Bur¬ 
ton  et  al.,  1986;  Craik  and  Lockhart,  1972;  Duncan, 
1980;  Watkins,  1983).  Additionally,  Smith  and  Ragan 
(2005)  proposed  principles  for  making  decisions  regard¬ 
ing  the  optimal  design  position  along  the  supplantive  - 
generative  continuum,  based  on  variables  within  the 
learners,  context,  and  task.  They  noted  that  the  current 
state  of  theory  and  research  allowed  only  hypothetical 
principles  and  suggested  research  to  investigate  the  prin¬ 
ciples  and  interactions  of  the  variables  to  determine 
what  recommendations  and  overall  principles  could  be 
proposed  within  this  conditions-based  perspective. 

Merrill,  Li,  and  Jones:  ID2 

In  reaction  to  a  number  of  limitations  that  they  per¬ 
ceived  in  existing  instructional  design  theories  and 
models  (including  Merrill’s  own  models),  Merrill,  Li, 
and  Jones  (1990a,b)  set  out  to  construct  a  second  gen¬ 
eration  theory  of  instructional  design  (ID2).  One  of  the 
specific  goals  of  its  developers  was  to  expedite  the 
design  of  an  automated  ID  system  (ID  Expert)  and 
thereby  expedite  the  instructional  design  process  itself. 
Ultimately,  the  developers  hoped  that  the  system 
would  provide  both  authoring  and  delivery  environ¬ 
ments  that  were  increasingly  grounded  in  a  knowledge 
base  and  rule  base.  Of  all  the  models  we  have  described 
in  this  chapter,  ID2  has  reflected  the  highest  ambition 
in  the  goal  to  thoroughly  prescribe  the  instructional 
conditions  for  each  type  of  learning.  The  ID2  model 
has  been  developed  ( 1 )  to  analyze, represent,  and  guide 
instruction  to  teach  integrated  sets  of  knowledge  and 
skill;  (2)  to  produce  pedagogic  prescriptions  about 
selection  and  sequence;  and  (3)  to  be  an  open  system 
capable  of  responding  to  new  theory.  As  Merrill  and 
his  associates  have  elaborated  on  the  relationships 
between  outcomes  and  internal-external  conditions, 
the  ID2  model  has  retained  its  conditions-based  orien¬ 
tation  (Merrill  et  al.,  1990,  p.  8): 

a)  A  given  learned  performance  results  from  a  given  orga¬ 
nized  and  elaborated  cognitive  structure,  which  we  will 
call  a  mental  model.  Different  learning  outcomes  require 
different  types  of  mental  models;  b)  the  construction  of 
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a  mental  model  by  a  learner  is  facilitated  by  instruction 
that  explicitly  organizes  and  elaborates  the  knowledge 
being  taught,  during  the  instruction;  c)  there  are  different 
organizations  and  elaborations  of  knowledge  required  to 
promote  different  learning  outcomes. 

Within  ID2,  outcomes  of  instruction  were  considered 
to  be  enterprises  composed  of  entities,  activities,  or 
processes,  which  might  loosely  be  interpreted  as  con¬ 
cepts,  procedures,  and  principles,  respectively.  Merrill 
and  his  associates  have  spent  a  vast  amount  of  effort 
describing  the  structure  of  knowledge  relating  to  these 
types  of  knowledge  and  how  these  types  of  knowledge 
relate  to  each  other. 

Merrill  and  his  colleagues  have  described  a  number 
of  external  conditions  or  instructional  methods  that 
could  be  controlled  by  either  system  or  learner.  These 
conditions  were  described  as  transactions  of  various 
classes,  and  evidence  of  Merrill’s  CDT  could  be  found 
in  the  prescriptions  for  these  transactions.  To  create 
this  system  based  on  his  ID2  model,  Merrill  et  al. 
(1991,  1992a,b,  1993)  have  attempted  to  identify  the 
decisions  that  designers  must  make  about  the  types  of 
information  in  order  to  build  them  into  the  system  and 
the  methods  by  which  this  information  could  be  made 
available  to  learners.  They  have  conducted  similar 
analyses  of  information  that  could  be  made  available 
to  learners  when  learning  entities,  activities,  or  pro¬ 
cesses.  In  addition  to  detailing  the  options  for  peda¬ 
gogy  and  information  that  could  be  made  available  in 
instruction,  the  system  developers  could  also  establish 
the  rules  by  which  the  system  made  choices  for  select¬ 
ing  options  to  present  to  learners. 

Parts  of  the  ID  Expert  system  have  been  evaluated 
by  Spector  and  Muraida  (1991)  and  by  Canfield  and 
Spector  (1991).  As  yet,  there  have  been  no  comparison 
data  with  more  conventional  design  processes.  In  their 
effort  to  carefully  explicate  the  necessary  knowledge 
for  learning  and  instruction  as  well  as  the  means  by 
which  these  entities  interacted  with  one  another,  the 
developers  have  created  a  model  that  is  quite  complex. 
One  benefit  of  the  model  is  it  has  reflected  and  made 
concrete  much  of  the  complexity  of  the  instructional 
design  process.  Unfortunately,  it  seems  that  terminol¬ 
ogy  has  shifted  during  development.  Among  the  criti¬ 
cisms  that  have  frequently  been  leveled  at  ID2  are  (1) 
its  utility  when  used  by  novices,  (2)  the  lack  of  evi¬ 
dence  of  theory  base,  (3)  issues  regarding  sufficient 
agreement  to  generate  strategies,  and  (4)  the  likelihood 
of  sameness  of  results  in  multiple  applications. 

ID2  and  its  accompanying  ID  Expert  are  no  longer 
alone  in  the  development  of  ID  expert  systems 
(Kasowitz,  1998).  Other  expert  systems  that  have  been 
developed  as  part  of  the  Advanced  Instructional  Design 


Advisor  (AIDA)  research  project  (Muraida  and  Spec- 
tor,  1993;  Spector  et  al.,  1991)  include  the  Guided 
Approach  to  Instructional  Design  Advising  (GAIDA), 
which  used  tutorials  and  context-specific  advice  and 
examples,  and  the  Experimental  Advanced  Instruc¬ 
tional  Design  Advisor  (XAIDA),  which  used  the  trans¬ 
actional  theory  framework  to  encapsulate  context-spe¬ 
cific  knowledge.  Additional  advisory  system  models 
have  also  been  proposed,  such  as  Duchastel’s  (1990) 
Instructional  Design  Advanced  Workbench.  Some  of 
these  automated  instructional  design  tools  have  oper¬ 
ationalized  instructional  design  theories  and  models 
by  focusing  on  the  cognitive  aspects  of  instructional 
design,  while  other  tools  have  highlighted  the  proce¬ 
dural  steps  of  instructional  design.  Other  tools  have 
supported  only  the  production  phase  of  instructional 
design.  Some  of  these  tools  have  been  within  the 
framework  of  an  outcome-referenced,  conditions- 
based  tradition,  but  others  have  not.  Continued 
research  on  the  efficiency  and  effectiveness  of  these 
models  and  tools,  as  well  as  comparisons  across  the 
tools,  would  be  highly  lucrative  areas  for  further  study. 

Other  Applications  of 
Conditions-Based  Theory 

The  theories  and  models  discussed  in  the  previous 
sections  represent  the  strongest  body  of  development, 
research,  and  evaluation  on  outcome-referenced,  con- 
ditions-based  perspectives  within  instructional  design. 
Mention  of  additional  theories  and  models  that  have 
less  extensive  research  and  evaluation  or  represent 
somewhat  incomplete  instructional  design  models  are 
important  given  that:  (1)  a  number  of  notable  scholars 
within  and  outside  the  instructional  design  field  have 
utilized  a  conditions-based  theory  as  a  basis  for  much 
of  their  work,  and  (2)  they  illustrate  how  pervasive  and 
influential  the  conditions-based  theory  has  been.  Six 
examples  of  such  theories  and  models  include:  (1) 
Landa’s  (1983)  algoheuristic  theory  of  instruction,  or 
Landamatics,  which  made  a  distinction  between 
knowledge  and  skills  seemingly  equivalent  to  Gagne’s 
notion  of  declarative  and  procedural  knowledge;  (2) 
Tennyson  and  Rasch's  (1988)  description  of  a  model 
of  how  instructional  prescriptions  might  be  linked  to 
cognitive  learning  theory;  (3)  Jonassen  et  al.’s  (1991) 
decision  model  for  selecting  strategies  and  tactics  of 
instruction  based  on  three  levels  of  decisions;  (4) 
Horn’s  (1976)  approach  to  text  design  which  had  many 
elements  of  a  design  model  and  clearly  reflected  a 
conditions-based  set  of  assumptions;  (5)  West  et  al.’s 
(1991)  work,  which  referred  to  three  kinds  of  knowl¬ 
edge  and  described  cognitive  strategies  that  could  sup¬ 
port  the  acquisition  of  each  of  these  learning  types, 
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and  (6)  E.  Gagne’s  work  (Gagne  et  al.,  1993),  which 
was  primarily  descriptive  rather  than  prescriptive  but 
utilized  the  conditions-based  theory  as  she  discussed 
the  internal  processes  required  in  the  acquisition  of 
each  of  the  types  of  knowledge  and  the  instructional 
support  that  could  promote  this  acquisition.  For  a 
review  of  each  of  these  examples,  see  Ragan  and  Smith 
(2004). 

CONCLUSIONS 

In  its  overview  and  analysis  of  the  assumptions  of  the 
outcome-referenced,  conditions-based  perspective,  the 
development  and  research  on  various  theories  and 
models,  this  chapter  presented  numerous  areas  for  fur¬ 
ther  research  that  would  serve  to  improve  the  theories 
that  guide  the  development  of  instruction.  Despite  the 
call  made  by  various  authors  for  continued  rigorous, 
empirical  evaluation  of  design  practice  and  tools,  very 
few  models  have  validated  answers  such  as  whether  a 
particular  model  has  enhanced  the  quality  of  instruc¬ 
tion  (Botturi,  2005).  Dick  (1997)  suggested  that  we 
improve  our  current  theories  through  revision  that 
reflected  the  outcomes  that  were  being  achieved  with 
its  current  use  and  was  exemplified  by  the  work  of 
English  and  Reigeluth  (1996,  p.  48): 

For  example,  when  a  design  report  has  been  critiqued 
and  revised,  why  not  use  that  information  to  determine 
how  the  process  that  led  to  the  development  of  the  design 
could  be  improved?  If  the  design  was  not  acceptable  to 
the  client,  what  went  wrong  and  why?  Was  it  a  skills 
problem,  or  should  the  process,  i.e.,  our  theory,  be 
improved? 

This  same  directive  was  presented  by  Wang  and 
Hannafin  (2005)  in  their  call  for  increased  use  of  the 
design-based  research  paradigm.  A  thorough  review  of 
their  identification  of  the  underlying  characteristics 
and  goals  of  design-based  research  approaches  and 
proposed  principles  for  implementing  design-based 
research  is  urged  by  all  theorists  and  designers  inter¬ 
ested  in  furthering  research  aimed  “to  improve  educa¬ 
tional  practices  through  iterative  analysis,  design, 
development,  and  implementation  based  on  collabora¬ 
tion  among  researchers  and  practitioners  in  real-world 
setting,  and  leading  to  contextually-sensitive  design 
principles  and  theories”  (pp.  6-7). 

Although  Wang  and  Hannafin  focused  their  prin¬ 
ciples  on  the  application  of  design-based  research  to 
technology-enhanced  learning  environments,  others 
have  echoed  their  call  across  contexts  and  areas  of 
application  (Reigeluth  and  Frick,  1999;  Richey  et  al., 
2003;  Richey  and  Nelson,  1996;  van  den  Akker,  1999). 


We  believe  that  full  engagement  in  design-based 
research  approaches  is  the  most  likely  category  of 
future  research  that  will  serve  to  inform,  improve,  and 
advance  outcome -referenced,  conditions-based  theo¬ 
ries  and  models  as  well  as  advancing  our  knowledge 
in  other  perspectives  on  the  design  of  instruction. 
Additionally,  continued  research  in  the  direction  of 
Merrill  and  associates  (see  Chapter  14  in  this  Hand¬ 
book ;  see  also  Merrill,  2006)  that  proposes  a  frame¬ 
work  for  integrated  research  on  instructional  strategies 
that  examines  the  interrelationships  among  instruc¬ 
tional  strategy  principles  would  also  serve  to  advance 
instructional  thinking  in  the  context  of  conditions- 
based  theories  and  models. 

We  can  draw  some  conclusions  from  this  review. 
We  reflected  on  the  conclusions  drawn  in  our  chapter 
in  the  second  edition  of  this  volume  (Ragan  and  Smith, 
2004)  and  have  modified  them  accordingly.  Conditions 
models  have  a  long  history  of  interest  in  psychology, 
educational  psychology,  and  instructional  technology. 
This  history  illustrates  work  that  may  not  be  widely 
known  among  instructional  technologists  today:  work 
that  can  be  instructive  as  to  the  actual  base  and  signif¬ 
icance  of  the  outcome-referenced,  conditions-based 
approach.  Perhaps  we  will  see  fewer  erroneous  state¬ 
ments  in  our  literature  about  what  is  known  regarding 
types  of  learning,  learning  hierarchies,  and  conditions 
of  learning.  We  also  envision  a  renewed  interest  in 
building  on  this  research  base  with  continued  research 
examining  the  theoretical  propositions  as  well  as  the 
application  of  theory  to  practical  contexts. 

We  continue  to  see  utility  in  thinking  of  learning 
as  more  than  one  kind  of  thing,  especially  for  prac¬ 
titioners.  It  is  too  easy,  in  the  heat  of  practitioners’ 
struggles,  to  slip  into  the  assumption  that  all  knowl¬ 
edge  is  declarative  (as  is  so  often  seen  in  the  learning 
outcomes  statements  of  large-scale  instructional  sys¬ 
tems)  or  all  problem  solving  (as  is  so  often  assumed 
in  the  pronouncements  of  pundits  and  critics  of  public 
education)  and,  as  a  result,  fail  to  consider  either  the 
vast  arena  of  application  of  declarative  knowledge  or 
the  multitude  of  prerequisites  for  problem  solving.  It 
is  unhelpful  to  develop  new  systems  of  types  of  learn¬ 
ing  for  the  mere  purpose  of  naming.  Improvements 
in  categorization  schemes  should  be  based  on  known 
differences  in  cognitive  processing  and  required  dif¬ 
ferences  in  external  conditions .  Our  field  would  ben¬ 
efit  from  research  designed  to  provide  evidence  to 
support  a  current  categorization  scheme  or  with  the 
potential  to  identify  additional  unique  types  of  learn¬ 
ing  based  on  new  evidence  from  cognition  and 
instruction. 

We  have  reached  a  conclusion  about  the  work  of 
Robert  Gagne  that  we  would  like  to  share  and  suggest 
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that  readers  examine  their  own  conclusions  from 
reading.  We  find  Gagne’s  work,  cast  within  so  much 
that  preceded  and  followed  it,  to  remain  both  dom¬ 
inating  in  its  appeal  and  utility  and,  paradoxically, 
somewhat  limited  and  in  need  of  improvement.  The 
utility  and  appeal  of  this  work  appear  to  derive 
greatly  from  the  solid  scholarship  and  cogent  writing 
that  Gagne  brought  to  bear,  as  well  as  his  willingness 
to  change  the  formulation  to  keep  up  with  changing 
times  and  new  knowledge.  Many  of  the  gaps  and 
flaws,  in  keeping  with  the  paradox,  appear  to  be  a 
product  of  the  very  changes  that  he  made  to  keep  up 
with  current  interests.  We  believe  those  changes  to 
be  beneficial  in  the  main  but  see  a  clear  need  for 
systematic  and  rigorous  scholarship  on  issues  raised 
by  those  changes. 

Conditions-based  theory  is  characterized  by  a  par¬ 
ticular  combination:  on  the  one  hand,  its  utility  in 
helping  specify  instructional  strategies,  and,  on  the 
other  hand,  the  sizable  gaps  and  inconsistencies  that 
exist  in  current  formulations  (although  in  an  absolute 
sense  these  gaps  are  substantial,  in  a  relative  sense, 
compared  to  competing  perspectives,  the  gaps  are  from 
within  a  well-tested  framework).  The  combination  of 
high  utility  and  gaps  in  knowledge  creates  a  need  for 
more  work.  We  have  described  in  this  chapter  many 
fruitful  areas  for  further  research. 

Substantial  weakness  exists  in  the  tie  between  cat¬ 
egories  of  learning  and  external  conditions  of  learning. 
What  is  missing  is  the  explication  of  the  internal  con¬ 
ditions  involved  in  the  acquisition  and  storage  of  dif¬ 
ferent  kinds  of  learning.  Research  on  the  transition 
from  expert  to  novice  and  artificial  intelligence 
research  that  attempts  to  describe  the  knowledge  of 
experts  should  be  particularly  fruitful  in  helping  us  fill 
this  void.  Perhaps  this  void  is  a  result  of  the  failure  to 
place  sufficient  emphasis  on  qualitative  analysis  of 
learners’  actions  and  thoughts  in  our  field. 

Research  supports  the  conclusion  that  different 
external  events  of  instruction  lead  to  different  kinds  of 
learning,  especially  looking  at  the  declarative  or  pro¬ 
cedural  level.  What  appears  to  be  lacking  is  any  sys¬ 
tematic  body  of  research  directly  on  the  central  tenet, 
not  just  of  conditions-based  theory  but  of  practically 
anyone  who  would  attempt  to  teach,  much  less  design, 
instruction:  What  is  the  relationship  between  internal 
learner  conditions  and  subsequent  learning  from 
instruction?  This  topic  seems  to  be  a  far  cry  from 
studies  that  would  directly  inform  designers  about  pro¬ 
cedures  and  techniques,  yet  a  very  great  deal  seems  to 
hinge  on  this  one  question.  With  more  insight  into  it, 
many  quibbles  and  debates  may  disappear  and  the 
work  of  translation  into  design  principles  may  begin 
at  a  new  level  of  efficacy. 
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ABSTRACT 

Technology  may  be  more  productively  utilized  when 
it  is  combined  with  cooperative  learning.  There  are 
four  types  of  cooperative  learning:  formal  cooperative 
learning,  informal  cooperative  learning,  cooperative 
base  groups,  and  academic  controversy.  Cooperative 
learning’s  worldwide  use  is  based  on  a  well-formulated 
theory  (i.e.,  social  interdependence  theory)  that  has 
been  validated  by  numerous  research  studies  and  oper¬ 
ationalized  into  practical  procedures  that  can  be  used 
at  any  level  of  education.  Technology-supported  coop¬ 
erative  learning  tends  to  increase  achievement  (both 
academic  achievement  and  learning  how  to  use  tech¬ 
nology),  positive  attitudes  (toward  technology  and 
cooperation) ,  healthy  development  (cognitive  develop¬ 
ment,  learning  control,  social  competencies),  positive 
relationships  with  team  members  (including  social 
presence),  and  innovation  in  groupware  and  hardware. 
Cooperative  learning  and  technology-supported 
instruction  have  complementary  strengths.  The  com¬ 
puter,  for  example,  can  control  the  flow  of  work,  mon¬ 
itor  accuracy,  give  electronic  feedback,  and  do  calcu¬ 
lations.  Cooperative  learning  provides  a  sense  of 
belonging,  the  opportunity  to  explain  and  summarize 
what  is  being  learned,  shared  mental  models,  social 
models,  respect  and  approval  for  efforts  to  achieve, 
encouragement  of  divergent  thinking,  and  interper¬ 
sonal  feedback  on  academic  learning  and  the  use  of 
the  technology.  The  use  of  cooperative  learning  with 
technology-assisted-instruction  results  in  more  posi¬ 
tive  attitudes  toward  technology  (especially  by 
females)  and  allows  for  argumentation  (i.e.,  construc¬ 
tive  controversy)  to  be  part  of  lessons  utilizing  tech¬ 
nology.  Cooperative  learning  is  an  important  part  of 
enhancing  the  effectiveness  of  interacting  around  com¬ 
puters  and  interacting  through  computers  (e.g.,  local 
networks,  e-mail,  videogames  and  simulations ,  adven¬ 
ture  learning,  and  interacting  with  electronic  pedagog¬ 
ical  agents). 


KEYWORDS 

Collaborative  learning:  Students  working  together  to 
maximize  their  own  and  each  other’s  learning  (i.e., 
to  achieve  shared  learning  goals);  synonymous 
with  cooperative  learning  in  this  chapter. 

Competitive  learning:  Students  working  individually 
to  achieve  a  grade  or  recognition  attainable  by  or 
restricted  to  only  a  few. 

Computer-supported  cooperative  learning:  The 
instructional  use  of  technology  combined  with  the 
use  of  cooperative  learning. 

Constructive  controversy:  Occurs  when  group  mem¬ 
bers  have  different  information,  perceptions,  opin¬ 
ions,  reasoning  processes,  theories,  and  conclu¬ 
sions,  and  they  must  reach  agreement  in  order  to 
make  progress  or  proceed. 

Cooperative  learning:  Students  working  together  to 
maximize  their  own  and  each  other’s  learning  (i.e., 
to  achieve  shared  learning  goals);  synonymous 
with  collaborative  learning  in  this  chapter. 

Positive  interdependence:  Occurs  when  individuals 
perceive  that  they  can  reach  their  goals  if  and  only 
if  the  other  individuals  with  whom  they  are  coop¬ 
eratively  linked  also  reach  their  goals. 


TECHNOLOGY  IN  THE  CLASSROOM 

We  live  in  a  historical  period  where  knowledge  is  the 
most  critical  resource  for  social  and  economic  devel¬ 
opment  and  people  need  to  be  able  to  participate  in  a 
networked,  information-based  society.  Whereas  previ¬ 
ously  people  engaged  in  manufacturing-based  work 
where  they  generally  competed  with  or  worked  inde¬ 
pendently  from  each  other,  people  now  engage  in  infor¬ 
mation-  and  technological-rich  work  where  they  work 
in  teams.  People  need  to  be  able  to  work  cooperatively 
designing,  using,  and  maintaining  the  tools  of  technol¬ 
ogy.  Technology  and  teamwork  will  continue  to  play 
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a  larger  role  in  most  people’s  lives.  Children,  adoles¬ 
cents,  and  young  adults  have  no  choice  but  to  develop 
and  increase  their  technological  and  teamwork  literacy. 
There  is  no  better  place  for  them  to  begin  than  in 
school.  Learning  in  cooperative  groups  (both  face  to 
face  and  online)  while  utilizing  the  tools  of  technology 
should  occur  in  all  grade  levels  and  subject  areas. 

The  failure  of  schools  to  adopt  available  instruc¬ 
tional  technologies  and  to  maintain  (let  alone  contin¬ 
uously  improve)  their  use  may  be  at  least  in  part  due 
to  two  barriers:  (1)  the  individual  assumption  under¬ 
lying  most  hardware  and  software  development,  and 
(2)  the  failure  to  utilize  cooperative  learning  as  an 
inherent  part  of  using  instructional  technologies.  Tech¬ 
nology  can  either  facilitate  or  obstruct  learning, 
depending  on  the  conditions  under  which  it  is  used. 
The  ways  in  which  technology  may  enhance  or  inter¬ 
fere  with  learning  have  not  been  conceptualized, 
placed  in  a  theoretical  framework,  researched,  or 
applied  in  classrooms.  The  purpose  of  this  chapter  is 
to  clarify  the  relationship  between  instructional  tech¬ 
nologies  and  cooperation  among  students.  To  under¬ 
stand  how  cooperative  learning  may  be  used  with  tech¬ 
nology,  the  nature  of  cooperative  learning  must  be 
defined,  the  theoretical  foundations  on  which  it  is 
based  must  be  clarified,  the  basic  elements  that  make 
cooperation  work  must  be  defined,  and  the  research 
validating  its  use  must  be  reviewed.  At  that  point,  the 
interrelationships  between  cooperative  learning  and 
technology-supported  instruction  can  be  noted  and 
their  complementary  strengths  delineated.  The  future 
of  technology  supported  cooperative  learning  can  then 
be  discussed. 


THE  INDIVIDUAL  ASSUMPTION 

Before  the  1990s,  most  of  the  research  on  computer- 
supported  learning  was  based  on  the  single-learner 
assumption.  The  individual  assumption  is  that  instruc¬ 
tion  should  be  tailored  to  each  student’s  personal  apti¬ 
tude,  learning  style,  personality  characteristics,  moti¬ 
vation,  and  needs.  Computers  were  viewed  as  an 
important  tool  for  individualizing  learning  experi¬ 
ences,  especially  for  programmed  learning  programs 
and  learning  experiences  derived  from  constructivist 
principles  (Crook,  1994).  Many  hardware  and  software 
designers  (as  well  as  teachers)  assumed  that  all  tech¬ 
nology-supported  instruction  should  be  structured 
individualistically  (one  student  to  a  computer),  and 
computer  programs  were  written  accordingly. 

The  ability  of  designers  to  adapt  instruction 
sequences  to  the  cognitive  and  affective  needs  of  each 
learner,  however,  is  limited  by  several  factors.  One  is 


the  substantial  variation  that  exists  in  types  of  learning 
styles  and  personality  traits  and,  although  many  of 
them  are  sometimes  correlated  with  achievement,  few 
have  been  shown  to  predict  achievement  consistently. 
Another  is  that  little  agreement  exists  on  how  to  trans¬ 
late  differences  in  learning  styles  and  personal  traits 
into  instructional  prescriptions.  The  only  design  rule 
that  is  widely  accepted  is  that  students  should  control 
the  flow  of  information.  A  third  is  that  it  is  time  con¬ 
suming  and  expensive  to  create  algorithms  to  adapt 
instruction  to  individual  needs  and  design  and  produce 
multiple  versions  of  lessons.  Finally,  each  person  has 
multiple  characteristics  and  traits  that  interact  in 
unknown  and  unpredictable  ways.  Instruction  cannot 
truly  adapt  to  the  complex  of  characteristics  and  traits 
that  make  up  one  person. 

More  recently,  Web  courses  have  been  developed 
with  the  assumption  that  each  student  will  be  taking 
the  course  individualistically.  The  individualistic 
assumption  may  be  as  strong  today  in  Web  courses  as 
it  was  in  the  1970s  and  1980s  in  the  instructional  use 
of  the  computer.  In  addition  to  all  of  the  problems 
noted  above,  such  individualized  instruction  assumes 
that  students  will  work  in  isolation  (which  may  lower 
motivation  by  increasing  boredom,  frustration,  anxiety, 
and  the  view  that  learning  is  impersonal),  with  only 
their  own  and  the  course-provided  resources ,  thus  they 
lack  support  and  encouragement  from  their  peers  and 
cannot  take  advantage  of  the  cognitive  benefits  asso¬ 
ciated  with  explaining  to  peers  and  developing  shared 
mental  models. 

The  omission  of  social  interaction  in  technologi¬ 
cally  assisted  learning  experiences  worried  many  edu¬ 
cators  in  the  1980s  (Baker,  1985;  Cuban,  1986;  Hawk¬ 
ins  et  al.,  1982;  Isenberg,  1992).  Given  the  limitations 
and  shortcomings  of  the  individual  assumption,  tech¬ 
nology  may  be  more  effective  when  it  is  combined 
with  cooperative  learning.  The  spontaneous  coopera¬ 
tion  often  reported  around  technology,  in  addition, 
both  casts  doubt  on  the  individual  assumption  and 
points  toward  the  use  of  cooperative  learning  in  tech¬ 
nology-supported  instruction  (Dwyer,  1994).  To  use 
cooperative  learning,  however,  educators  must  under¬ 
stand  its  nature. 


THE  NATURE  OF 
COOPERATIVE  LEARNING 

There  are  advantages  to  embedding  technology-sup¬ 
ported  instruction  in  cooperative  learning.  Cooperative 
learning  may  be  distinguished  from  traditional  direct 
transfer  models  of  instruction  in  which  the  instructor 
is  assumed  to  be  the  distributor  of  knowledge  and 
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skills.  To  understand  computer-supported  cooperative 
learning,  one  must  understand  the  nature  of  coopera¬ 
tive  learning,  the  theoretical  foundations  on  which  it 
is  based,  the  research  validating  its  use,  the  distinctions 
between  cooperative  learning  and  other  types  of 
instructional  groups,  and  the  basic  elements  that  make 
cooperation  work. 

Cooperative  Learning 

Cooperative  learning  is  students  working  together  to 
maximize  their  own  and  each  other’s  learning  (i.e., 
achieve  shared  learning  goals)  (Johnson  et  al.,  2002). 
Cooperative  learning  is  usually  contrasted  with  com¬ 
petitive  and  individualistic  learning.  Competitive 
learning  is  students  working  against  each  other  to 
achieve  an  academic  goal  such  as  a  grade  that  only 
one  or  a  few  students  can  attain.  Individualistic  learn¬ 
ing  is  students  working  by  themselves  to  accomplish 
learning  goals  unrelated  to  those  of  the  other  students . 
Technology-supported  cooperative  learning  (TSCL), 
or  computer- supported  cooperative  learning  (CSCL), 
exists  when  the  instructional  use  of  technology  is  com¬ 
bined  with  the  use  of  cooperative  learning.  Although 
the  use  of  instructional  technology  has  progressed 
beyond  the  computer,  and  it  may  be  more  accurate  to 
discuss  technology-supported  cooperative  learning  in 
this  chapter,  for  historical  reasons  the  term  computer- 
supported  cooperative  learning  will  be  used  in  most 
cases  in  this  chapter. 

Collaborative  Learning 

Cooperative  learning  is  sometimes  differentiated  from 
collaborative  learning ,  which  has  its  roots  in  the  work 
of  Sir  James  Britton  and  others  in  England  in  the  1970s 
(Britton,  1990).  Citing  Vygotsky  (1978),  Britton  noted 
that,  just  as  the  individual  mind  is  derived  from  society, 
a  student’s  learning  is  derived  from  the  community  of 
learners.  Britton  was  quite  critical  of  educators  who 
wish  to  provide  specific  definitions  of  the  teacher’s  role. 
He  recommended  placing  students  in  groups  and  letting 
them  generate  their  own  culture,  community,  and  pro¬ 
cedures  for  learning.  Britton  believed  in  natural  learn¬ 
ing  (learning  something  by  making  intuitive  responses 
to  whatever  group  members’  efforts  generate)  rather 
than  training  (the  application  of  explanations,  instruc¬ 
tions,  structures,  or  recipes  for  action).  The  source  of 
learning  is  interpersonal;  learning  is  derived  from  dia¬ 
logs  and  interactions  with  other  students  and  sometimes 
the  teacher.  Britton  viewed  structure  provided  by  teach¬ 
ers  as  manipulation  that  creates  training,  not  learning; 
therefore,  teachers  should  assign  students  to  groups, 
provide  no  guidelines  or  instructions,  and  stay  out  of 


their  way  until  the  class  is  over.  As  an  educational 
procedure,  therefore,  collaborative  learning  has  histor¬ 
ically  been  much  less  structured  and  more  student 
directed  than  cooperative  learning,  with  only  vague 
directions  given  to  teachers  about  its  use.  The  vagueness 
in  the  role  of  the  teacher  and  students  results  in  a  vague¬ 
ness  of  definition  of  the  nature  of  collaborative  learning. 

Many  of  the  current  proponents  of  collaborative 
learning,  however,  have  developed  definitions  beyond 
Britton’s.  Dillenbourg  (1999),  for  example,  stated  that 
collaborative  learning  is  a  situation  in  which  two  or 
more  people  learn  or  attempt  to  learn  something 
together.  He  noted  that  the  terms  “two  or  more,”  “learn 
something,”  and  “together”  are  ambiguous.  He  then 
added  that  the  concept  of  collaboration  is  in  itself 
ambiguous,  as  the  term  may  refer  to  the  situation,  inter¬ 
actions,  learning  mechanisms,  or  its  effects.  What 
seems  to  be  lacking  is  a  foundational  theory  of  collab¬ 
oration  similar  to  Deutsch’s  (1949)  theory  of  coopera¬ 
tion  and  competition  that  gives  a  single,  unambiguous 
definition  of  collaboration.  In  breaking  with  Britton’s 
(1990)  position,  furthermore,  current  proponents  of 
collaborative  learning  recommend  structures,  proce¬ 
dures,  and  cognitive  strategies  to  scaffold  interaction 
among  students  and  joint  knowledge  building.  This 
more  structured  approach  blurs  the  differences  between 
cooperative  and  collaborative  learning;  therefore,  the 
two  terms  ( cooperative  learning  and  collaborative 
learning )  are  increasingly  interchangeable  and  synon¬ 
ymous.  In  this  chapter,  the  two  concepts  are  considered 
to  be  synonymous. 

Types  of  Cooperative  Learning 

The  four  types  of  cooperative  learning  that  may  be 
used  in  combination  with  instructional  technology  are 
(Johnson  et  al.,  2002):  (1)  formal  cooperative  learning, 
(2)  informal  cooperative  learning,  (3)  cooperative  base 
groups,  and  (4)  constructive  controversy.  Formal  coop¬ 
erative  learning  consists  of  students  working  together, 
for  one  class  period  to  several  weeks,  to  achieve  shared 
learning  goals  and  complete  jointly  specific  tasks  and 
assignments  (such  as  problem  solving,  completing  a 
curriculum  unit,  writing  a  report,  conducting  an  exper¬ 
iment,  or  having  a  dialog  about  assigned  text  material). 
Any  course  requirement  or  assignment  may  be  struc¬ 
tured  to  be  cooperative.  In  formal  cooperative  learning, 
teachers: 

•  Make  a  number  of  preinstructional  deci¬ 
sions.  A  teacher  has  to  decide  on  the  objec¬ 
tives  of  the  lesson  (both  academic  and  social 
skills  objectives),  size  of  groups,  the  method 
of  assigning  students  to  groups,  the  roles 
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students  will  be  assigned,  the  materials 
needed  to  conduct  the  lesson,  and  the  way 
the  room  will  be  arranged. 

•  Explain  the  task  and  the  positive  interdepen¬ 
dence.  A  teacher  clearly  defines  the  assign¬ 
ment,  teaches  the  required  concepts  and 
strategies,  specifies  the  positive  interdepen¬ 
dence  and  individual  accountability,  gives 
the  criteria  for  success,  and  explains  the 
expected  social  skills  to  be  engaged  in. 

•  Monitor  students’  learning  and  intervene 
within  the  groups  to  provide  task  assistance 
or  to  increase  students’  interpersonal  and 
group  skills.  A  teacher  systematically 
observes  and  collects  data  on  each  group  as 
it  works.  When  necessary,  the  teacher  inter¬ 
venes  to  assist  students  in  completing  the  task 
accurately  and  working  together  effectively. 

•  Evaluate  students’  learning  and  help  stu¬ 
dents  process  how  well  their  groups  func¬ 
tioned.  Students’  learning  is  carefully 
assessed  and  their  performances  are  evalu¬ 
ated.  Members  of  the  learning  groups  then 
process  how  effectively  they  have  been 
working  together. 

Informal  cooperative  learning  consists  of  having 
students  work  together  to  achieve  a  joint  learning  goal 
in  temporary,  ad  hoc  groups  that  last  from  a  few  min¬ 
utes  to  one  class  period  (Johnson  et  al.,  2002,  2006). 
Students  engage  in  quick  dialogs  or  activities  in  tem¬ 
porary,  ad  hoc  groups  in  response  to  a  limited  number 
of  questions  about  what  is  being  learned.  The  brief 
dialogs  or  activities  may  be  used  to  focus  student  atten¬ 
tion  on  the  material  to  be  learned,  set  a  mood  condu¬ 
cive  to  learning,  help  set  expectations  as  to  what  will 
be  covered  in  a  class  session,  ensure  that  students 
cognitively  process  the  material  being  taught,  and  pro¬ 
vide  closure  to  an  instructional  session.  Informal  coop¬ 
erative  learning  groups  are  often  organized  so  students 
engage  in  3-  to  5-minute  focused  discussions  before 
and  after  a  lecture  and  2-  to  3-minute  turn-to-your- 
partner  discussions  interspersed  every  10  to  15  min¬ 
utes  throughout  a  lecture. 

Cooperative  base  groups  are  long-term,  heteroge¬ 
neous  cooperative  learning  groups  with  stable  mem¬ 
bership  whose  primary  responsibilities  are  to  provide 
support,  encouragement,  and  assistance  to  make  aca¬ 
demic  progress  and  develop  cognitively  and  socially 
in  healthy  ways  as  well  as  holding  each  other  account¬ 
able  for  striving  to  learn  (Johnson  et  al.,  2002,  2006). 
Typically,  cooperative  base  groups  (1)  are  heteroge¬ 
neous  in  membership,  (2)  meet  regularly  (for  example, 
daily  or  biweekly),  and  (3)  last  for  the  duration  of  the 


semester  or  year  or  until  all  members  are  graduated. 
Students  are  assigned  to  base  groups  of  three  to  four 
members  that  meet  at  the  beginning  and  end  of  each 
class  session  (or  week)  to  complete  academic  tasks 
such  as  checking  each  members’  homework,  routine 
tasks  such  as  taking  attendance,  and  personal  support 
tasks  such  as  listening  sympathetically  to  personal 
problems  or  providing  guidance  for  writing  a  paper. 

These  three  types  of  cooperative  learning  may  be 
used  together.  A  typical  class  session  may  begin  with 
a  base  group  meeting,  which  is  followed  by  a  short 
lecture  in  which  informal  cooperative  learning  is  used. 
The  lecture  is  followed  by  a  formal  cooperative  learn¬ 
ing  lesson.  Near  the  end  of  the  class  session  another 
short  lecture  may  be  delivered  with  the  use  of  informal 
cooperative  learning.  The  class  ends  with  a  base  group 
meeting. 

Constructive  controversy  exists  when  one  student’s 
ideas,  information,  conclusions,  theories,  and  opinions 
are  incompatible  with  those  of  another,  and  the  two 
seek  to  reach  an  agreement  (Johnson  and  Johnson, 
1979, 1995, 2003, 2006).  Teachers  structure  academic 
controversies  by  choosing  an  academic  issue,  assign¬ 
ing  students  to  groups  of  four,  dividing  the  group  into 
two  pairs,  and  assigning  one  pair  the  pro  position  and 
the  other  pair  a  con  position.  Students  then  follow  the 
five-step  controversy  procedure:  (1)  prepare  the  best 
case  possible  for  their  assigned  position,  (2)  persua¬ 
sively  present  the  best  case  possible  for  their  position 
to  the  opposing  pair,  (3)  have  an  open  discussion  in 
which  the  two  sides  argue  forcefully  and  persuasively 
for  their  position  while  subjecting  the  opposing  posi¬ 
tion  to  critical  analysis,  (4)  reverse  perspectives,  and 
(5)  drop  all  advocacy  and  come  to  a  consensus  as  to 
their  best  reasoned  judgment  about  the  issue.  Contro¬ 
versies  tend  to  be  constructive  when  the  situational 
context  is  cooperative,  group  members  are  heteroge¬ 
neous,  information  and  expertise  are  distributed  within 
the  group,  members  have  the  necessary  conflict  skills, 
and  the  canons  of  rational  argumentation  are  followed. 

THEORETICAL  FOUNDATIONS 
OF  COOPERATIVE  LEARNING 

At  least  three  general  theoretical  perspectives  have 
guided  research  on  cooperative  learning:  (1)  cogni¬ 
tive-developmental,  (2)  behavioral,  and  (3)  social  inter¬ 
dependence.  The  cognitive-developmental  or  construc¬ 
tivist  perspective  is  largely  based  on  the  theories  of 
Piaget(1950)  and  Vygotsky  (1978).  The  work  of  Piaget 
and  related  theorists  is  based  on  the  premise  that  when 
individuals  cooperate  on  the  environment,  sociocogni- 
tive  conflict  occurs  that  creates  cognitive  disequilibrium. 
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which  in  turn  stimulates  perspective-taking  ability  and 
cognitive  development.  The  work  of  Vygotsky  and 
related  theorists  is  based  on  the  premise  that  knowl¬ 
edge  is  social,  constructed  from  cooperative  efforts  to 
learn,  understand,  and  solve  problems.  The  behavioral 
learning  theory  perspective  focuses  on  the  impact  of 
group  reinforcers  and  rewards  on  learning.  Skinner 
(1968)  focused  on  group  contingencies,  Bandura 
(1977)  focused  on  imitation  and  joint  efficacy,  and 
JJomans  (1961)  as  well  as  Kelley  and  Thibaut  (1978) 
focused  on  the  balance  of  rewards  and  costs  in  social 
exchange  among  interdependent  individuals.  Although 
the  cognitive-developmental  and  behavioral  theoreti¬ 
cal  orientations  have  their  followers,  by  far  the  theory 
dealing  with  cooperation  that  has  generated  the  most 
research  is  social  interdependence  theory. 

Social  Interdependence  Theory 

Social  interdependence  exists  when  the  accomplish¬ 
ment  of  each  individual’s  goals  is  affected  by  the 
actions  of  others  (Deutsch,  1949, 1962;  Johnson,  1970, 
2003;  Johnson  and  Johnson,  1989,  2005).  There  are 
two  types  of  social  interdependence:  positive  (cooper¬ 
ation)  and  negative  (competition).  Positive  interdepen¬ 
dence  exists  when  individuals  perceive  that  they  can 
reach  their  goals  if  and  only  if  the  other  individuals 
with  whom  they  are  cooperatively  linked  also  reach 
their  goals.  Participants,  therefore,  promote  each 
other’s  efforts  to  achieve  the  goals.  Negative  interde¬ 
pendence  exists  when  individuals  perceive  that  they 
can  obtain  their  goals  if  and  only  if  the  other  individ¬ 
uals  with  whom  they  are  competitively  linked  fail  to 
obtain  their  goals .  Participants ,  therefore ,  obstruct  each 
other’s  efforts  to  achieve  the  goals.  No  interdepen¬ 
dence  results  in  a  situation  in  which  individuals  per¬ 
ceive  that  they  can  reach  their  goal  regardless  of 
whether  other  individuals  in  the  situation  attain  or  do 
not  attain  their  goals. 

Each  type  of  interdependence  results  in  certain 
psychological  processes.  Positive  interdependence 
tends  to  result  in  substitutability  (i.e.,  the  degree  to 
which  the  actions  of  one  person  substitute  for  the 
actions  of  another  person),  inducibility  (i.e.,  openness 
to  being  influenced  and  to  influencing  others),  and 
positive  cathexis  (i.e.,  investment  of  positive  psycho¬ 
logical  energy  in  objects  outside  of  oneself)  (Deutsch, 
1949,  1962;  Johnson,  2003;  Johnson  and  Johnson, 
1989,  2005).  Negative  interdependence  tends  to  result 
in  nonsubstitutability,  resistance  to  being  influenced 
by  others,  and  negative  cathexis.  No  interdependence 
detaches  a  person  from  others,  thereby  creating  non¬ 
substitutability,  no  inducibility  or  resistance,  and 
cathexis  only  to  one’s  own  actions. 


The  basic  premise  of  social  interdependence  theory 
is  that  the  way  in  which  goals  are  structured  determines 
how  participants  interact,  and  those  interaction  pat¬ 
terns  determine  the  outcomes  of  the  situation.  Positive 
interdependence  results  in  promotive  interaction,  neg¬ 
ative  interdependence  results  in  oppositional  or  con- 
trie  nt  interaction,  and  no  interdependence  results  in  the 
absence  of  interaction.  Promotive  interaction  may  be 
defined  as  individuals  encouraging  and  facilitating 
each  other’s  efforts  to  complete  tasks,  achieve,  or  pro¬ 
duce  to  reach  the  group’s  goals.  It  consists  of  a  number 
of  variables,  including  mutual  help  and  assistance, 
exchange  of  needed  resources,  effective  communica¬ 
tion,  mutual  influence,  trust,  and  constructive  manage¬ 
ment  of  conflict.  Oppositional  interaction  may  be 
defined  as  individuals  discouraging  and  obstructing 
each  other’s  efforts  to  complete  tasks,  achieve,  or  pro¬ 
duce  to  reach  their  goals;  individuals  focus  both  on 
increasing  their  own  productivity  and  on  preventing 
any  other  person  from  producing  more  than  they  do. 
It  consists  of  such  variables  as  obstruction  of  each 
other’s  goal  achievement  efforts,  tactics  of  threat  and 
coercion,  ineffective  and  misleading  communication, 
distrust,  and  striving  to  win  in  conflicts.  No  interaction 
may  be  defined  as  individuals  acting  independently 
without  any  interchange  with  each  other  while  they 
work  to  achieve  their  goals;  individuals  focus  only  on 
increasing  their  own  productivity  and  achievement  and 
ignore  as  irrelevant  the  efforts  of  others. 

The  Basic  Elements  of  Cooperation 

Potential  Group  Performance 

Not  all  groups  are  cooperative  (Johnson  and  Johnson, 
2006).  Placing  people  in  the  same  room,  seating  them 
together,  or  telling  them  that  they  are  a  group  does  not 
mean  they  will  cooperate  effectively.  To  be  coopera¬ 
tive,  to  reach  the  full  potential  of  the  group,  five  essen¬ 
tial  elements  must  be  carefully  structured  into  the  sit¬ 
uation:  positive  interdependence,  individual  and  group 
accountability,  promotive  interaction,  appropriate  use 
of  social  skills,  and  group  processing. 

Positive  Interdependence:  We  Instead  of  Me 

The  heart  of  cooperation  is  positive  interdependence. 
There  are  three  major  categories  of  interdependence: 
outcome,  means,  and  boundary  interdependence 
(Johnson,  2003;  Johnson  and  Johnson,  2005).  When 
persons  are  in  a  cooperative  or  competitive  situation, 
they  are  oriented  toward  a  desired  outcome,  end  state, 
goal,  or  reward.  If  there  is  no  outcome  interdependence 
(goal  and  reward  interdependence),  there  is  no  coop¬ 
eration  or  competition.  In  addition,  the  means  through 
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which  the  mutual  goals  or  rewards  are  to  be  accom¬ 
plished  specify  the  actions  required  on  the  part  of 
group  members.  Means  interdependence  includes 
resource,  role,  and  task  interdependence  (which  are 
overlapping  and  not  independent  from  each  other). 
Finally,  boundaries  (i.e.,  discontinuities  existing 
among  individuals  and  groups)  can  define  who  is  inter¬ 
dependent  with  whom  (Koffka,  1935;  Wertheimer, 
1923).  Discontinuity  may  be  created  by  environmental 
factors  (different  parts  of  the  room  or  different  rooms), 
similarity  (all  wearing  the  same  color  shirt  or  jacket), 
proximity  (seated  together),  past  history  together, 
expectations  of  being  grouped  together,  and  differen¬ 
tiation  from  other  competing  groups.  Boundary  inter¬ 
dependence  thus  includes  outside  enemy  (i.e.,  negative 
interdependence  with  another  group),  identity  (which 
binds  them  together  as  an  entity),  and  environmental 
(such  as  a  specific  work  area)  interdependence  (which 
are  overlapping  and  not  independent  from  each  other) . 

The  authors  have  conducted  a  series  of  studies 
investigating  the  nature  of  positive  interdependence 
and  the  relative  power  of  the  different  types  of  positive 
interdependence  (Johnson,  2003;  Johnson  and 
Johnson,  2005).  Our  research  indicates  that  positive 
interdependence  provides  the  context  within  which 
promotive  interaction  takes  place,  that  group  member¬ 
ship  and  interpersonal  interaction  among  students  do 
not  produce  higher  achievement  unless  positive  inter¬ 
dependence  is  clearly  structured,  that  the  combination 
of  goal  and  reward  interdependence  increases  achieve¬ 
ment  over  goal  interdependence  alone,  and  that 
resource  interdependence  does  not  increase  achieve¬ 
ment  unless  goal  interdependence  is  present  also. 

Individual  Accountability/Personal  Responsibility 

Individual  accountability  exists  when  the  performance 
of  each  individual  member  is  assessed,  the  results  are 
given  back  to  the  individual  and  the  group  to  compare 
against  a  standard  of  performance,  and  the  member  is 
held  responsible  by  groupmates  for  contributing  his  or 
her  fair  share  to  the  group’s  success  (Johnson,  2003; 
Johnson  and  Johnson,  2005).  Individual  accountability 
tends  to  increase  achievement  in  cooperative  learning, 
including  computer-supported  cooperative  learning 
(Hooper  et  al.,  1989;  Johnson,  2003;  Johnson  and 
Johnson,  1989,  2005). 

Promotive  Interaction 

The  greater  the  promotive  interaction,  the  stronger  the 
effects  of  cooperation  (Johnson,  2003;  Johnson  and 
Johnson,  2005).  Technology  enables  individuals  to 
promote  each  other  success  all  across  the  world,  but 


it  does  not  always  substitute  for  face-to-face  interac¬ 
tion.  Face-to-face  communication  has  a  richness  that 
electronic  communication  may  never  match  (Prusak 
and  Cohen,  2001).  Evidence  suggests  that  up  to  93% 
of  people’s  intent  is  conveyed  by  facial  expression  and 
tone  of  voice,  with  the  most  important  channel  being 
facial  expression  (Druckman  et  al.,  1982;  Mehrabian, 
1971).  Harold  Geneen,  the  former  head  of  ITT, 
believed  that  his  response  to  requests  was  different  in 
face-to-face  encounters  than  through  electronic  means, 
stating  that  it  is  easy  to  say  “no”  to  an  electronic 
request  but  face  to  face  the  answer  may  be  “yes” 
because  of  the  nonverbal  cues  attached  to  the  request. 
He,  therefore,  made  it  company  policy  for  problems 
to  be  solved  face  to  face  (cited  in  Trevino  et  al.,  1987). 
Office  spaces  are  increasingly  built  to  maximize  face- 
to-face  interaction.  The  biggest  complaint  of  students 
in  a  virtual  high  school  was  that  interactions  with 
online  students  just  did  not  measure  up  to  face-to-face 
context  (Allen,  2001).  On  the  other  hand,  Bonk  and 
King  (1998)  suggested  that  promotive  interaction  in 
electronic  environments  has  some  advantages  over  live 
discussion  in  terms  of  engagement  in  learning,  depth 
of  discussion,  time  on  task,  and  the  promotion  of 
higher-order  thinking  skills.  Instructional  programs, 
therefore,  may  be  most  effective  when  they  include 
multiple  ways  for  students  to  promote  each  other’s 
success,  both  electronically  and  face  to  face  whenever 
possible. 

Interpersonal  and  Small  Group  Skills 

Cooperative  learning  is  inherently  more  complex  than 
competitive  or  individualistic  learning  because  stu¬ 
dents  have  to  simultaneously  engage  in  taskwork  and 
teamwork.  The  greater  the  members’  teamwork  skills, 
the  higher  will  be  the  quality  and  quantity  of  their 
learning  (Johnson,  1991,  2003,  2006;  Johnson  and 
Johnson,  2005,  2006).  The  combination  of  positive 
interdependence,  an  academic  contingency  for  high 
performance  by  all  group  members,  and  a  social  skills 
contingency  promoted  tends  to  promote  high  levels  of 
achievement  and  positive  relationships  among  stu¬ 
dents. 

Group  Processing 

Group  processing  occurs  when  members  discuss  how 
well  they  are  achieving  their  goals  and  maintaining 
effective  working  relationships  among  members 
(Johnson  and  Johnson,  1989,  2005).  The  purposes  of 
group  processing  are  to  clarify  and  improve  the  effec¬ 
tiveness  of  members  in  contributing  to  the  cooperative 
efforts  to  achieve  the  group’s  goals  by:  (1)  enabling 
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Figure  33.1  Outcomes  of  cooperation.  (From  Johnson,  D.W.  and  Johnson,  R.,  Cooperation  and  Competition:  Theory  and  Research, 
Interaction  Book  Company,  Edina,  MN,  1989.  With  permission.) 


groups  to  improve  continuously  the  quality  of  mem¬ 
ber’s  work,  (2)  facilitating  the  learning  of  teamwork 
skills,  (3)  ensuring  that  members  receive  feedback  on 
their  participation,  and  (4)  enabling  groups  to  focus  on 
group  maintenance.  Groups  that  process  how  effec¬ 
tively  members  are  working  together  tend  to  achieve 
higher  than  do  groups  that  do  not  process  or  individ¬ 
uals  working  alone. 


RESEARCH  ON 
COMPUTER-SUPPORTED 
COOPERATIVE  UEARNING 

Promotive,  oppositional,  and  no  interaction  have  dif¬ 
ferential  effects  on  the  outcomes  of  the  situation 
(Johnson  and  Johnson,  1989,  2005).  The  research  has 
focused  on  numerous  outcomes,  which  may  be  sub¬ 
sumed  within  the  broad  and  interrelated  categories  of 
effort  to  achieve,  quality  of  relationships,  and  psycho¬ 
logical  health  (Johnson  and  Johnson,  1989)  (see  Figure 
33.1).  Overall,  the  evidence  is  very  strong  that  coop¬ 
eration  (compared  with  competitive  and  individualistic 
efforts)  promotes  (Johnson  and  Johnson,  1989,  2005): 

•  Greater  effort  exerted  to  achieve  (e.g.,  higher 
achievement  and  greater  productivity,  more 
frequent  use  of  higher-level  reasoning,  more 
frequent  generation  of  new  ideas  and  solu¬ 
tions,  greater  intrinsic  and  achievement 


motivation,  greater  long-term  retention, 
more  on-task  behavior,  and  greater  transfer 
of  what  is  learned  within  one  situation  to 
another) 

•  Higher  quality  of  relationships  among  par¬ 
ticipants  (e.g.,  greater  interpersonal  attrac¬ 
tion,  liking,  cohesion  or  esprit  de  corps, 
appreciation  for  heterogeneity,  and  task-ori¬ 
ented  and  personal  support) 

•  Greater  psychological  adjustment  (e.g., 
greater  psychological  health,  greater  social 
competencies,  higher  self-esteem,  a  shared 
identity,  and  greater  ability  to  cope  with 
stress  and  adversity) 

These  outcomes  have  been  discussed  extensively  else¬ 
where  (Johnson  and  Johnson,  1989,  2005).  This  chap¬ 
ter  focuses  on  the  outcomes  dealing  with  computer- 
supported  cooperative  learning. 

Computer-Supported  Cooperative 
Learning  and  Achievement 

Two  large  meta-analyses  concluded  that  the  use  of 
technology  markedly  improved  learning  outcomes 
(Fletcher-Flinn  and  Gravatt,  1995;  Khalili  and 
Shashaani,  1994),  but  no  differentiation  was  made 
among  teaching  practices  and  the  ways  technology  was 
implemented  in  the  classrooms.  We  conducted  several 
studies  examining  the  use  of  cooperative,  competitive. 
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and  individualistic  learning  activities  at  the  computer 
(Johnson  et  al„  1985,  1989,  1990,  1986a,b,c,  1987). 
The  studies  included  students  from  the  eighth  grade 
through  college  freshmen  and  lasted  from  3  to  30 
instructional  hours.  The  tasks  were  a  computerized 
navigational  and  map-reading  problem-solving  task 
and  word-processing  assignments.  Compared  with 
competitive  and  individualistic  efforts  at  the  computer, 
computer- supported  cooperative  learning  promoted: 
(1)  higher  quantity  of  daily  achievement,  (2)  higher 
quality  of  daily  achievement,  (3)  greater  mastery  of 
factual  information,  (4)  greater  ability  to  apply  one’s 
factual  knowledge  in  test  questions  requiring  applica¬ 
tion  of  facts,  (5)  greater  ability  to  use  factual  informa¬ 
tion  to  answer  problem-solving  questions,  and  (6) 
greater  success  in  problem  solving.  Cooperation  at  the 
computer  promoted  greater  motivation  to  persist  on 
problem-solving  tasks.  Students  in  the  cooperative 
condition  were  more  successful  in  operating  computer 
programs.  In  terms  of  oral  participation,  students  in 
the  cooperative  condition,  compared  with  students  in 
the  competitive  and  individualistic  conditions,  made 
fewer  statements  to  the  teacher  and  more  to  each  other, 
made  more  task-oriented  statements  and  fewer  social 
statements,  and  generally  engaged  in  more  positive, 
task-oriented  interaction  with  each  other  (especially 
when  the  social  skill  responsibilities  were  specified 
and  group  processing  was  conducted).  Finally,  the 
studies  provided  evidence  that  females  were  perceived 
to  be  of  higher  status  in  the  cooperative  than  in  the 
competitive  or  individualistic  conditions. 

In  addition  to  our  work,  a  number  of  studies  have 
found  that  students  using  a  combination  of  cooperative 
learning  and  computer-based  instruction  learn  better 
than  do  students  using  computer-based  instruction 
while  working  individualistic  ally  (Anderson  et  al., 
1995;  Cockayne,  1991;  Cox  and  Berger,  1985;  Dalton, 
1990a,b;  Dalton  et  al.,  1987;  Dees,  1991;  Fletcher, 
1985;  Hooper,  1992a,b;  Hooper  et  al.,  1993;  Hythecker 
et  al.,  1985;  Inkpen  et  al.,  1995;  King,  1989;  Lin  et 
al.,  1999;  Love,  1969;  Mclnerney  et  al.,  1997; 
Mevarech,  1993;  Mevarech  et  al.,  1991,  1987;  Okey 
and  Majer,  1976;  Postthast,  1995;  Regin,  1990;  Rep- 
man,  1993;  Rocklin  et  al.,  1985;  Shlechter,  1990; 
Stephenson,  1992;  Underwood  and  McCaffrey,  1990; 
Webb,  1984;  Weinberger  et  al.,  2005;  Whitelock  et  al., 
1995;  Yeuh  and  Alessi,  1988).  In  contrast,  a  number 
of  studies  have  found  no  statistically  significant  differ¬ 
ences  in  achievement  between  subjects  who  worked  in 
groups  and  subjects  who  worked  alone  (Carrier  and 
Sales,  1987;  Cosen  and  English,  1987;  Hooper  and 
Hannafin,  1988;  Trowbridge  and  Durnin,  1984).  No 
study  has  reported  significantly  greater  learning  when 
students  worked  alone.  Many  of  these  studies,  how¬ 


ever,  are  short-term  experiments  focused  on  a  small 
number  of  students.  Several  experiments  provide  evi¬ 
dence  that  the  well-known  CSCL  programs  such  as 
CSILE  and  Belvedere  have  proved  to  be  helpful  for 
higher-order  social  interaction  and,  subsequently,  for 
better  learning  in  terms  of  deep  understanding  (Scar- 
damalia  et  al.,  1994;  Suthers,  1998).  What  is  still  lack¬ 
ing  is  evidence  that  the  same  results  could  be  found 
in  normal  classrooms.  Some  CSCL  projects,  such  as 
CoVis,  have  been  widely  implemented  (Pea  et  al., 
1994),  but  few  well-controlled  follow-up  evaluations 
have  been  published. 

Hooper  and  his  colleagues  have  conducted  a  series 
of  studies  on  computer-supported  cooperative  learning 
involving  fifth,  sixth,  seventh,  and  eighth  graders  and 
college  students  (Dyer,  1993;  Hooper,  1992a,b;  Hooper 
et  al.,  1988,  1991,  1989;  Huang,  1993;  McDonald, 
1993).  Their  findings  included  the  following: 

•  Cooperative  group  members  achieved  sig¬ 
nificantly  higher  than  did  students  working 
under  individualistic  conditions. 

•  Cooperative  learning  groups  in  which  indi¬ 
vidual  accountability  was  carefully  struc¬ 
tured  achieved  higher  than  did  cooperative 
learning  groups  in  which  no  individual 
accountability  was  structured. 

•  The  achievement  of  low-ability  students  in 
heterogeneous  cooperative  groups  was  con¬ 
sistently  higher  than  the  achievement  of  low- 
ability  students  in  homogeneous  groups. 

•  A  positive  and  significant  correlation  was 
found  between  achievement  and  helping 
behaviors,  and  increases  in  achievement  and 
cooperation  were  significantly  related  within 
heterogeneous  groups. 

•  Cooperative  (compared  with  individualistic) 
learning  resulted  in  greater  willingness  to 
learn  the  material,  options  selection,  time  on 
task,  perceived  interdependence,  and  sup¬ 
portiveness  for  partners. 

Carlson  and  Falk  (1989)  and  Noell  and  Camine 
(1989)  found  that  students  in  cooperative  groups  per¬ 
formed  higher  than  students  working  alone  on  learning 
tasks  involving  interactive  videodiscs.  Adams  et  al. 
(1990)  suggested  that  cooperative  learning  can  influ¬ 
ence  attention,  motivation,  and  achievement  when  stu¬ 
dents  use  the  medium  of  television.  Finally,  Ocker  and 
Yaverbaum  (1999)  found  that  asynchronous  coopera¬ 
tion  is  as  effective  as  face-to-face  cooperation  in  terms 
of  learning,  quality  of  solution,  solution  content,  and 
satisfaction  with  the  solution  quality.  Online  learners, 
however,  were  significantly  less  satisfied  with  the 
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asynchronous  learning  experience,  both  in  terms  of  the 
group  interaction  process  and  the  quality  of  group 
discussions.  Hiltz  (1998)  argued  that  an  online  learn¬ 
ing  community  structured  cooperatively  is  more  effec¬ 
tive  than  working  individually. 

Learning  How  to  Use  Technology 

In  addition  to  academic  achievement,  cooperative 
learning  may  reduce  hardware  and  software  problems 
that  interfere  with  achievement  when  students  work 
alone  (Hativa,  1988).  Students  naturally  form  groups 
when  learning  how  to  use  a  new  technology  or  software 
program  (Becker,  1984).  Generally,  this  evidence  indi¬ 
cates  that  students  will  learn  how  to  use  hardware  and 
software  more  quickly  and  effectively  when  they  learn 
in  cooperative  groups  rather  than  alone,  especially 
when  it  involves  new  and  complex  procedures  (Dwyer, 
1994;  Dyer,  1993;  Hooper,  1992b;  Hooper  et  al.,  1993; 
Keeler  and  Anson,  1995;  McDonald,  1993;  Trowbridge 
and  Durnin,  1984;  Webb,  1984;  Webb  et  al.,  1986). 
When  teachers  wish  to  introduce  new  technology  and 
new  software  programs  of  some  complexity,  they 
would  be  well-advised  to  use  cooperative  learning. 

Learner  Control 

Combining  cooperative  learning  and  computer-sup- 
ported  instruction  results  in  students  having  more  con¬ 
trol  over  their  learning.  Hooper  and  his  associates 
(Hooper,  1992b;  Hooper  et  al.,  1993)  noted  that  three 
forms  of  lesson  control  are  used  in  the  design  of  tech¬ 
nology-based  instruction:  learner,  program,  and  adap¬ 
tive  control.  Learner  control  involves  delegating 
instructional  decisions  to  learners  so  they  can  deter¬ 
mine  what  help  they  need,  what  difficulty  level  or 
content  density  of  material  they  wish  to  study,  in  what 
sequence  they  wish  to  learn  material,  and  how  much 
they  want  to  learn.  Learner-controlled  environments 
include  simulations,  hypermedia,  and  online  data¬ 
bases.  Program  or  linear  control  prescribes  an  identi¬ 
cal  instructional  sequence  for  all  students  regardless 
of  interest  or  need.  Adaptive  control  modifies  lesson 
features  according  to  student  aptitude  (Snow,  1980), 
prior  performance  (Tobias,  1987),  or  ongoing  lesson 
needs  (Tennyson  et  al.,  1984).  Linear  or  program  con¬ 
trol  may  impose  an  inappropriate  lesson  sequence  on 
learners  and  thereby  lower  their  motivation,  and  adap¬ 
tive  instruction  may  foster  learner  dependence  (Hanna- 
fin  and  Rieber,  1989).  As  learner  control  increases  so 
does  instructional  effectiveness  and  efficiency  (Reige- 
luth  and  Stein,  1983),  as  well  as  learner  independence, 
efficiency,  mental  effort,  and  motivation  (Federico, 
1980;  Salomon,  1983,  1985;  Steinberg,  1984). 


Computer-supported  cooperative  learning  tends 
to  increase  the  effectiveness  of  learner  control.  When 
students  work  alone,  in  isolation  from  their  peers, 
they  tend  not  to  control  the  learning  situation  pro¬ 
ductively,  making  ineffective  instructional  decisions 
and  leaving  instruction  prematurely  (Carrier,  1984; 
Hannafin,  1984;  Milheim  and  Martin,  1991;  Stein¬ 
berg,  1977,  1989).  Students  working  cooperatively 
tend  to  motivate  each  other  to  seek  elaborative  feed¬ 
back  for  their  responses  to  practice  items  during 
learning  control  and  to  seek  a  greater  variety  of  feed¬ 
back  types  more  frequently  than  did  those  working 
alone  (Carrier  and  Sales,  1987).  Cooperative  pairs 
spent  longer  times  inspecting  information  on  the 
computer  screen  as  they  discussed  which  level  of 
feedback  they  needed  and  what  the  answers  were  to 
practice  items.  Students  in  the  learner-controlled 
cooperative  learning  condition  selected  more  options 
during  the  lesson  and  spent  more  time  interacting 
with  the  tutorial  than  did  students  in  the  learner- 
controlled  individual  learning  condition  (McDonald, 
1993).  Hooper  et  al.  (1993)  found  that  students  in  the 
program-control  conditions  attempted  more  than  four 
times  as  many  examples  and  nearly  twice  as  many 
practice  questions  as  did  the  students  in  the  learner- 
control  conditions.  The  LOGO  computer  environ¬ 
ment  tends  to  promote  more  actual  learner  control 
over  the  task  structure  and  the  making  of  rules  to 
govern  it  than  does  the  CAI  computer  environment 
(Battista  and  Clements,  1986;  Clements  and  Nastasi, 
1985,  1988;  Nastasi  et  al.,  1990).  Learner  control 
seems  to  be  most  effective  when  prior  knowledge  is 
high  or  when  students  possess  well-developed  meta- 
cognitive  abilities  (Garhart  and  Hannafin,  1986). 
What  these  studies  imply  is  that  cooperative  learning 
is  an  important  variable  in  improving  the  effective¬ 
ness  of  learner-controlled  environments. 

Computer-Supported  Cooperative 
Learning  and  Relationships 

The  impact  of  cooperative  efforts  on  relationships  is 
as  powerful  for  online  relationships  as  it  is  for  face- 
to-face  relationships .  Individuals  who  are  widely  sep¬ 
arated  geographically  may  develop  positive  relation¬ 
ships  through  cooperative  efforts  conducted  through 
e-mail,  chat  rooms,  bulletin  boards,  and  conferences 
(Simon,  2003).  Communication  can  be  asynchronous 
and  extremely  fast  in  comparison  with  telephone  con¬ 
versations  and  interoffice  mail.  Participation  may  be 
more  equalized  and  less  affected  by  prestige  and  sta¬ 
tus  (McGuire  et  al.,  1987;  Siegel  et  al.,  1986).  The 
egalitarian  network  structures  may  coexist  with  sub¬ 
stantial  hierarchy  and  centralization  in  patterns  of 
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communication  (Ahuja  and  Carley,  1998).  Digital  con¬ 
ferencing,  however,  may  make  employees  less  risk 
averse  and  render  group  decision  making  less  predict¬ 
able,  more  time  consuming,  and  more  egalitarian 
(Sproull  and  Kiesler,  1991;  Wellman  et  al.,  1996), 
depending  on  the  specific  ways  the  technological  sys¬ 
tems  are  designed  and  implemented  (O’Mahony  and 
Barley,  1999;  Sproull  and  Kiesler,  1991). 

Significant  differences  may  be  found  in  relation¬ 
ships  built  electronically  and  those  built  face  to  face. 
In  electronic  communication,  it  is  easy  for  a  sender  to 
be  out  of  touch  with  the  receiver  and  to  be  less  con¬ 
strained  by  conventional  norms  and  rules  for  behavior 
in  composing  messages.  Communicators  can  feel  a 
greater  sense  of  anonymity,  detect  less  individuality  in 
others,  feel  less  empathy,  feel  less  guilt,  be  less  con¬ 
cerned  over  how  they  compare  with  others,  and  be  less 
influenced  by  social  conventions  (Kiesler  et  al.,  1984; 
Short  et  al.,  1976).  Such  influences  can  lead  both  to 
more  honesty  and  more  flaming  (name  calling  and 
epithets). 

Social  Presence 

Building  relationships  electronically  may  depend  on 
the  ability  of  individuals  to  create  social  presence. 
Social  presence  may  be  defined  as  the  ability  to  project 
one’s  self  and  establish  personal  and  purposeful  rela¬ 
tionships  or  the  sense  of  being  with  another  (Biocca, 

2003) .  To  reduce  the  drawbacks  of  distance  learning, 
such  as  social  isolation  and  decreased  motivation, 
online  individuals  engaging  in  cooperative  efforts  need 
to  develop  a  social  presence  to  interact  effectively.  Just 
because  a  technology  will  allow  participants  to  interact 
with  each  other  does  not  mean  that  participants  will 
(Kreijns,  2004).  The  instructional  strategies  used  must 
require  a  consistent  level  of  interaction  and  sustain  the 
interaction  in  online  learning  environments  (Muirhead, 

2004)  .  The  lack  of  social  presence  can  negatively  affect 
the  quality  of  interaction  and  cooperation  among  stu¬ 
dents  (Kreijns  et  al.,  2003;  Rourke,  2000).  Social  pres¬ 
ence  is  essential  for  online  cooperation,  as  it  encour¬ 
ages  constructive  discourse  and  builds  a  climate  for 
exchanging  ideas  (Rourke  et  al.,  1999)  and  impacts 
online  learning  (Tu  and  Corry,  2002;  Tu  and  Me  Isaac, 
2002).  Social  presence  is  also  a  predictor  of  online 
learners’  satisfaction  with  their  learning  (Gunawar- 
dena  and  Zittle,  1997)  and  is  crucial  for  building  an 
online  learning  community  (Hiltz,  1998).  Ways  to  pro¬ 
mote  social  presence  include:  (1)  social  interaction,  (2) 
self-disclosure,  (3)  graphic  representations  of  facial 
expressions,  (4)  ice  breakers  or  discussion  starters,  and 
(5)  development  of  group  identity  (e.g.,  group  name 
or  logo) . 


Computer-Supported  Cooperative 
Learning  and  Social  Competencies 

Numerous  studies  on  computer-supported  cooperative 
learning  have  demonstrated  positive  effects  on  the 
amount  and  quality  of  social  interaction  (Amigues  and 
Agostinelli,  1992;  Crook,  1994;  Davis  and  Hutten- 
locher,  1995;  Fishman  and  Gomez,  1997;  McConnell, 
1994;  Rysavy  and  Sales,  1991).  When  teamwork  pro¬ 
cedures  and  skills  are  present,  cooperative  learning 
tends  to  result  in  higher  achievement  in  computer-sup- 
ported  instructional  lessons  than  individualistic  learn¬ 
ing  (Hooper  and  Hannafin,  1988,  1991;  Johnson  et  al., 
1985,  1986c;  Susman,  1998).  In  studies  where  team¬ 
work  procedures  and  skills  were  not  emphasized,  reli¬ 
able  differences  in  achievement  in  cooperative  and  indi¬ 
vidualistic  technology  supported  instruction  tended  not 
to  be  found  (McCaffrey,  1990;  Mevarech  et  al.,  1987; 
Susman,  1998;  Underwood  and  Hooper  et  al.,  1989). 

Software  designers  may  be  able  to  facilitate  the 
development  of  the  interpersonal  and  small  group 
skills  required  for  teamwork  by  requiring  initial  tuto¬ 
rial  activities  on  cooperative  skills,  suggesting  roles 
for  group  members  (keyboarder,  recorder,  checker  for 
understanding,  encourager  of  participation) ,  providing 
students  time  for  group  processing  to  analyze  and  dis¬ 
cuss  how  effectively  they  are  working  together  and 
how  they  may  work  more  effectively  together  in  the 
future,  giving  reminders  for  students  to  monitor  their 
own  performance  and  to  assist  in  optimizing  group 
performance,  and  providing  tangible  prizes  for  indi¬ 
vidual  successes  and  group  achievement  may  increase 
motivation  to  succeed. 

Attitudes  toward 
Technology-Based  Instruction 

Students  tend  to  develop  more  positive  attitudes  toward 
the  computer-based  instructional  lesson  and  learning 
with  a  computer  when  they  work  in  cooperative  learn¬ 
ing  groups  than  when  they  work  individually  (Hooper 
et  al.,  1993;  Huang,  1993;  McDonald,  1993;  Sutton, 
1996);  they  also  tended  to  experience  more  enjoyment 
using  the  computer  to  engage  in  cooperative  activities. 

Attitudes  toward  Cooperative  Learning 

Students  involved  in  computer-based  cooperative 
learning  tend  to  have  more  positive  towards  cooper¬ 
ative  learning  and  how  it  affects  them  personally  than 
students  who  work  individually  (Hooper  et  al.,  1993; 
Mevarech  et  al.,  1987;  Rocklin  et  al.,  1985).  Students 
participating  in  structured  cooperative  learning  tend 
to  develop  more  positive  attitudes  toward  working 


411 


David  W.  Johnson  and  Roger  T.  Johnson 


cooperatively  than  students  participating  in  unstruc¬ 
tured  cooperative  learning  or  learning  individualisti- 
cally  (Dyer,  1993;  Hooper  et  al.,  1993;  Huang,  1993; 
McDonald,  1993). 

Preference  for  Using 
Technology  Cooperatively 

A  natural  partnership  exists  between  technology  and 
cooperation.  The  introduction  of  computers  into  class¬ 
rooms  tends  to  increase  cooperative  behavior  and  task- 
oriented  verbal  interaction  (Chemick  and  White,  1981 , 
1983;  Hawkins  et  al.,  1982;  Levin  and  Kareev,  1980; 
Rubin,  1983;  Webb,  1984).  Individuals  tend  to  prefer 
to  work  cooperatively  at  the  computer  (Hawkins  et  al., 
1982;  Levin  and  Kareev,  1980;  Muller  and  Perlmutter, 
1985).  Students  are  more  likely  to  seek  each  other  out 
at  the  computer  than  they  normally  would  for  other 
school  work.  Even  when  students  play  electronic 
games,  they  prefer  to  have  partners  and  associates. 
Working  at  a  computer  cooperatively  with  classmates 
seems  to  be  more  fun  and  enjoyable  as  well  as  more 
effective  to  most  students. 

Individual  Differences 

Ability  and  Group  Composition 

A  subject  of  considerable  disagreement  is  whether  the 
success  of  computer-supported  cooperative  learning  is 
affected  by  the  homogeneity  or  heterogeneity  of  the 
group  composition.  Students  in  heterogeneous  ability 
groups  tend  to  learn  more  than  students  in  homogeneous 
ability  groups  (Yager  et  al.,  1985,  1986),  especially 
high-ability  students  (Beane  and  Lemke,  1971).  The 
academic  discussion  and  peer  interaction  in  heteroge¬ 
neous  (as  compared  with  homogeneous)  groups  pro¬ 
motes  the  discovery  of  more  effective  reasoning  strate¬ 
gies  (Bemdt  et  al.,  1988;  Johnson  and  Johnson,  1979). 

In  a  week-long  study  on  the  learning  of  LOGO, 
Webb  (1984)  investigated  whether  the  higher  ability 
students  in  cooperative  groups  of  three  would  try  to 
monopolize  the  computer.  She  found  that:  (1)  student 
ability  did  not  relate  to  contact  time  with  the  computer, 
and  (2)  student  success  in  programming  was  predicted 
by  different  profiles  of  abilities  and  by  group  process 
variables  such  as  verbal  interaction.  Yeuh  and  Alessi 
(1988)  used  group  ability  composition  as  one  of  their 
treatments  for  students  utilizing  the  computer  to  learn 
three  topics  in  algebra.  They  formed  groups  of 
medium- ability  students  and  groups  of  mixed-ability 
students  and  found  that  group  composition  had  no 
significant  effect  on  achievement.  Hooper  and  Hanna- 
fin  (1988),  in  a  study  with  40  eighth  grade  students, 
found  that  on  a  computer  task  low-ability  students 


working  with  high-ability  partners  achieved  higher 
than  did  low- ability  students  studying  in  homogeneous 
groups  or  alone,  without  lowering  the  achievement  of 
high-ability  students.  In  a  subsequent  study  involving 
125  sixth-  and  seventh-grade  students.  Hooper  and 
Hannafin  (1991)  randomly  assigned  students  to  homo¬ 
geneous  or  heterogeneous  pairs  and  pairs  to  coopera¬ 
tive  or  individualistic  conditions .  The  high-ability  stu¬ 
dents  interacted  equally  across  treatments,  but  low- 
ability  students  interacted  30%  more  when  placed  in 
heterogeneous  pairs.  Students  in  the  heterogeneous 
groups  achieved  and  cooperated  significantly  more 
than  did  students  in  the  homogeneous  pairs  (or  the 
individualistic  condition) . 

In  an  interactive  videodisc  science  lesson,  both 
high-  and  low-ability  students  spent  more  time  on  task 
and  performed  better  on  the  post-test  when  they 
learned  in  cooperative  groups  (as  well  as  developing 
more  positive  attitudes  toward  instruction,  teamwork, 
and  peers)  than  did  students  studying  alone  (Simsek 
and  Hooper,  1992),  and  low-achieving  students  in  het¬ 
erogeneous  cooperative  groups  tended  to  achieve 
higher  and  had  more  positive  attitudes  toward  the  expe¬ 
rience  than  did  their  counterparts  in  homogeneous 
cooperative  groups  (Simsek  and  Tsai,  1992). 

Hooper  (1992a)  compared  individual  and  cooper¬ 
ative  learning  in  an  investigation  of  the  effects  of  ability 
grouping  on  achievement,  instructional  efficacy,  and 
discourse  during  computer-based  mathematics  instruc¬ 
tion.  Students  completed  the  instruction  more  effec¬ 
tively  in  groups  than  alone.  In  groups,  achievement  and 
efficiency  were  highest  for  high-ability,  homoge¬ 
neously  grouped  students  and  lowest  for  average-abil¬ 
ity,  homogeneously  grouped  students.  Generating  and 
receiving  help  were  significant  predictors  of  achieve¬ 
ment,  and  average-ability  students  generated  and 
received  significantly  more  help  in  heterogeneous 
groups  than  in  homogeneous  ones.  Hooper  et  al.  (1993) 
compared  cooperative  and  individualistic  learning  on 
academically  high  and  average  or  low  performing  stu¬ 
dents.  They  found  that  the  students  in  the  cooperative 
conditions  performed  higher  on  a  computer-supported 
symbolic  reasoning  task  than  did  the  students  in  the 
individualistic  conditions.  Those  who  benefited  most 
from  the  group  learning  experience  appeared  to  be  the 
highest  performing  students.  Overall  achievement 
increased  by  almost  20%  for  high-ability  students,  but 
only  4%  for  average-  or  low-ability  students. 

The  results  of  these  studies  indicate  that  in  tech¬ 
nologically  supported  learning,  cooperative  learning 
may  be  used  effectively  with  both  homogeneous  and 
heterogeneous  groups,  but  that  the  greatest  educational 
benefits  may  be  derived  when  heterogeneous  groups 
work  with  technology-supported  instruction. 
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Gender 

The  gender  of  group  members  has  been  hypothesized 
to  be  an  important  factor  in  determining  the  success  of 
computer-supported  cooperative  learning.  Johnson  et 
al.  (1986b)  found  that  computer-supported  cooperative 
learning,  compared  with  competitive  and  individualis¬ 
tic  computer- supported  learning,  increased  the  posi¬ 
tiveness  of  female  students’  attitudes  toward  comput¬ 
ers,  equalized  the  status  and  respect  among  group 
members  regardless  of  gender,  and  resulted  in  a  more 
equal  participation  pattern  between  male  and  female 
members.  Whereas  females  in  cooperative  groups  liked 
working  with  the  computer  more  than  males  did,  there 
was  no  significant  difference  in  oral  interactions 
between  males  and  females.  Dalton  et  al.  (1989)  exam¬ 
ined  interactions  between  instructional  method  and 
gender  and  found  that  cooperative  learning  was  rated 
more  favorably  by  low-ability  females  than  by  low- 
ability  males.  Other  studies  noted  no  significant  differ¬ 
ences  in  performance  between  males  and  females  in 
computer-based  instruction  cooperative  learning  set¬ 
tings  (Mevarech  et  al.,  1987;  Webb,  1984).  Carrier  and 
Sales  (1987)  compared  female  pairs,  male  pairs,  and 
mixed  pairs  among  college  juniors  and  noted  that 
female  pairs  verbalized  the  most,  while  male  pairs  ver¬ 
balized  the  least,  and  male-females  pairs  demonstrated 
the  most  off-task  behavior.  Lee  ( 1993)  found  that  males 
tended  to  become  more  verbally  active  and  females 
tended  to  become  less  verbally  active  in  equal-ratio, 
mixed-gender  groups.  Underwood  and  McCaffrey 
(1990)  found  that  single-gender  pairs  completed  more 
work  with  greater  accuracy  than  did  mixed  gender  pairs. 

Evidence  concerning  the  impact  of  computer-sup- 
ported  instruction  on  males  and  females  is  mixed.  A 
conservative  interpretation  of  the  existing  research  is 
that  there  will  be  no  performance  differences  between 
males  and  females  on  computer-supported  cooperative 
learning,  but  females  will  have  more  positive  attitudes 
toward  using  technology  when  they  learn  in  coopera¬ 
tive  groups. 

Constructive  Controversy:  Argumentation 

Constructive  controversy  exists  when  group  members 
have  different  information,  perceptions,  opinions,  rea¬ 
soning  processes,  theories,  and  conclusions,  and  they 
must  reach  agreement  (Johnson  and  Johnson,  1979, 
1989,  1995,  2003).  Being  confronted  with  an  oppos¬ 
ing  point  of  view  promotes  uncertainty  about  the  cor¬ 
rectness  of  one’s  views,  an  active  search  for  more 
information,  a  reconceptualization  of  one’s  knowl¬ 
edge  and  conclusions,  and,  consequently,  greater  mas¬ 
tery  and  retention  of  the  material  being  studied  and 


a  more  reasoned  judgment  on  the  issue  being  consid¬ 
ered.  Individuals  working  alone  in  competitive  and 
individualistic  situations  do  not  have  the  opportunity 
to  experience  such  a  process;  therefore,  their  produc¬ 
tivity,  quality  of  decision  making,  and  achievement 
suffer  (Johnson  and  Johnson,  1979,  1995,  2003).  In 
addition,  compared  with  concurrence-seeking, 
debate,  and  individualistic  efforts,  controversy  results 
in  higher  quality  problem  solving,  greater  creativity 
in  thinking,  greater  motivation  to  learn  more  about 
the  topic,  more  productive  exchange  of  expertise 
among  group  members,  greater  task  involvement, 
more  positive  relationships  among  group  members, 
more  accurate  perspective  taking,  higher  self-esteem, 
and  greater  enjoyment  of  the  experience  (Johnson  and 
Johnson,  2003).  Controversies  tend  to  be  constructive 
when  the  situational  context  is  cooperative,  group 
members  are  heterogeneous,  information  and  exper¬ 
tise  are  distributed  within  the  group,  members  have 
the  necessary  conflict  skills,  and  the  canons  of  rational 
argumentation  are  followed. 

Clements  and  Nastasi  conducted  a  series  of  studies 
on  the  occurrence  of  cooperation  and  controversy  in 
computer-supported  instruction  (Battista  and  Clem¬ 
ents,  1986;  Clements  and  Nastasi,  1985, 1988;  Nastasi 
and  Clements,  1992;  Nastasi  et  al.,  1990).  They  found 
that  both  LOGO  and  CAI/CBI-W  computer  environ¬ 
ments  promoted  considerable  cooperative  work  and 
conflict  (both  social  and  cognitive).  The  LOGO  envi¬ 
ronment  (compared  to  CAI/CBI-W  computer  and  tra¬ 
ditional  classroom  tasks  environments)  promoted:  (1) 
greater  peer  interaction  focused  on  learning  and  prob¬ 
lem  solving,  (2)  self-directed  problem-solving  (i.e., 
learners  solving  problems  they  themselves  have  posed) 
with  mutual  ownership  of  the  problem,  (3)  more  fre¬ 
quent  occurrence  and  resolution  of  cognitive  conflicts, 
and  (4)  greater  development  of  executive-level  prob¬ 
lem-solving  skills  (planning,  monitoring,  decision 
making),  as  well  as  higher  levels  of  reasoning  and  cog¬ 
nitive  development.  The  development  of  higher  level 
cognitive  processes  seemed  to  be  facilitated  by  the  res¬ 
olution  of  cognitive  conflict  that  arises  out  of  cooper¬ 
ating.  They  also  found  that  the  LOGO  (compared  with 
the  CAI)  computer  environment  resulted  in  greater 
learner  satisfaction  and  expressions  of  pleasure  at  the 
discovery  of  new  information  and  in  their  work,  vari¬ 
ables  reflective  of  intrinsic  and  competence  motivation. 

More  recently,  Bell  (2001)  developed  a  program 
to  create  arguments  to  be  used  in  discussions  with 
other  students  (SenseMaker).  It  is  designed  to  support 
a  rhetorical  construction  of  arguments  by  individuals 
by  connecting  evidence  dots  with  claim  frames.  The 
intent  is  to  teach  students  the  nature  of  scientific 
inquiry  by  coordinating  emerging  evidence  with  an 
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Figure  33.2  Outcomes  of  technology-supported  cooperative  learning. 


existing  set  of  theories.  McAlister  et  al.  (2004)  devel¬ 
oped  a  tool  called  Academic  Talk  to  support  synchro¬ 
nous  arguing  among  peers.  Pilkington  and  Walker 
(2003)  assigned  online  students  roles  (i.e challenger, 
asker  for  clarification,  provider  of  information)  to 
structure  their  online  arguments.  Baker  (2004)  devel¬ 
oped  a  program  (CONNECT)  that  allows  online  stu¬ 
dents  to  agree  or  disagree  with  each  statement  made 
by  other  students.  Suthers  (2003)  examined  an  argu¬ 
ment  map  system  (Belvedere  3.0)  to  help  students 
engage  in  complex  scientific  argumentation.  Studies 
of  the  Virtual  Collaborative  Research  Institute  (VCRI) 
tool  (Jaspers  and  Erkens,  2002;  Munneke  et  al.,  2003) 
used  argumentative  diagrams  to  broaden  students’ 
arguments.  Graphic  representations  can  enhance  the 
controversy  process  by  forcing  participants  to  be 
explicit  and  complete  in  presenting  their  ideas  (Suthers 
and  Hundhausen,  2003;  van  Bruggen  et  al.,  2003),  can 
be  used  to  develop  resources  for  discussion  and  rea¬ 
soning  (Baker,  2003;  Suthers,  2003),  can  lead  to  a 


shared  focus  (Veerman,  2000),  and  can  organize  stu¬ 
dents’  thinking  and  maintain  coherence  during  prob¬ 
lem  solving  (Suthers,  2001). 

COOPERATIVE  LEARNING  AND 

TECHNOLOGY-SUPPORTED 

INSTRUCTION 

Learning  tends  to  be  enhanced  when  technology  pro¬ 
motes  cooperation  among  students  and  creates  a  shared 
experience.  In  analyzing  the  relationship  between  tech¬ 
nology  and  cooperative  learning,  a  distinction  can  be 
made  between  (Crook,  1994)  (see  Figure  33.2): 

•  Interacting  around  computers;  for  example, 
computers  may  be  used  to  facilitate  face-to- 
face  cooperation  among  students  (e.g.,  by 
providing  students  with  points  of  shared  ref¬ 
erence) 
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•  Interacting  through  computers;  for  example, 
local-area  networks  (LANs)  and  wide-area 
networks  (WANs)  and  the  global  version  of 
the  latter  (the  Internet)  provide  a  variety  of 
mediating  tools  for  cooperation,  such  as  e- 
mail  and  text  messaging,  chat  rooms,  bulle¬ 
tin  boards,  conferencing  systems,  Web 
pages,  blogs,  and  specialized  groupware. 

Cooperating  with  computers  (i.e.,  pedagogical  agents) 
is  a  growing  area  of  research. 

Cooperation  and  Computers 

Single-User  Programs  Reapplied 
to  Cooperative  Learning 

Many  computer  programs  were  developed  to  tailor 
learning  situations  to  individual  students;  however, 
applying  these  programs  to  cooperative  learning 
among  students  offers  several  advantages  (Crook, 
1994;  Hawkins  et  al.,  1982).  The  technical  extension 
of  LOGO  (Papert,  1980)  to  legoLOGO  (LEGO-brick 
robots  controlled  by  LOGO  programs)  has  been  an 
especially  promising  tool  for  creating  cooperation 
among  students  (Eraut,  1995;  Jarvela,  1996).  Cooper¬ 
ative  learning  has  been  promoted  by  many  different 
types  of  programs,  such  as  databases,  spreadsheets, 
math  programs,  programming  languages,  simulations, 
and  multimedia  authoring  tools  (Amigues  and  Agosti- 
nelli,  1992;  Brush,  1997;  Eraut,  1995;  Lehtinen  and 
Repo,  1996). 

Programs  Developed  to  Promote  Cooperation 

For  cooperation  to  take  place,  students  must  have  a 
joint  workspace.  One  of  the  promises  of  the  computer 
is  that  it  allows  students  to  create  shared  spaces. 
Instead  of  sharing  a  blackboard  or  a  worktable,  stu¬ 
dents  can  share  a  computer  screen.  Such  groupware 
(aimed  at  supporting  group  rather  than  individual 
work)  has  expanded  dramatically  the  past  20  years. 
Numerous  programs  in  a  variety  of  subject  areas  have 
been  developed  to  externalize  the  problem-solving 
process  by  displaying  the  student’s  solution  or  learning 
paths  on  the  screen,  and  they  generally  tend  to  be 
helpful  for  both  individual  reflection  and  cooperative 
problem  solving  (Pauli  and  Reusser,  1997;  Lehtinen  et 
al.,  1998).  Technology  and  cooperative  learning  have 
been  integrated  in  numerous  ways— for  example,  in 
CSILE,  the  Belvedere  system,  and  CoVis.  CSILE 
(Computer-Supported  Intentional  Learning  Environ¬ 
ment),  originally  developed  in  the  late  1980s  (Scarda- 
malia  et  al.,  1989),  uses  a  network  to  help  students 


build,  articulate,  explore,  and  structure  knowledge. 
The  system  contains  tools  for  text  and  chart  processing 
and  a  communal  database  for  producing,  searching, 
classifying,  and  linking  knowledge.  The  Belvedere 
system,  developed  by  Lesgold  et  al.  (1995),  focuses 
and  prompts  students’  cognitive  activity  by  giving 
them  a  graphical  language  to  express  the  steps  of 
hypothesizing,  data-gathering,  and  weighing  of  infor¬ 
mation.  CoVis  (Learning  through  Collaborative  Visu¬ 
alization  Project)  focuses  on  cooperative  project  work 
in  high-school  science  (Pea  et  al.,  1994)  by  utilizing 
advanced  networking  technologies,  collaborative  soft¬ 
ware,  and  visualization  tools  to  enable  students  and 
others  to  work  together  in  classrooms  and  across  the 
country  at  the  same  time  (synchronously)  or  at  differ¬ 
ent  times  (asynchronously).  These  and  other  group- 
ware  systems  provide  powerful  opportunities  for  coop¬ 
erative  learning. 

Cooperation  through  Computers 

The  rapid  expansion  of  computer  network  technology 
has  allowed  students  all  over  the  world  to  create  pow¬ 
erful  shared  spaces  on  the  computer  screen.  In  a  net¬ 
work-based  environment,  students  and  teachers  can 
interact  through  the  computer  free  of  the  limitations 
of  time  and  place.  The  speed  at  which  asynchronous 
and  distance  communication  may  be  completed  makes 
more  intensive  cooperation  possible  with  out-of¬ 
school  experts,  brings  students  from  different  schools 
into  contact  with  each  other,  and  creates  powerful  tools 
for  joint  writing  and  knowledge  sharing. 

The  network  environment  supports  cooperation  at 
different  levels.  From  a  series  of  studies,  Bonk  and 
King  (1995)  concluded  that  networks  can  (1)  change 
the  way  students  and  instructors  interact,  (2)  enhance 
cooperative  learning  opportunities,  (3)  facilitate  class 
discussion,  and  (4)  move  writing  from  being  a  solitary 
to  a  more  active  type  of  social  learning.  The  network 
tools  include  LAN-based  client-server  systems ,  which 
feature  software  programs  based  on  local  area  networks 
and  client-server  architectures  (e.g.,  CSILE,  Belved¬ 
ere,  and  CoVis).  Another  tool  is  e-mail,  which  is  used 
to  deliver  information  to  students,  supervise  students, 
and  support  national  and  international  communication 
between  cooperative  learning  groups  and  schools 
located  far  away  from  each  other.  With  the  help  of 
mailing  lists,  groups  of  students  can  use  e-mail  to  share 
joint  documents  and  comment  on  each  other’s  work. 

Also,  the  Internet  and  World  Wide  Web  may  be 
used  for  cooperative  learning.  Internet -based  confer¬ 
encing  systems  and  e-mail  systems  are  very  similar. 
Developments  in  broad-bandwidth  technology  have 
resulted  in  synchronous  shared  workspaces  and  two- 
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way  audio-visual  communication.  Computer  confer¬ 
encing  has  existed  since  the  first  computer  networks 
but  has  only  recently  been  implemented  as  part  of 
cooperative  learning.  Web-based  cooperative  learning 
is  time  independent  and  location  independent,  thus 
allowing  a  combination  of  synchronous  and  asynchro¬ 
nous  discussions.  Creating  and  using  shared  databases 
is  especially  helpful  for  network-based  cooperative 
learning  systems.  Live  video  makes  it  easier  for  indi¬ 
viduals  and  groups  to  conference  with  each  other.  Per¬ 
sonal  Web  pages  that  link  with  those  of  friends  and 
group  members  allow  individuals  to  create  networks 
of  potential  cooperators.  Web  logs  (blogs)  allow  indi¬ 
viduals  to  share  their  daily  lives  with  others,  exchange 
opinions  and  critiques  of  each  other’s  work,  carry  on 
a  dialog  with  others,  and  comment  on  current  affairs. 
Personal  Web  pages  and  blogs  make  it  easier  for  indi¬ 
viduals  to  create  a  social  presence  and  get  to  know 
each  other  on  a  personal  level.  Finally,  courses  and 
degree  programs  are  offered  on  the  Web  through  exist¬ 
ing  schools  and  through  universities  that  primarily 
operate  on  the  Web. 

A  fourth  network  tool  is  an  immersive,  interactive 
digital  entertainment,  or  videogame  playing,  which  has 
considerable  economic,  cultural,  and  social  influence 
(Squire,  2006)  and  is  often  used  for  cooperative 
endeavors.  Many  children,  adolescents,  and  young 
adults  spend  more  time  playing  in  digital  worlds  than 
they  do  watching  television,  reading,  or  watching  films 
(Funk  et  al.,  1999;  Williams,  2003).  These  games  are 
the  leading  edge  of  a  culture  of  simulation.  Simulations 
such  as  Full  Spectrum  Warrior  and  America’s  Army 
have  changed  the  training  of  soldiers  in  the  U.S.  Army. 
Farming  and  town  simulations  such  as  Animal  Cross¬ 
ing  and  Harvest  Moon  have  make  it  possible  for  young 
children  to  plan  and  plant  crops,  pay  off  mortgages, 
and  essentially  run  a  farm.  Disney’s  Toontown  allows 
children  from  around  the  world  to  interact  in  a  real¬ 
time,  three-dimensional  world  where  they  meet  and 
engage  in  cooperative  quests. 

Simulations  can  utilize  cooperative  efforts  to 
engage  participants  in  ideological  systems,  learning 
by  performing,  and  designed  experiences  (Squire, 
2006).  First,  ideologies  are  taught  by  games  such  as 
Grand  Theft  Auto  (which  teaches  students  how  to 
survive  in  a  crime-tilled  society)  and  Civilization  III 
(which  involves  ruling  a  civilization  from  4000  B.C. 
to  the  present,  primarily  teaching  the  ideologies  of 
colonialization,  economic  growth,  and  democracy). 
The  games  immerse  players  in  complex  cooperative 
systems,  allowing  them  to  learn  the  points  of  view  of 
those  systems  and  develop  identities  within  the  sys¬ 
tems;  thus,  organizations  such  as  the  U.S.  military  are 
using  games  to  support  their  ideological  agenda.  Sec¬ 


ond,  a  core  characteristic  of  games  is  that  they  are 
organized  for  learning  through  doing  or  performance 
(Squire,  2006).  Through  recurring  cycles  of  perceiv¬ 
ing  and  acting,  thinking  and  doing,  players  learn  from 
their  experiences,  usually  in  cooperation  with  other 
players.  In  Toontown ,  for  example,  players  band 
together  in  teams  to  play  pranks  on  cos,  evil  cartoon 
villains  who  want  to  turn  Toontown  into  a  drab  office 
environment.  Finally,  games  provide  designed  expe¬ 
riences  from  which  participants  are  active  construc¬ 
tors  of  meaning  with  their  own  drives,  goals,  and 
motivations.  In  complex  games  such  as  The  Sims, 
players  form  families  and  live  in  communities;  in 
World  of  Warcraft,  large  numbers  of  players  (over  7 
million  worldwide)  form  teams  to  solve  puzzles ,  over¬ 
come  challenges ,  and  achieve  tasks .  Augmented  real¬ 
ity  simulation  games  (Klopfer  and  Squire,  in  press) 
place  students  in  roles  as  investigators,  scientists,  or 
activists  and  have  students  work  in  cooperative  teams 
to  identify  problems,  pose  data-gathering  strategies, 
draw  conclusions,  and  reframe  their  hypotheses  as 
they  work.  Increasingly,  immersive,  interactive  digital 
entertainment  will  become  part  of  classroom  experi¬ 
ence,  especially  when  it  is  coupled  with  cooperative 
learning. 

The  fifth  network  tool,  and  one  of  the  most  inter¬ 
esting  uses  of  the  Web  utilizing  cooperative  learning, 
is  adventure  learning  programs,  such  as  GoNorth  (a 
free  adventure  learning  program  for  K-12  classes; 
www.PolarHusky.com)  (Doering,  2006).  From  2006 
to  2010  a  team  of  educators,  scientists,  and  explorers 
will  dogsled  live  to  five  circumpolar  Arctic  locations 
and  share  those  experiences  with  students  around  the 
world.  A  curriculum  and  activity  guide  are  provided 
for  each  trek.  Activities  on  the  trail  are  synched  real 
time  to  the  curriculum  so  students  can  engage  in  pow¬ 
erful  cooperative  experiences  with  the  educators,  sci¬ 
entists,  and  explorers  as  well  with  the  huskies  pulling 
the  dogsleds  (i.e.,  classes  can  adopt  a  dog  and  get 
involved  in  its  feeding  and  care  as  well  as  its  daily 
performance).  Live  field  updates  and  field  research 
findings  are  presented  in  collaboration  with  NASA  and 
the  National  Science  Foundation.  The  result  is  a  com¬ 
munity  of  over  3  million  learners  throughout  the  world 
acquiring  knowledge  from  the  expedition,  the  Arctic 
peoples,  subject-matter  experts,  and  each  other  (Doer¬ 
ing,  in  press). 

Adding  technology  to  a  lesson  inherently  increases 
the  lesson’s  complexity.  When  students  participate  in 
technology-supported  instruction,  they  have  the  dual 
tasks  of  learning  how  to  use  the  technology  (i.e.,  the 
hardware  and  software  required  by  the  lesson)  and 
mastering  the  information,  skills,  procedures,  and  pro¬ 
cesses  being  presented  within  the  technology.  When 


416 


Cooperation  and  the  Use  of  Technology 


cooperative  learning  groups  are  used,  students  have  the 
additional  task  of  learning  teamwork  procedures  and 
skills;  consequently,  the  initial  use  of  technology-sup¬ 
ported  cooperative  learning  may  take  more  time,  but 
once  students  and  teachers  master  the  new  systems  the 
results  will  be  worth  the  effort. 

Cooperation  with  Computers 

The  computer  itself  may  be  a  member  of  a  cooperative 
group.  Pedagogical  agents,  human-like  virtual  charac¬ 
ters  employed  in  electronic  learning  environments 
(Baylor,  2002),  can  interact  directly  with  students  as 
if  they  themselves  were  human  (Doering  and  Veletsi- 
anos,  in  press;  Doering  et  al.,  in  press;  Veletsianos, 
2006,  2007).  Pedagogical  agents  have  been  utilized  in 
a  variety  of  subject  areas  (Veletsianos,  2006,  2007). 
Because  humans  tend  to  interact  with  computers  and 
media  as  if  they  were  human,  social  interactions 
between  humans  and  computers  may  approximate 
social  interactions  among  humans  (Reeves  and  Nass, 
1996;  Veletsianos,  2006).  If  the  computer  is  repre¬ 
sented  as  a  human-like  character  able  to  dynamically 
converse  and  interact  in  real  time  with  the  user,  the 
distinction  between  human-human  and  human-com¬ 
puter  interaction  may  blur,  creating  a  situation  in 
which  the  pedagogical  agent  and  the  student  work 
together  as  a  cooperatively  pair.  An  interesting  aspect 
of  student-pedagogical  agent  interaction  is  that  con¬ 
flicts  can  arise  when  students  become  frustrated  by  the 
way  the  agent  interacts  or  by  the  lack  of  agent 
response.  Student  anger  at  an  agent  indicates  that  ped¬ 
agogical  agents  may  soon  be  created  that  will  be  effec¬ 
tive  members  of  students’  cooperative  groups  and  will 
be  interacted  with  similarly  to  interaction  among 
human  members.  Some  day  further  in  the  future,  ped¬ 
agogical  agents  may  even  exhibit  emotions  and  a 
human-like  intelligence  (Veletsianos,  2006). 

Cost  Effectiveness 

The  use  of  cooperative  learning  increases  the  cost  effec¬ 
tiveness  of  technology.  Although  the  range  of  technol¬ 
ogy  that  could  be  used  in  schools  is  increasing  yearly, 
the  cost  of  adopting  new  technologies  is  an  inhibiting 
factor  to  its  use.  Giving  cooperative  learning  groups 
access  to  the  latest  technology  is  usually  more  cost 
effective  (not  only  for  initial  cost,  but  also  for  mainte¬ 
nance  and  upkeep)  than  giving  each  individual  student 
access.  An  historical  example  is  the  adoption  of  com¬ 
puters  by  schools.  By  having  groups  work  at  computers 
(instead  of  individuals)  schools  have  been  able  to  sig¬ 
nificantly  reduce  the  cost  of  obtaining  and  maintaining 
computers  (Johnson  and  Johnson,  1986;  Wizer,  1987). 


THE  FUTURE  OF 
TECHNOUOGY-SUPPORTED 
COOPERATIVE  UEARNING 

The  future  of  technology-supported  cooperative  learn¬ 
ing  seems  bright,  yet  it  is  built  primary  upon  its  rapid 
evolution  and  its  great  promise.  The  full  promise  of 
technology-supported  cooperative  learning  will  be 
realized  when  a  number  of  factors  are  emphasized. 
Social  interdependence  theory  provides  a  framework 
for  understanding  the  relationship  between  the  instruc¬ 
tional  use  of  technology  and  cooperative  learning. 
When  technology  isolates  students  or  places  them  in 
competition  with  each  other,  learning  will  tend  to  be 
obstructed.  When  technology  is  combined  with  coop¬ 
erative  learning,  academic  achievement  is  likely  to  be 
enhanced. 

Research  on  technology-supported  cooperative 
learning  must  be  increased  and  should  be  more  theo¬ 
retically  based.  Almost  all  of  the  existing  research  has 
focused  on  the  impact  of  specific  software  programs 
on  achievement  and  related  variables  without  testing 
theory.  In  the  future,  theoretically  oriented  research 
should  be  conducted,  and  the  potential  outcomes  stud¬ 
ied  should  be  expanded.  The  unique  strengths  of  tech¬ 
nology-supported  cooperative  learning  have  not  been 
documented. 

The  lack  of  theorizing  and  the  narrowness  of  the 
research  have  created  a  corresponding  limitation  on 
the  operational  procedures  for  implementation.  The 
implementation  of  technology-based  cooperative 
learning  has  been  driven  by  software  development  and 
by  the  traditional  ways  of  structuring  cooperation.  As 
technology  opens  new  avenues  for  cooperation, 
research  should  be  conducted  to  provide  better  guid¬ 
ance  as  to  how  to  structure  technology-supported 
cooperative  learning  in  ways  that  optimize  its  impact 
and  how  to  train  instructors  and  students  in  implement¬ 
ing  new  procedures. 

It  is  clear  from  social  interdependence  theory  and 
its  validating  research  that  the  effective  creation  of 
operational  procedures  depends  on  the  inclusion  of  five 
basic  elements:  positive  interdependence,  individual 
accountability,  promotive  interaction,  appropriate  use 
of  social  skills,  and  group  processing.  Groupware  pro¬ 
grams  need  to  include  these  five  elements  more  explic¬ 
itly  in  their  framework. 

Longitudinal  studies  are  necessary  that  track  the 
use  of  technology-supported  cooperative  learning  for 
at  least  one  school  year  and  ideally  for  several  years . 
Although  short-term  studies  of  initial  use  can  be 
quite  helpful,  investigating  the  long-term  effects  of 
using  technology-supported  cooperative  learning  is 
essential. 
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One  might  ask  whether  or  not  technology-sup¬ 
ported  instruction  will  increase  inequality  in  educa¬ 
tional  outcomes  (Becker  and  Sterling,  1987).  Students 
who  learn  how  to  use  technology  in  a  cooperative 
effort  will  have  an  advantage  over  those  who  do  not 
and  will  tend  to  have  more  learning  opportunities 
involving  technology. 

Whereas  advocates  of  technology  see  a  revolution 
coming  in  instruction,  historians  point  to  the  virtual 
absence  of  lasting  or  profound  changes  in  classroom 
practice  over  the  past  100  years.  Lepper  and  Gurtner 
(1989)  argued  that  the  last  technology  to  have  had  a 
major  impact  on  the  way  schools  are  run  is  the  black¬ 
board.  Most  often  new  technologies  are  used  in  ways 
that  do  not  disrupt  regular  classroom  practices,  which 
means  that  they  can  be  ignored  or  dropped  with  no 
disruption  to  ongoing  classroom  life.  Similarly,  soft¬ 
ware  selection  is  often  conducted  with  the  intention  of 
supporting  existing  classroom  practices  rather  than 
transforming  them.  Considerably  more  research  is 
needed  on  the  implementation  process  by  which  the 
combination  of  cooperative  learning  and  learning  tech¬ 
nologies  becomes  integrated  and  institutionalized  in 
classroom  and  schools. 


SUMMARY 

We  live  in  a  networked,  information-based  society  in 
which  teams  and  technology  must  manage  the  com¬ 
plexity  of  learning,  work,  and  living.  For  education  to 
develop  the  technological  and  teamwork  competencies 
of  children,  adolescents,  and  young  adults,  we  must 
overcome  the  individualistic  assumption  that  is  histor¬ 
ically  connected  with  technology-supported  instruc¬ 
tion  and  should  utilize  cooperative  learning  as  an 
inherent  part  of  instruction.  Technology  may  be  more 
productively  utilized  when  it  is  combined  with  coop¬ 
erative  learning.  To  be  a  cooperative  group,  five  basic 
elements  must  be  structured  within  the  learning  situa¬ 
tion:  positive  interdependence,  promotive  interaction, 
individual  accountability,  social  skills ,  and  group  pro¬ 
cessing.  The  four  types  of  cooperative  learning  are 
formal  cooperative  learning,  informal  cooperative 
learning,  cooperative  base  groups,  and  academic  con¬ 
troversy.  The  worldwide  use  of  cooperative  learning  is 
based  on  a  well-formulated  theory  (i.e.,  social  interde¬ 
pendence  theory)  that  has  been  validated  by  numerous 
research  studies  and  operationalized  into  practical  pro¬ 
cedures  that  can  be  used  at  any  level  of  education. 
Technology-supported  cooperative  learning  tends  to 
encourage  achievement  (both  academic  achievement 
and  learning  how  to  use  technology),  positive  attitudes 
(toward  technology  and  cooperation),  healthy  devel¬ 


opment  (cognitive  development,  learning  control, 
social  competencies),  positive  relationships  with  team 
members,  positive  effects  on  both  high-  and  low-per¬ 
forming  students  and  both  male  and  female  students, 
cost  effectiveness,  and  innovation  in  groupware  and 
hardware. 

What  this  research  illuminates  is  that  cooperative 
learning  and  technology-supported  instruction  have 
complementary  strengths.  The  more  technology  is 
used  to  teach,  the  more  necessary  is  cooperative  learn¬ 
ing.  The  computer,  for  example,  can  control  the  flow 
of  work,  monitor  accuracy,  give  electronic  feedback, 
and  do  calculations.  Cooperative  learning  provides  a 
sense  of  belonging,  the  opportunity  to  explain  and 
summarize  what  is  being  learned,  shared  mental  mod¬ 
els,  social  models,  respect  and  approval  for  efforts  to 
achieve,  encouragement  of  divergent  thinking,  and 
interpersonal  feedback  on  academic  learning  and  the 
use  of  the  technology. 

There  are  few  educational  innovations  that  have 
the  promise  of  technology-supported  cooperative 
learning.  The  combination  of  cooperation  and  technol¬ 
ogy  may  change  the  way  courses  are  being  delivered, 
instruction  is  taking  place,  and  the  nature  of  classroom 
and  school  life.  More  theorizing,  research,  and  refine¬ 
ment  of  practice  are  needed  to  help  the  field  realize  its 
possibilities. 
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ABSTRACT 

Cognitive  apprenticeship  is  a  process  by  which  learn¬ 
ers  learn  from  a  more  experienced  person  by  way  of 
cognitive  and  metacognitive  skills  and  processes.  This 
chapter  explores  the  elements  of  cognitive  apprentice¬ 
ship,  first  offering  definitions  and  a  historical  context, 
then  moving  into  a  review  of  research.  The  research 
review  is  organized  with  a  three-part  focus:  on  studies 
that  investigate  a  holistic  approaches  to  educational 
applications  of  the  process  of  cognitive  apprenticeship; 
on  studies  that  investigate  portions  of  the  process,  such 
as  scaffolding  or  mentoring;  and  on  studies  that  inves¬ 
tigate  cognitive  apprenticeship  activities  within  com¬ 
munities  of  practice.  Discussion  about  the  intersection 
of  technology  and  cognitive  apprenticeship  research  is 
imbedded  within  each  of  the  three  areas  of  focus, 
reflecting  the  steady  increase  of  systematically 
designed,  computer- mediated  instruction  that  is  based 
in  social  learning  theories,  especially  cognitive  appren¬ 
ticeship  theories.  Empirical  studies  have  confirmed 
much  of  what  theories  have  suggested:  (1)  that  the 
cognitive  apprenticeship  model  is  an  accurate  descrip¬ 
tion  of  how  learning  occurs,  and  (2)  that  the  instruc¬ 
tional  strategies  that  have  been  extracted  from  these 
observations  of  everyday  life  can  be  designed  into 
more  formal  learning  contexts  with  positive  effect.  The 
chapter  concludes  with  a  call  for  more  systematic  and 
integrated  program  of  studies  working  toward  the 
development  of  guiding  principles  to  support  instruc¬ 
tional  design,  teaching,  and  learning  based  on  the  cog¬ 
nitive  apprenticeship  model. 

KEYWORDS 

Apprenticeship:  A  process  through  which  a  more  expe¬ 
rienced  person  assists  a  less  experienced  one  by 
way  of  demonstration,  support,  and  examples. 
Articulation:  In  cognitive  apprenticeship,  verbalizing 
the  results  of  reflective  acts. 

Coaching:  In  cognitive  apprenticeship,  assisting  and 
supporting  learners’  cognitive  activities. 

Cognitive  apprenticeship:  An  apprenticeship  process 
that  utilizes  cognitive  and  metacognitve  skills  and 
processes  to  guide  learning. 

Community  of  practice:  A  group  of  people  bound  by 
participation  in  an  activity  common  to  them  all; 
may  be  formal  or  informal. 

Exploration:  In  cognitive  apprenticeship,  forming  and 
testing  a  personal  hypothesis  in  pursuit  of  learning. 
Modeling:  In  cognitive  apprenticeship,  demonstrating 
thought  processes. 

Reflection:  In  cognitive  apprenticeship,  self-analysis 
and  self-assessment. 


Scaffolding:  Support  that  is  provided  to  assist  learners 
in  reaching  skill  levels  beyond  their  current  abili¬ 
ties;  essential  to  scaffolding  is  fading  the  support 
inversely  to  the  learners’  acquisition  of  the  skill 
that  is  being  supported. 

Situatedness:  The  context  or  constellation  of  influen¬ 
tial  events  and  elements  that  govern  and  shape 
human  life. 

Zone  of  proximal  development  (ZPD):  A  term  coined 
by  Vygotsky  to  describe  the  space  between  a 
learner’s  current  skill  level  and  the  next  skill  level 
that  the  learner  cannot  reach  without  assistance. 


INTRODUCTION 

Long  before  education  was  a  field  studied  in  universi¬ 
ties— indeed,  long  before  universities  even  existed— 
people  were  learning  via  apprenticeship.  Most  simply 
put,  it  is  a  process  through  which  a  more  experienced 
person  assists  a  less  experienced  one,  providing  sup¬ 
port  and  examples,  so  the  less  experienced  person 
gains  new  knowledge  and  skills.  Apprenticeship  is  the 
process  through  which  a  parent  may  teach  a  child  how 
to  tie  her  shoes  and  the  process  through  which  a  person 
may  learn  to  become  a  chef  or  a  tailor.  In  the  first 
example,  one  would  not  expect  the  child  to  see  a  dem¬ 
onstration  and  be  able  to  tie  a  shoe  with  no  assistance 
on  the  first  try.  Similarly,  it  seems  logical  that  a  new 
chef  starts  out  with  simpler  tasks,  such  as  chopping 
ingredients  or  garnishing  plates,  and  works  his  way  up 
to  preparing  entire  dishes  and  meals.  Often  larger  skills 
are  broken  into  smaller  ones,  and  supports  are  provided 
so  that  tasks  that  are  given  to  the  apprenticing  learner 
are  within  the  reach  of  the  learner’s  current  ability  level 
or  zone  of  proximal  development  (ZPD)  (Vygotsky, 
1978).  Also  critical  to  apprenticeship  is  that  tasks  must 
be  representative  of  authentic  skills  and  not  merely 
classroom- type  exercises. 

Apprenticeship  programs  have  been  formalized  in 
many  vocational  education  programs;  for  example,  to 
become  a  journeyman  electrician,  one  must  work 
through  various  levels  of  apprenticeship.  The  educa¬ 
tional  value  of  apprenticeship,  however,  is  not  limited 
to  the  learning  psychomotor  skills  or  vocational  trades. 
Apprenticeships  can  just  as  readily  support  cognitive 
and  metacognitive  learning  processes  and  may  appear 
in  both  formal  and  informal  learning  environments. 
This  chapter  first  provides  a  brief  description  of  con¬ 
cepts  related  to  the  cognitive  apprenticeship  model, 
followed  by  a  description  of  the  instructional  strategies 
that  comprise  this  model.  Finally,  it  presents  a  summary 
of  recent  research  related  to  the  use  of  cognitive  appren¬ 
ticeship  and  its  component  instructional  strategies. 
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COGNITIVE  APPRENTICESHIP 
DEFINED 

The  concept  of  a  cognitive  apprenticeship— defined  as 
“learning  through  guided  experience  on  cognitive  and 
metacognitive,  rather  than  physical,  skills  and  pro¬ 
cesses”  by  Collins  et  al.  (1989,  p.  456)  — has  its  roots 
in  social  learning  theories.  One  cannot  engage  in  a 
cognitive  apprenticeship  alone,  but  rather  it  is  depen¬ 
dent  on  expert  demonstration  (modeling)  and  guidance 
(coaching)  in  the  initial  phases  of  learning.  Learners 
are  challenged  with  tasks  slightly  more  difficult  than 
they  can  accomplish  on  their  own  and  must  rely  on 
assistance  from  and  collaboration  with  others  to 
achieve  these  tasks.  In  other  words,  learners  must  work 
with  more  experienced  others  and  with  time  move  from 
a  position  of  observation  to  one  of  active  practice.  The 
learning  tasks  in  cognitive  apprenticeship  are  holistic 
in  nature  (see  Chapter  35  on  whole-task  models  in  this 
Handbook)  and  increase  in  complexity  and  diversity 
over  time  as  the  learner  becomes  more  experienced.  A 
major  advantage  of  learning  by  cognitive  apprentice¬ 
ship  as  opposed  to  traditional  classroom-based  meth¬ 
ods  is  the  opportunity  to  see  the  subtle,  tacit  elements 
of  expert  practice  that  may  not  otherwise  be  explicated 
in  a  lecture  or  knowledge-dissemination  format. 

Instructional  Strategies  and  Models 
Associated  with  Cognitive  Apprenticeship 

Although  cognitive  apprenticeships  readily  occur  on 
their  own,  without  intervention,  certain  instructional 
strategies  are  hallmarks  of  the  theory  and  can  be  pur¬ 
posely  implemented  to  support  learning.  Intentional 
teaching  and  learning  through  cognitive  apprenticeship 
require  making  tacit  processes  visible  to  learners  so  they 
can  observe  and  then  practice  them  (Collins  et  al.,  1989). 
The  basic  model  consists  of  the  following  strategies: 

•  Modeling— Demonstrating  the  thinking  pro¬ 
cess 

•  Coaching— Assisting  and  supporting  student 
cognitive  activities  as  needed  (includes  scaf¬ 
folding) 

•  Reflection— Self-analysis  and  assessment 

•  Articulation— Verbalizing  the  results  of 
reflection 

•  Exploration— Formation  and  testing  of  one’s 
own  hypotheses 

Note  that  these  strategies  refer  to  the  teacher’s  or 
expert’s  actions:  the  learners  in  cognitive  apprentice¬ 
ships  (CAs)  are  engaged  in  acts  of  observation,  prac¬ 
tice,  and  reflection. 


Collins  and  colleagues’  (1989)  model  generally  is 
considered  the  foundational  one,  but  other  slightly  dif¬ 
ferent  versions  have  been  proposed.  Gallimore  and 
Tharp  (1990)  identified  six  forms  of  scaffolded  assis¬ 
tance:  (1)  instructing,  (2)  questioning,  (3)  modeling,  (4) 
feeding  back,  (5)  cognitive  structuring,  and  (6)  contin¬ 
gency  management.  Enkenberg  (2001)  added  scaffold¬ 
ing  and  explanation  as  key  strategies.  LeGrand  Brandt 
et  al.  (1993)  presented  a  sequential  model  of  modeling 
(both  behavioral  and  cognitive),  approximating,  fading, 
self-directed  learning,  and  generalizing.  Liu  (2005), 
who  used  a  cognitive  apprenticeship  approach  to  sup¬ 
port  preservice  education,  offers  instructional  designers 
a  three-phase  Web-based  CA  model  with  a  dynamic 
relationship  between  the  initial  modeling-observing 
phase  and  the  second  scaffolding-practice  phase,  which 
then  is  followed  by  the  guiding-generalizing  phase. 
The  similarities  across  these  models  are  their  reliance 
on  instructional  strategies  that  provide  learner  guidance 
and  engage  learners  in  different  types  of  practice  until 
the  guidance  is  no  longer  needed. 

Concepts  Associated  with 
Cognitive  Apprenticeship 

Four  key  concepts  commonly  discussed  in  the  cognitive 
apprenticeship  literature  are  (1)  situatedness,  (2)  legit¬ 
imate  peripheral  participation,  (3)  guided  participation, 
and  (4)  membership  in  a  community  of  practice. 

Situatedness 

Situated  learning  is  active  learning  that  takes  place  via 
one’s  participation  in  an  authentic  task  or  setting  (Lave 
and  Wenger,  1991).  Context,  or  situatedness,  reflects 
the  ways  in  which  cultural,  historical,  and  institutional 
factors  influence  the  actions  of  our  everyday  lives 
(Brown  et  al.,  1989;  Rogoff,  1990;  Wertsch,  1998). 
Learning  that  occurs  within  the  context  of  application 
is  considered  more  likely  to  result  in  improved  prac¬ 
tice;  for  example,  would  you  prefer  to  receive  medical 
treatment  from  someone  who  has  classroom  training 
only  or  someone  who  has  trained  on  actual  patients  in 
a  clinical  setting?  As  Brown  et  al.  (1998,  p.  230)  indi¬ 
cated,  “The  central  issue  in  learning  is  becoming  a 
practitioner,  not  learning  about  practice.”  Current  edu¬ 
cational  systems,  particularly  universities,  have  been 
criticized  for  separating  learning  from  practice,  result¬ 
ing  in  an  education  that  does  not  sufficiently  prepare 
students  for  job  performance  (Enkenberg,  2001);  in 
other  words,  these  systems  are  criticized  when  they 
lack  situatedness  and  fail  to  engage  learners  in  authen¬ 
tic  practices  with  cultural  tools  and  natural  perfor¬ 
mance  conditions. 
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Legitimate  Peripheral  Participation 

In  cognitive  apprenticeship,  a  newcomer  who  prima¬ 
rily  observes  is  considered  a  legitimate  peripheral  par¬ 
ticipant.  In  essence,  this  label  validates  observation  as 
a  learning  activity.  It  would  be  unreasonable  to  expect 
a  newcomer  to  be  a  full  participant  in  an  activity.  One 
must  learn  not  only  the  whole  tasks  to  be  accomplished 
and  their  assessment  criteria  but  also  the  smaller  tasks 
that  comprise  them.  An  apprentice  can  gain  initial 
experience  through  observing  a  holistic  process  from 
the  periphery.  Once  the  big  picture  is  understood,  par¬ 
ticipation  can  shift  from  peripheral  to  active,  with  the 
learner  completing  smaller,  component  parts  of  the 
larger  task  while  receiving  iterations  of  feedback  from 
someone  who  is  more  experienced.  At  this  point,  the 
learner  is  no  longer  a  legitimate  peripheral  participant, 
but  instead  is  inbound,  beginning  to  identify  more  with 
insiders  of  the  community’s  practice. 

Guided  Participation 

Guided  participation  is  the  social  element  of  cognitive 
apprenticeship.  Often  the  guidance  is  provided  tacitly, 
as  one  naturally  participates  in  everyday  life  (Rogoff, 
1990);  there  is  an  inherently  situated  component  to 
guided  participation.  Guided  participation,  to  be  suc¬ 
cessful,  must  take  place  within  a  learner’s  zone  of 
proximal  development  (ZPD).  The  ZPD,  as  originally 
defined  by  Vygotsky  (1978),  is  a  dynamic  region  that 
is  just  beyond  the  learner’s  current  ability  level;  the 
ZPD  of  a  learner  gaining  new  skills  and  understanding 
moves  with  that  learner’s  development.  This  space 
between  actual  and  potential  performance  is  assessed 
through  social  interaction  between  the  learner  and 
someone  who  is  more  experienced  — potentially  a 
teacher,  parent,  or  even  an  advanced  peer.  Tharp  and 
Gallimore  (1988)  used  a  four-stage  model  to  describe 
the  dynamic  and  recursive  process  through  which 
learners  work  within  their  ZPDs  and  come  to  internal¬ 
ize  knowledge,  only  to  begin  again  with  newly  defined 
ZPDs.  Rogoff  ( 1990,  p.  16)  noted  that  cultural  learning 
and  development,  in  addition  to  individual  cognitive 
development,  occur  as  a  result  of  teaching  and  learning 
in  the  ZPD: 

Interactions  in  the  zone  of  proximal  development  are  the 
crucible  of  development  and  of  culture,  in  that  they  allow 
children  to  participate  in  activities  that  would  be  impos¬ 
sible  for  them  alone,  using  cultural  tools  that  themselves 
must  be  adapted  to  the  specific  practical  activities  at  hand. 

This  observation  again  stresses  the  situated  nature  and 
social  interconnectedness  of  learning  through  cogni¬ 
tive  apprenticeship. 


Community  of  Practice 

Although  learning  organizations  and  institutions  have 
sought  to  implement  elements  of  cognitive  apprentice¬ 
ships  in  formal  learning  situations,  cognitive  appren¬ 
ticeships  often  naturally  occur  within  a  community  of 
practice  (CoP).  A  community  of  practice  is  a  group  of 
people  — either  formally  or  informally  bound— who 
engage  in  and  identify  themselves  with  a  common 
practice.  Examples  of  a  CoP  might  be  educators  within 
a  given  school  district  or  members  of  a  professional 
organization  for  clarinetists.  What  brings  these  people 
together  as  a  CoP  are  three  critical  elements: 

•  Mutual  engagement— A  shared  task  or  inter¬ 
est  and  a  resulting  identity 

•  Joint  enterprise— A  common  set  of  commu¬ 
nity  standards  and  expectations 

•  Shared  repertoire— A  common  vocabulary 
that  differentiates  the  CoP  from  others 

Wenger  (1998)  suggested  the  following  trajectories  as 
a  model  of  how  membership  within  a  community  of 
practice  occurs: 

•  Peripheral— One  who  may  not  become  an 
insider  to  the  community  but  who  neverthe¬ 
less  takes  part  in  community  events  (e.g., 
parents  who  volunteer  in  the  classroom) 

•  Inbound—  A  person  who  is  becoming  a  fully 
participating  member  of  the  community 
(e.g.,  a  student  teacher  or  brand  new  teacher) 

•  Insider— A  person  who  has  become  a  fully 
participating  member  of  a  community  (e.g., 
a  teacher) 

•  Boundary— A  person  who  is  not  a  fully  par¬ 
ticipating  member  of  the  community  but 
who  participates  by  bringing  a  different  set 
of  skills  or  services  to  the  community  (e.g., 
a  technology  specialist) 

•  Outbound— A  person  who  is  preparing  to 
leave  the  community  (e.g.,  a  teacher  who  is 
moving  to  an  administrative  position  or  pre¬ 
paring  to  retire) 

Although  one  may  enter  a  community  on  one  tra¬ 
jectory  point  and  move  to  other  points  (e.g.,  inbound 
to  insider  and  eventually  to  outbound),  such  a  path  is 
not  mandated.  Learning  need  not  occur  through  the 
interactions  of  participants  on  different  levels  of  the 
trajectory;  for  example,  teacher  professional  develop¬ 
ment  may  occur  through  peer  reciprocal  teaching 
(Glazer  and  Hannafin,  2006).  Movement  through  the 
trajectories  is  fluid  in  many  contexts,  although  some 
communities  may  have  formal  levels  of  membership 
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that  are  aligned  with  these  labels.  The  utility  of  the 
labels  is  that  they  support  understanding  of  the  differ¬ 
ent  ways  in  which  people  might  participate  in  a  CoP 
or  a  cognitive  apprenticeship. 

RESEARCH  ON  COGNITIVE 
APPRENTICESHIP 

The  body  of  research  on  cognitive  apprenticeship  has 
been  growing  steadily  and  in  many  ways  overlaps  with 
research  on  other  constructivist  learning  theories  and 
methods.  For  this  chapter,  we  sought  to  include  recent 
reports  of  empirical  research  on  cognitive  apprentice¬ 
ship.  Theoretical  works  or  ones  merely  describing 
instructional  projects,  software,  or  elements  of  practice 
have  not  been  included  here.  We  have  separated  the 
research  into  studies  focusing  on  (1)  the  whole  CA 
model  as  enacted  in  instructional  settings,  (2)  individ¬ 
ual  instructional  strategies  associated  with  CA  (men¬ 
toring,  scaffolding),  and  (3)  cognitive  apprenticeship 
within  communities  of  practice.  We  note,  however,  that 
this  separation  is  somewhat  artificial  given  the  inter¬ 
relatedness  of  concepts  and  strategies.  Additionally, 
we  have  limited  this  review  to  studies  with  a  primary 
focus  on  cognitive  apprenticeship  and  related  strate¬ 
gies.  Studies  on  trade  and  vocational  apprenticeships 
were  not  included  because  they  tend  to  focus  on  issues 
other  than  cognitive  problem-solving  skills.  Similarly, 
studies  that  merely  mention  instructional  strategies 
related  to  CA  but  do  not  focus  explicitly  on  how  those 
strategies  relate  to  CA  were  not  included.  In  excluding 
such  studies,  this  chapter  is  not  an  exhaustive  one,  but 
it  is  representative  of  the  types  of  research  being  done. 

Studying  the  Enactment 
of  Cognitive  Apprenticeship 

Studies  examining  cognitive  apprenticeship  have 
researched  both  the  parts  and  the  whole.  The  parts  are 
generally  understood  to  be  the  instructional  phases 
outlined  by  Collins  et  al.  (1989),  whereas  the  whole 
consists  of  the  process  of  these  events  occurring  at  a 
specific  time  and  place  with  unique  individuals  co¬ 
constructing  the  series  of  apprenticeship  moments. 
These  studies  attempt  to  identify  the  critical  elements 
of  the  CA  episodes  across  settings  and  with  varied 
populations.  Cognitive  apprenticeship  is  especially 
appealing  to  designers  of  Web-based  learning  environ¬ 
ments  who  are  embracing  a  more  constructivist 
approach  to  learning  and  instruction.  Similarly,  CA  has 
begun  to  find  a  home  in  both  K-12  education  and 
teacher  education  programs,  both  of  which  have  been 
researched  contexts. 


Cognitive  Apprenticeship 
in  Multimedia  Environments 

One  dominant  belief  is  that  multimedia  and  Web-based 
environments  can  be  programmed  to  support  cognitive 
apprenticeship  processes.  Many  such  environments 
focus  on  one  or  two  strategies  related  to  cognitive 
apprenticeship  and  are  discussed  later  in  this  chapter, 
but  a  few  look  at  how  to  address  the  entire  CA  model. 
Wang  and  Bonk  (2005)  proposed  using  the  CA  model 
as  the  basis  for  constructing  a  groupware  environment. 
Seel  and  Schenk  (2003)  used  a  CA  model  that 
sequences  activities  in  the  same  order  as  Collins  et  al. 
(1989),  with  an  interest  in  developing  a  multimedia- 
based  system  to  support  model-based  learning.  Their 
formative  evaluation  of  five  replication  studies  showed 
that  CA  may  be  effective  as  a  guide  for  the  design  of 
multimedia  learning  environments,  with  scaffolding 
being  the  weakest  spot.  Their  findings  substantiated 
earlier  studies  on  integrating  CA  and  multimedia 
(Casey,  1996)  on  computer-based  coaching  (Lajoie 
and  Lesgold,  1989).  Generally,  addressing  individual 
learner  needs  in  a  programmed  environment  has 
proven  challenging  to  do  but  promising  for  supporting 
learning;  thus,  researchers  and  developers  continue  to 
work  on  ways  of  implementing  elements  of  cognitive 
apprenticeship  in  multimedia  environments. 

Cognitive  Apprenticeship  in  Higher  Education 

Many  of  the  studies  of  cognitive  apprenticeship  in 
higher  education  are  focused  on  teacher  education  pro¬ 
grams.  Two  studies  in  particular  exemplify  the  types 
of  research  being  conducted  on  CA  in  the  field  of 
teacher  education:  de  Jager  et  al.  (2002)  and  Liu  (2005) 
each  looked  at  CA  and  teacher  training.  In  the  first, 
participants  were  trained  in  CA  and  in  the  second 
participants  were  trained  using  a  CA  approach. 

Targeting  the  instructional  design  behavior  of  mid¬ 
dle  grade  in-service  teachers,  de  Jager  and  colleagues 
(2002)  showed  that,  simply  put,  teacher  training  results 
in  a  change  in  teacher  teaching  behaviors.  Specifically, 
their  study  offered  teachers  training  in  a  CA  approach 
or  a  directed  instruction  approach  to  reading  compre¬ 
hension  then  compared  their  behaviors  with  a  control 
group  of  teachers  who  used  the  established  curricular 
approach.  Both  experimental  groups  showed  a  change 
in  teaching  behaviors,  according  to  their  treatment 
group;  however,  the  authors  concluded  that  changing 
to  a  CA  instructional  approach  is  no  more  or  less 
difficult  than  changing  to  a  direct  instruction 
approach.  Because  both  approaches  were  founded  in 
constructivist  theory,  the  authors  further  concluded 
that  their  study  shows  that  it  is  possible  to  “translate 
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new  theoretical  insights  in  learning  and  instruction  into 
regular  school  practices”  (de  Jager  et  al.,  2002,  p.  841). 

Cognitive  apprenticeship  environments  also  may 
be  used  to  help  train  preservice  teachers.  Liu  (2005) 
studied  the  effects  of  a  Web-based  CA  learning  envi¬ 
ronment  in  preservice  teaching  education.  Compared 
with  a  traditional  classroom  approach,  the  Web-based 
CA  approach  resulted  in  better  performance  and  atti¬ 
tudes  toward  instructional  planning. 

Others  have  looked  at  using  the  CA  model  in  fields 
such  as  instructional  technology  (Darabi,  2005),  nurs¬ 
ing  (Cope  et  al.,  2000),  chemistry  (Stewart  and 
Lagowski,  2003),  and  engineering.  Also  studied  is  how 
CA  impacts  higher  education  teaching  practices  in 
general.  Hendricks  (2001)  conducted  an  experimental 
study  to  determine  whether  CA  was  more  likely  to 
result  in  transferable  knowledge  than  traditional 
instruction  and  found  that  the  treatment  group  had 
greater  post-test  gains  but  did  not  perform  significantly 
better  on  a  transfer  activity  two  weeks  later. 

Cognitive  Apprenticeship  in  K-12  Education 

Teachers  are  being  trained  in  and  via  CA  learning 
environments  and  are  conversely  creating  CA  learning 
environments  for  their  students.  How  do  the  students 
perceive  these  environments  and  do  they  benefit  from 
them?  Tsai  (2005)  developed  and  validated  a  question¬ 
naire  that  was  then  used  to  determine  student  attitudes 
toward  a  computer-based  science  instruction;  one  of 
the  scales  asked  about  cognitive  apprenticeship. 
Among  other  things,  students  who  took  the  survey 
indicated  that  they  preferred  learning  environments 
that  connected  concepts  and  reality.  Considering  stu¬ 
dent  epistemological  beliefs  and  learning  preferences, 
Tsai  pointed  out,  can  be  a  useful  and  fundamental  step 
when  designing  an  instructional  environment  for  a  spe¬ 
cific  group  of  learners.  Teong  (2003)  did  not  use  infor¬ 
mation  about  learners’  preferences  in  the  intervention 
for  his  study;  instead,  the  study  examined  the  effect  of 
metacognitive  training  using  a  word-problem-solving 
strategy,  CRIME,  on  the  experimental  group’s  work 
with  a  CA-based  instructional  software,  WordMath. 
The  experimental  group,  which  received  metacognitive 
training,  outperformed  the  other  students  in  word- 
problem-solving  skills  in  terms  of  both  timing  and 
quality  of  decisions. 


RESEARCH  ON  MENTORING 

The  word  mentoring  often  brings  to  mind  formal  pro¬ 
grams  in  which  a  more  experienced  practitioner  is 
paired  with  a  less  experienced  one  to  provide  guidance 


in  a  new  career  or  environment.  Mentoring  programs 
and  tips  on  how  to  create  and  engage  in  them  are  fairly 
common,  with  published  empirical  reviews  of  them 
being  much  less  common.  The  study  of  such  programs 
and  their  effectiveness  may  well  occur  more  often  than 
is  published,  via  internal  or  informal  evaluations. 
Additionally,  studies  have  been  conducted  to  examine 
different  mentoring  practices  or  strategies  as  well  as 
the  use  of  technologies  to  support  mentoring. 

Formal  Mentoring  Programs 

The  results  of  a  review  of  ten  evaluations  of  youth 
mentoring  programs  (Jekielek  et  al.,  2002)  found  that 
their  impact  was  felt  in  multiple  areas,  including  aca¬ 
demic  achievement  (in  terms  of  attendance,  attitudes, 
and  continuing  education,  although  not  necessarily 
grades);  health  and  safety  (in  terms  of  preventing  and 
reducing  negative  behaviors);  and  social  and  emotional 
development.  Productive  mentoring  practices  were 
found  to  be  structure,  regular  meetings,  mentor  train¬ 
ing  and  preparation,  and  a  focus  on  the  mentees’  needs 
rather  than  the  mentors’  expectations. 

Lucas  (2001)  studied  an  after-school  mentoring 
program  for  sixth- grade  students.  Mentors  were  col¬ 
lege  undergraduates  who  were  enrolled  in  a  for-credit 
course,  and  mentee  participants  were  volunteers  who 
were  promised  support  for  academic  achievement. 
Lucas  found  that  the  relationship  between  mentor  and 
mentee  is  heavily  based  on  individual  factors,  includ¬ 
ing  personal  preferences,  prior  experiences,  and  goals 
and  expectations;  essentially,  the  nature  of  the  experi¬ 
ence  transcends  any  traditional  definition  or  training 
that  may  take  place  and  is  heavily  shaped  by  the  indi¬ 
viduals  who  are  involved  in  it.  Lucas  also  found  a 
much  greater  desire  to  engage  in  mentor-mentee  inter¬ 
action  when  it  was  focused  around  an  activity  that  the 
mentee  could  not  successfully  complete  alone. 

Langer  (2001),  in  his  study  of  the  nature  of  man¬ 
datory  mentoring  at  SUNY  Empire  State  College 
(ESC),  found  a  gap  between  his  results  and  the  pre¬ 
dominant  views  in  the  theoretical  literature  about  men¬ 
toring.  Although  the  literature  base  tends  to  place  a 
heavy  emphasis  on  the  close  interpersonal  relation¬ 
ships  developed  between  mentors  and  mentees,  Langer 
in  contrast  observed  a  process  that  was  almost  exclu¬ 
sively  focused  on  goal  attainment.  What  Langer  and 
ESC  are  referring  to  as  mentoring  might  better  fit  the 
definition  of  coaching,  which  is  more  task  focused  than 
relationship  focused. 

Billet  (2000)  studied  the  learning  process  of  men¬ 
tees  in  a  formal  workplace  mentoring  program  over  a 
six-month  period.  This  prolonged  engagement  allowed 
him  to  identify  learning  sources  and  strategies  that 
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were  influential  on  the  mentees’  development.  Mentors 
were  trained  in  workshops  that  introduced  guided 
learning  strategies  such  as  questioning,  modeling,  and 
coaching  and  helped  them  to  identify  ways  in  which 
these  strategies  might  be  used  in  their  workplace. 
Engagement  in  everyday  work  was  found  to  have  the 
greatest  influence  on  mentee  development,  supporting 
the  concept  of  situated  cognition,  and  Billet  suggests 
that  the  guided  learning  strategies  were  used  to  enhance 
this  engagement.  Questioning,  modeling,  and  coaching 
were  perceived  as  most  useful.  Less  used  strategies, 
such  as  diagrams  and  analogies,  were  less  valued. 

Young  and  Perrewe  (2000)  looked  at  career  and 
social  support  factors  and  their  effects  on  participant 
perceptions  of  the  success  of  a  mentoring  relationship, 
finding  that  mentors’  expectations  generally  were  met 
when  a  protege  (mentee)  was  involved  in  career  sup¬ 
port  behavior.  Conversely,  proteges  tended  to  measure 
the  success  of  their  mentoring  relationship  in  terms  of 
the  amount  of  social  support  they  received.  Young  and 
Perrewe  hypothesized  that  this  difference  in  perception 
may  be  due  to  the  mentors’  established  status,  which 
may  have  them  focused  on  successes  directly  related 
to  the  mentoring  goal  (career  enhancement),  while 
their  more  novice  proteges  may  not  yet  be  able  to 
predict  the  impact  of  particular  career-related  behav¬ 
iors  but  will  look  for  encouragement  and  friendship  as 
indicators  that  they  are  performing  as  expected. 

Bonnett  et  al.  (2006)  studied  20  mentor-protege 
pairings  of  research  scientists  and  university-level 
biology  students  who  used  an  electronic  mentoring 
program.  The  more  effective  pairs  were  found  to  have 
been  more  prolific  and  structured  in  their  posting  and 
to  have  focused  more  on  topics  than  relationship  man¬ 
agement  issues. 

Hudson  et  al.  (2005)  created  and  validated  an 
instrument  based  on  the  literature  in  primary  science 
teaching  by  selecting  five  factors  that  seemed  related 
to  mentoring  effectiveness  (personal  attributes,  system 
requirements,  pedagogical  knowledge,  modeling,  and 
feedback).  This  instrument,  called  Mentoring  for 
Effective  Primary  Science  Teaching  (MEPST),  is 
intended  to  assess  mentee  perceptions  of  their  mentors 
for  their  intern  or  practicum  experiences. 

The  Internet  has  encouraged  the  exploration  of 
mentoring  in  environments  where  mentors  and  learners 
are  not  colocated.  A  series  of  studies  investigated  the 
effects  of  online  mentoring  of  preservice  teachers  in  a 
project  called  Conference  on  the  Web  (COW),  which 
spanned  multiple  years  and  involved  collaborations 
from  faculty  and  preservice  teachers  at  other  schools 
and  universities  internationally  (Bonk  et  al.,  2000, 
2001a,b).  Post-class  surveys  and  interviews  indicated 
that  the  students  valued  the  mentoring  they  received 


and  felt  that  the  computer-mediated  forum  was  an 
appropriate  outlet.  The  quality  of  student  reflection 
was  not  as  high  as  it  might  be,  and  further  work  is 
needed  to  develop  better  scaffolding  and  mentoring 
strategies  for  use  in  online  environments. 

Mentoring  Strategies 

Integrative  teaching  is  one  mentor  strategy  that  may  be 
used.  In  this  strategy,  the  mentor  combines  theory  and 
practice  in  their  explanation  to  the  mentee.  Hayward  et 
al.  (2001)  found  that  most  mentors  provided  far  more 
information  than  the  mentees  had  requested.  A  common 
strategy,  used  by  one  third  of  the  mentors,  was  expert 
push ,  in  which  a  mentor  did  not  directly  answer  the 
mentee’s  question  but  instead  returned  questions 
intended  to  help  the  mentee  find  the  correct  answer. 

In  a  qualitative  study  examining  the  effects  of  elec¬ 
tronic  peer  mentoring  in  a  university  physical  therapy 
class,  it  was  found  that  both  mentors  and  mentees 
learned  through  the  process  of  reflection  and  articula¬ 
tion  (Hayward  et  al.,  2001).  Mentees  benefited  from 
the  mentors’  stories  and  experiences  which  made  the 
learning  more  concrete  and  authentic,  and  the  mentors 
reinforced  concepts  already  learned  by  connecting  the¬ 
ory  to  practice.  Also  studying  mentoring  in  a  university 
setting.  Beck  (2004)  found  that  linking  a  writing 
course  to  an  engineering  department’s  course  could 
help  students  better  learn  how  to  write  lab  reports.  In 
this  instance,  the  writing  instructor  provided  mentor¬ 
ship  that  carried  over  to  the  engineering  curriculum. 

Peers  also  may  serve  as  mentors  to  each  other,  with 
learners  in  some  instances  identifying  on  their  own 
both  their  knowledge  gap  (given  their  learning  goals) 
and  peers  who  can  help  them  attain  their  learning 
goals.  Engaging  in  study  groups  and  asking  for  peer 
assistance  is  a  common  practice  in  many  educational 
settings,  as  students  realize  that  their  peers  can  often 
supply  the  learning  assistance  that  they  need.  Loong 
(1998)  studied  the  peer  apprenticeship  that  developed 
between  two  students  engaged  in  a  computer-mediated 
mathematical  task.  Initially,  the  students  had  different 
approaches  and  worked  rather  independently,  with  one 
student  focused  on  mathematical  rules  and  the  other 
focused  more  on  concepts.  Over  time,  however,  the 
rule -focused  student  noticed  that  the  concept-focused 
student’s  expertise  was  needed,  and  he  assigned  him¬ 
self  to  this  peer  in  an  apprentice  role. 

Pear  and  Crone-Todd  (2002)  examined  ways  of 
using  computers  to  provide  feedback  to  college-level 
students  in  a  manner  consistent  with  the  tenets  of  social 
constructivism  in  a  course  that  used  a  teaching  system 
referred  to  as  a  computer-aided  personalized  system 
of  instruction  (CAPSI).  Drawing  on  the  concept  of 
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scaffolding,  course  material  was  arranged  in  manage¬ 
able  units.  A  peer-tutor  model  was  developed  in  which 
more  advanced  learners  provided  feedback  to  their 
classmates  in  an  open-ended  question  practice  test 
environment.  Although  the  findings  of  this  study  show 
that  the  method  works  to  help  ensure  that  students 
receive  a  high  amount  of  feedback  while  keeping  the 
process  manageable  on  instructors,  it  neglects  to  com¬ 
ment  on  the  impact  of  this  intervention  on  the  learning 
process  for  either  the  students  who  received  the  feed¬ 
back  or  the  peer  tutors  who  provided  it. 


RESEARCH  ON  SCAFFOLDING 

Research  on  scaffolding  has  focused  on  how  much  is 
needed,  what  type  is  needed,  and  how  to  best  provide 
it  to  both  individuals  and  groups.  The  term  scaffold 
appears  in  many  studies,  but  it  is  not  always  well 
applied.  Pea  (2004)  argued  that  the  term  has  become 
a  bit  overused,  to  the  point  where  it  has  lost  its  true 
meaning  and  significance.  He  traces  the  term  back  to 
its  origins,  first  published  in  an  article  by  Wood  et  al. 
(1976),  which  rather  tightly  tied  it  back  to  the  concept 
of  zone  of  proximal  development.  A  scaffold  was 
intended  to  be  a  tool  to  help  children  do  something 
they  could  not  do  without  assistance.  Within  this  con¬ 
cept  is  the  notion  that  the  scaffold,  when  no  longer 
needed  (the  ZPD  has  shifted  with  learning),  could  be 
faded.  Pea  (2004)  rightly  noted  that  in  much  of  the 
published  research  we  have  shifted  from  discussing 
scaffold  with  fading  to  a  different  interpretation:  scaf¬ 
fold  for  performance.  In  particular.  Pea  raised  the  issue 
that  many  so-called  software -based  scaffolds  really  are 
intended  as  performance  supports  that  may  never  be 
removed  from  the  learner;  however,  it  is  possible  that 
some  of  the  so-called  scaffold-for-performance  studies 
represent  situations  in  which  fading  might  be  possible 
but  was  outside  the  scope  of  the  study. 

Good  descriptions  of  fading  can  be  found  in  the 
literature  on  reciprocal  teaching  (Brown  and  Palincsar, 
1989;  Palincsar  and  Brown,  1984;  Palincsar  et  al., 
1993;  Rosenshine  and  Meister,  1994).  Fading  was 
studied  explicitly  by  Roehler  and  Cantlon  ( 1997) .  They 
examined  the  use  of  scaffolds  in  two  social  construc¬ 
tivist  classrooms,  exploring  the  types  and  characteris¬ 
tics  of  scaffolding  in  learning  conversations  taking 
place  during  elementary-school  language  instruction. 
Over  time,  students  took  more  responsibility  for  learn¬ 
ing  in  this  environment,  and  the  amount  of  scaffolding 
used  by  the  instructor  faded.  Bean  and  Patel  Stevens 
(2002)  obtained  somewhat  contradictory  results.  In 
their  study  of  how  scaffolding  affects  the  reflection 
process  for  teacher  education  students,  they  found  that 


students’  written  work  followed  the  models  given  as 
a  scaffold  but  did  not  extend  in  any  substantial  way 
beyond  the  scaffold.  The  authors  concluded  that, 
although  scaffolding  had  a  clear  effect,  it  did  not  help 
achieve  all  of  the  instructional  goals;  this  finding  may 
represent  an  inherent  issue  with  scaffolding  (particu¬ 
larly  a  scaffold  for  performance),  or  it  may  be  indica¬ 
tive  of  a  scaffold  that  did  not  fully  meet  the  learners’ 
needs. 

Distributed  Scaffolding  and  ZPD 

A  big  challenge  for  classroom  teachers  is  having  to 
teach  learners  who  all  have  different  zones  of  proximal 
development.  Within  a  class,  the  ZPD  for  many  stu¬ 
dents  may  be  similar,  but  there  likely  are  some  students 
whose  zone  is  quite  different.  Some  researchers  have 
begun  to  examine  how  scaffolding  can  be  flexibly 
designed  to  meet  the  needs  of  diverse  students,  recog¬ 
nizing  that  scaffolding  should  provide  that  extra  sup¬ 
port  learners  need  to  successfully  complete  a  just  out- 
of-reach  task. 

Savery  (1998)  found  evidence  that  learners  do  not 
all  need  the  same  amount  of  scaffolding.  He  noted  that 
instructors  in  a  business  writing  course  made  use  of 
all  six  of  Gallimore  and  Tharp’s  (1990)  forms  of  scaf¬ 
folded  assistance,  although  each  occurred  in  different 
amounts  based  on  student  need.  Instructing,  question¬ 
ing,  modeling,  and  cognitive  structuring  were  part  of 
the  teachers’  interaction  with  the  students.  Feeding 
back  occurred  through  grades  and  comments  on 
assignments.  Finally,  contingency  management  was 
largely  unspoken,  although  it  had  been  designed  into 
the  course  itself  that  students  would  face  repercussions 
for  unproductive  behavior. 

Puntambekar  and  Kolodner  (2005)  studied  stu¬ 
dents  learning  science  by  design.  They  used  a  design 
diary  with  learners  as  a  scaffold  for  their  design-related 
activities.  Their  findings  showed  that  one  form  of  scaf¬ 
folding  may  not  be  sufficient  to  meet  all  learners’ 
needs  at  all  times,  and  thus  recommended  the  concept 
of  distributed  scaffolding.  The  basic  concept  behind 
distributed  scaffolding  is  that  offering  more  support 
and  more  types  of  it  results  in  a  greater  chance  of 
effectively  scaffolding  the  learning  process  for  each 
student  in  a  meaningful  way. 

Building  on  this  idea  that  scaffolds  need  not  be 
limited  to  one  kind  per  instructional  intervention, 
Tabak  (2004)  discussed  how  distributed  scaffolding 
can  be  synergistic  in  nature;  for  example,  students 
might  use  software  programs  with  built-in  scaffolds 
but  also  rely  on  just-in-time  scaffolding  from  their 
instructors.  The  two  forms  of  scaffolds  together  are  a 
more  powerful  learning  support  than  either  on  its  own. 
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Teacher-Provided  Scaffolding  Strategies 

Discourse-based  scaffolding  is  one  form  of  coaching 
that  teachers  tend  to  implicitly  rely  on  in  classroom 
settings  as  they  respond  to  learning  needs.  To  study 
discourse-based  scaffolding,  researchers  typically 
examine  the  interactions  that  occur  between  teachers 
and  learners  and  how  they  support  the  learning  process 
on  different  types  of  projects.  Teacher  scaffolding  may 
seem  like  a  silent  activity  and  thus  not  be  immediately 
observed,  but  it  is  a  constant  for  good  teachers  (Mas¬ 
ters  and  Yelland,  2002).  Through  quiet  monitoring, 
teachers  are  able  to  enter  a  group  and  ask  questions  or 
propose  options  at  just  the  right  time  and  withdraw 
such  supports  when  they  are  no  longer  needed. 

Determining  student  needs  is  a  driving  force  for 
this  research.  Rasku-Puttonen  et  al.  (2003)  found  that 
students  need  extensive  scaffolding  when  working  on 
long-term  problem-based  learning  activities,  as  well  as 
ample  opportunity  for  reflection.  Teacher  flexibility  in 
response  to  learner  self-regulation  also  was  considered 
important.  Tabak  and  Baumgartner  (2004)  examined 
differences  in  the  effectiveness  of  teacher  modeling 
dependent  on  whether  the  teacher  and  students  have  a 
symmetric  ( partnerlike )  or  asymmetric  ( mentorlike ) 
relationship.  Symmetric  and  asymmetric  relationships 
result  in  different  discourse  structures  and  impact  mas¬ 
tery  of  cultural  tools.  They  recommend  a  partner  role 
for  teachers  helping  to  develop  students  identified  as 
people  who  can  work  with  scientific  concepts.  Meyer 
and  Turner  (2002)  found  that  nonscaffolding  class¬ 
room  discourse  (e.g.,  direct  instruction  or  focus  on 
objectives  questions)  is  not  as  effective  as  scaffolded 
discourse  at  helping  students  become  self-regulated 
math  learners. 

Another  way  in  which  students  may  need  scaffold¬ 
ing  assistance  is  task  structuring  (Tharp,  1993),  which 
may  include  activities  such  as  “chunking,  sequencing, 
detailing,  reviewing,  or  any  other  means  to  structure 
the  task  and  its  components  so  as  to  fit  it  into  the 
learner’s  zone  of  proximal  development”  (Sugar  and 
Bonk,  1998,  p.  142).  Supporting  this  theory,  Dennen 
(2000)  found  that  scaffolds  in  the  form  of  chunking 
and  sequencing  tasks  helped  motivate  students  and 
enabled  them  to  focus  more  on  the  content-based 
learning  goals  than  on  project  management  elements 
of  the  assignment.  Although  this  study  looked  at  a  one¬ 
time  project  and  thus  fading  did  not  occur,  in  the  con¬ 
text  of  a  larger  classroom  effort  one  might  fade  such 
scaffolds  during  successive  projects. 

Scaffolding  is  not  limited  to  classroom  situations; 
early  interactions  with  one’s  parents  ideally  provide 
scaffolding  as  a  child  is  guided  through  new  experi¬ 
ences  (Rogoff,  1990).  Neitzel  and  Stright  (2003)  stud¬ 


ied  how  mothers  scaffolded  their  preschool  children’s 
performance  on  problem-solving  tasks  and  then  mea¬ 
sured  the  children’s  self-regulatory  abilities  in  the  kin¬ 
dergarten  classroom.  They  found  that  more  highly  edu¬ 
cated  mothers  were  more  likely  to  scaffold  their 
children’s  work  and  engage  children  in  metacognitive 
discourse,  and  in  turn  these  behaviors  resulted  in  chil¬ 
dren  who  exhibited  higher  rates  of  task  persistence  and 
behavior  control  in  the  classroom. 

Software-Based  Scaffolding 

Software-based  scaffolding  has  been  a  developing 
topic  of  interest  as  educational  software  becomes 
increasingly  sophisticated.  Reiser  (2004)  suggests  that 
software -based  scaffolding  serves  two  major  purposes. 
First,  it  can  be  used  to  help  provide  structure  to  the 
learning  task,  guiding  them  through  the  major  stages 
or  tasks  and  prompting  them  at  appropriate  times.  Sec¬ 
ond,  it  can  be  used  to  create  a  problem  space  in  which 
learners  must  explore  the  content.  These  two  types  of 
scaffolds  may  work  harmoniously  or  may  conflict  with 
each  other.  Software -based  scaffolds  must  be  designed 
in  consideration  of  various  tradeoffs  such  as  level  of 
generality,  learner  control,  and  learner  choice,  with  an 
attempt  to  support  learners  without  stifling  or  over- 
directing  them. 

Shabo  et  al.  (1997)  designed  scaffolding  into 
Graphica,  a  computer-based  environment  focused  on 
graphics  learning.  Graphica  provides  scaffolds  that  are 
built  into  learning  exercises  in  the  form  of  resources 
(hints,  descriptions  of  expert  processes),  coaching 
(computer-based  critiques  of  student  work  that  are 
available  on  demand),  and  articulation  (a  newsgroup, 
the  one  form  of  human-human  interaction  built  into 
the  program).  In  a  formative  evaluation  of  Graphica, 
they  found  that  many  students  were  unsure  of  how  to 
use  its  various  components  to  support  their  learning 
processes.  The  practice  exercises  and  visualization 
components  were  popular,  but  scaffolds  such  as  the 
expert  analyses  and  hints  were  not  heavily  used.  The 
challenge  for  users  of  Graphica  and  similar  programs 
is  that  they  must  have  sufficient  metacognitive  devel¬ 
opment  to  identify  their  own  learning  needs,  and  their 
learning  goals  must  be  inline  with  the  goals  designed 
into  the  system. 

Picking  up  on  this  issue  of  metacognitive  develop¬ 
ment,  Graesser  et  al.  (2005)  designed  computer-based 
learning  environments  to  support  inquiry  and  metacog¬ 
nition.  They  have  been  able  to  develop  pedagogical 
agents  that  both  model  self-explanation  and  coach  stu¬ 
dents  in  metacognitive  strategies,  demonstrating  that 
the  computer  is  a  viable  tool  for  supporting  develop¬ 
ment  of  deeper  levels  of  metacognitive  thinking  and 
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when  explanatory  reasoning  is  involved.  Land  and 
Zembal-Saul  (2003)  similarly  found  that  software- 
based  scaffolds  are  a  useful  support  to  articulation  and 
reflection  processes. 

Davis  and  Linn  (2000)  and  Davis  (2003)  studied 
the  use  of  prompts  to  scaffold  the  reflection  process 
for  middle-school  science  students  working  within  a 
computer-based  system  known  as  the  Knowledge  Inte¬ 
gration  Environment  (KIE),  developed  by  Bell  et  al. 
(1995).  This  system  supports  the  scientific  process  by 
prompting  students  through  related  activities,  such  as 
identifying  the  needed  evidence  to  support  claims  and 
determining  whether  presented  evidence  is  adequate. 
Davis  and  Linn  (2000)  found  in  two  related  studies 
that  reflective  prompts  in  KIE  promoted  knowledge 
integration  in  students  working  on  science  projects. 
They  suggested  that  the  reflective  articulation  that  is 
involved  in  responding  to  self-monitoring  prompts 
helps  students  better  self-assess  their  understanding 
and  thus  engages  them  in  knowledge  integration. 

In  Davis’  2003  study,  students  working  in  pairs 
received  either  generic  prompts  asking  students  to 
share  their  thoughts  at  that  point  in  the  activity  or 
directed  prompts.  Learners  who  received  the  generic 
prompts  were  more  likely  to  develop  a  coherent  under¬ 
standing  of  the  overall  project  in  which  they  were 
participating  than  those  who  received  the  more  heavily 
scaffolded  or  controlled  direct  prompts.  Learner  auton¬ 
omy  was  also  a  factor,  with  autonomous  learners  dem¬ 
onstrating  the  greatest  comprehension  benefits  from 
the  generic  prompts. 

It  is  possible  that  the  directed  prompts,  which  were 
prescripted  and  programmed  into  the  KIE,  were  too 
limiting  or  narrow  for  these  learners  or  did  not  chal¬ 
lenge  them  enough.  It  will  thus  be  interesting  to  see 
the  results  of  recent  research  interests  in  scripting  for 
online  discourse  (Choi  et  al.,  2005;  Jonassen  and 
Remidez,  2005;  Makitalo  et  al.,  2005).  Also,  comput¬ 
ers  are  unable  to  adjust  to  learners’  unique  needs  in  as 
subtle  and  personalized  of  a  manner  as  a  teacher  might, 
making  it  difficult  for  a  program  to  sufficiently  and 
consistently  identify  each  learner’s  zone  of  proximal 
development  (Ainsworth  et  al.,  1998).  This  research 
also  indirectly  supports  the  calls  of  Tabak  (2004)  and 
Puntambekar  and  Kolodner  (2005)  for  the  use  of  dis¬ 
tributed  scaffolding. 

Scaffolding  and  Computer-Supported 
Collaborative  Learning 

Scaffolding  might  be  provided  by  human  interactants 
mediated  by  computers.  This  form  of  scaffolding  dif¬ 
fers  from  software-based  supports  in  that  a  live  person 
uses  computer-based  tools  to  assist  another’s  perfor¬ 


mance.  Learner-centered  strategies  are  important  here 
(Bonk  and  Dennen,  2007),  as  we  move  from  informa¬ 
tion  transmission  models  of  learning  which  tradition¬ 
ally  involve  flat  interactions  with  static  content  in  an 
online  environment.  Scaffolding  has  been  considered 
essential  to  the  development  of  deep  asynchronous 
discussion  (Oliver  and  Herrington,  2000);  however,  in 
an  online  context  the  metaphor  of  scaffolding  is  not 
only  appealing  but  also  elusive  and  problematic 
(McLoughlin,  2002).  Why  is  scaffolding  in  an  online 
environment  so  challenging?  In  part  because  it  raises 
the  question  of  whether  or  not  traditional  roles  of 
teacher  and  learner  will  be  relied  upon.  McLoughlin 
suggested  a  variety  of  technology  interventions  that 
rely  on  scaffolding,  including  Computer-Supported 
Intentional  Learning  Environments  (CSILEs),  which 
are  collaborative  learning  spaces  in  which  the  teacher 
is  a  facilitator  and  the  student  is  tasked  with  commu¬ 
nicating  and  creating  knowledge  objects  (Scardamalia 
and  Bereiter,  1994);  intelligent  tutoring  systems 
(ITSs),  which  help  break  down  and  manage  specific 
tasks;  and  goal-based  scenarios  (GBSs),  which  engage 
students  in  authentic  tasks  and  provide  computer-based 
resources  and  scaffolding  in  the  form  of  task  assistance 
and  hints  as  needed  (Schank  et  al.,  1999).  Other  recent 
studies  building  on  the  CSILE  foundation  have  looked 
at  how  Knowledge  Forum,  a  program  that  offers 
knowledge-building  scaffolds,  impacts  student  learn¬ 
ing  (Bereiter  and  Scardamalia,  2003;  Lax  et  al.,  2004; 
Nason  and  Woodruff,  2003;  Oshima  et  al.,  2003).  Stud¬ 
ies  in  this  area  often  use  a  design-based  research 
method  (see  Chapter  54  in  this  Handbook) . 

Oshima  and  Oshima  (2001)  studied  ways  to 
improve  learning  for  novices  through  the  use  of  dis¬ 
course  scaffolding;  specifically,  the  WebCSILE  tool 
was  used  to  support  their  interactions.  A  comparative 
analysis  of  two  groups’  discourse  showed  that, 
although  students  with  a  comprehension-oriented 
objective  discussed  content  at  the  metacognitive  level, 
those  with  a  synthesis-oriented  one  did  not.  Further, 
the  quality  of  writing  did  not  improve  in  the  group  that 
also  had  a  page  of  writing  tips  and  a  schedule  as 
additional  support.  The  researchers  felt  that  the  support 
in  fact  may  have  in  some  ways  limited  the  interactions 
that  took  place.  Learners  in  the  second  group  used  the 
provided  scaffolding  as  a  directive  for  what  to  do  and 
followed  its  suggestions  quite  literally,  like  a  task  list. 

Guzdial  and  Turns  (2000)  recommend  the  use  of 
anchors,  or  topics  that  students  wish  to  discuss  to  stim¬ 
ulate  interest  and  motivation.  Using  a  Collaborative  and 
Multimedia  Interactive  Learning  Environment  (CaMILE) , 
they  compared  anchored  discussion  to  the  use  of  a 
newsgroup  tool  lacking  CaMILE’s  management,  facil¬ 
itation  and  anchoring  features,  hypothesizing  that  the 
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anchored  threads  would  be  more  effective  (defined  as 
having  broad  participation  and  being  on-topic)  than  the 
unanchored  ones.  In  an  initial  study,  which  looked  at 
participation  across  multiple  classes,  findings  indicated 
that  discussion  threads  in  CaMILE  were  longer  than 
those  in  the  newsgroup,  with  low  variability  of  length 
in  the  newsgroups  but  high  variability  in  CaMILE.  No 
significant  difference  were  observed  between  the  two 
tools  in  terms  of  the  number  of  active  participants.  A 
second  study  focused  on  discussion  within  a  single 
class.  Findings  in  this  study  indicated  that  the  students 
who  used  CaMILE  participated  more  extensively  than 
their  newsgroup  counterparts  and  that  teacher  partici¬ 
pation  was  greater  in  the  number  of  messages  but  less 
in  the  percentage  of  messages. 

RESEARCH  ON  COMMUNITY 
OF  PRACTICE 

Cognitive  apprenticeships  are  a  natural  occurrence 
within  communities  of  practice,  and  the  CoP  model . 
as  pioneered  by  Etienne  Wenger  (Wenger,  1998; 
Wenger  et  al.,  2002),  has  been  promoted  as  a  way  to 
support  professional  learning  during  the  last  decade. 
This  movement  toward  thinking  about  professions  as 
communities  of  practice  has  very  much  paralleled  the 
rethinking  of  organizational  knowledge  and  develop¬ 
ment  of  knowledge  management  strategies.  Of  partic¬ 
ular  interest  to  many  researchers  has  been  the  experi¬ 
ence  of  new  employees  as  they  get  socialized  into  an 
organization.  In  other  words,  are  new  employees  learn¬ 
ing  and  assimilating  by  observing  the  practices  of  their 
more  experienced  peers?  This  research  on  people  on 
peripheral  and  inbound  trajectories  helps  examine  how 
prior  learning  and  initial  learning  within  an  organiza¬ 
tion  tend  to  shape  one’s  experience  and  overall  path 
within  a  community. 

Cognitive  Apprenticeship 
and  Newcomer  Adjustment 

Socialization  was  found  to  be  important  to  newcomer 
adjustment  by  Kammeyer-Mueller  and  Wanberg 
(2003).  This  study,  situated  within  the  organizational 
development  discourse  on  proximal  and  distal  indica¬ 
tors  of  newcomer  adjustment,  collected  data  from  new¬ 
comers  at  seven  different  organizations  four  times  dur¬ 
ing  a  12-month  period.  In  addition  to  socialization, 
both  pre -entry  knowledge  and  proactive  personality 
were  shown  to  be  related  to  positive  adjustment. 

Also  concerned  with  this  critical  point  in  one’s 
membership  in  a  community  of  practice,  Klein  et  al. 
(2006)  examined  the  impact  of  socialization  experi¬ 


ences  occurring  prior  to  and  immediately  upon  the 
hiring  on  194  new  employees  at  an  educational  insti¬ 
tution.  They  found  that  two  factors— realism  of  pre¬ 
entry  knowledge  and  agent  helpfulness  — had  a  positive 
impact  on  job  outcomes  as  measured  through  role  clar¬ 
ity,  satisfaction,  and  commitment  to  the  organization. 

Slaughter  and  Zickar  (2006)  found  that  role  under¬ 
standing,  as  indicated  by  the  two  variables  of  role  con¬ 
flict  and  role  ambiguity,  impacts  how  newcomers 
become  involved  in  organizational  activities.  They  con¬ 
cluded  that  the  behavior  of  community  insiders  influ¬ 
ences  the  attitudes  of  the  newcomers,  thus  it  matters 
with  whom  one  interacts  upon  entry  into  a  community. 
Their  study  was  conducted  within  a  university  depart¬ 
ment,  and  they  also  found  that  graduate  student  activi¬ 
ties  or  lack  thereof  in  a  department  also  may  be  indic¬ 
ative  of  different  community  alignments;  in  other  words, 
some  students  may  engage  in  activities  that  would  show 
their  commitment  to  their  anticipated  career  more  so 
than  to  their  department,  knowing  that  commitment  to 
the  department  will  not  necessarily  have  career  rewards. 

In  a  study  of  a  community  of  writers  at  an  urban 
nonprofit  organization,  Beaufort  (2000)  explored  the 
roles  the  writers  played  and  how  new  writers  were 
integrated  into  the  community  following  an  appren¬ 
ticeship  model.  Fifteen  roles  were  observed  in  this 
example,  ranging  from  observer,  reader/researcher, 
and  clerical  assistant  on  the  novice  end  up  to  author, 
inventor,  and  coach  on  the  expert  end.  New  or  less 
experienced  writers  learned  the  process  through  taking 
on  roles  such  as  the  clerical  assistant  (a  role  reserved 
for  new  members),  which  allowed  for  extended  obser¬ 
vation  of  the  expert  writers  at  work.  The  results  suggest 
that  learning  writing  through  a  social  process  with 
authentic  tasks  is  effective,  and  the  researcher  stated 
that  a  similar  model  may  be  useful  in  school  settings, 
where  writing  has  traditionally  been  an  individual, 
general- skills  learning  activity. 

Research  on  Identity  Development 

Identity  development— whether  or  not  one  immedi¬ 
ately  relates  to  a  particular  community  of  practice  — 
also  has  been  of  interest  to  researchers.  Identity  issues 
were  raised  by  Klein  and  colleagues  (2006),  as  those 
who  more  readily  understood  the  organization  and 
were  better  able  to  identify  their  own  role  within  it; 
however,  communities  of  practice  do  not  inherently 
transmit  a  sense  of  identity  to  those  who  are  peripheral 
or  on  an  inbound  path.  Davis  (2006)  found  that  the 
professional  relationships  one  builds  with  others  in  the 
CoP  and  particularly  with  those  in  a  supervisory  capac¬ 
ity  will  impact  trajectory  and  identity  development 
among  occupational  therapists.  Cope  et  al.  (2000) 
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found  that  the  nursing  community  of  practice  readily 
accepted  student  members  into  the  community,  but 
their  professional  acceptance  was  dependent  on  dis¬ 
played  competence.  Thus,  it  seems  that  identity  and 
acceptance  are  related,  but  other  factors  may  be 
involved  in  becoming  a  successful  practicing  member 
of  a  profession. 

Varelas  et  al.  (2005)  studied  the  relationship 
between  identifying  oneself  as  a  scientist  and  as  a 
science  teacher  in  a  population  of  new  teachers.  They 
found  that  these  new  teachers  were  identifying  as  sci¬ 
entists  when  engaging  students  in  science  learning 
activities  and  that  they  were  drawing  upon  instruc¬ 
tional  strategies,  such  as  mentoring,  modeling,  and 
articulation,  that  are  part  of  the  cognitive  apprentice¬ 
ship  model.  As  time  passed,  these  teachers  identified 
more  as  science  teachers  and  tried  to  create  a  commu¬ 
nity  of  scientists  in  their  classrooms. 

Research  on  Community  Interactions 

Communities  of  practice  also  are  often  mentioned  in 
relation  to  teachers  and  their  professional  develop¬ 
ment.  New  teachers  tend  to  learn  much  from  their 
interactions  with  more  experienced  teachers,  including 
how  to  engage  in  teaching-related  discourse  (Smith, 
2005);  however.  Smith  found  that,  despite  learning 
taking  place,  relationships  between  parties  on  different 
trajectories  may  not  always  be  tension-free  because  of 
different  personal  needs  and  objectives. 

Distributed  scaffolding  (also  discussed  in  the  Scaf¬ 
folding  Research  section)  is  one  way  of  addressing  the 
different  needs  of  a  group  of  learners,  but  one  also 
might  bring  the  learners  together  as  a  community  with 
a  common  goal,  all  working  jointly  within  a  ZPD. 
Goos  et  al.  (2002)  looked  at  how  a  collaborative  ZPD 
might  be  developed  among  learners  working  on 
inquiry-based  projects  in  a  math  community.  They 
coded  learners’  interactions  as  reading,  understanding, 
analysis,  exploration,  planning,  implementation,  or 
verification  (for  examples  of  the  coding  scheme,  see 
Artzt  and  Armour-Thomas,  1992;  Schoenfeld,  1992), 
as  well  as  by  metacognitive  act.  Their  findings  indicate 
that  the  social  interactions  of  learners  working  together 
can  lead  to  a  collaborative  ZPD. 

Research  on  communities  of  practice  need  not  be 
limited  to  work  or  school  settings.  Merriam  et  al. 
(2003)  studied  how  informal  learning  takes  place  via 
social  interactions  in  a  community  of  practicing 
witches.  Through  talking  to  representatives  of  different 
covens,  they  found  that  membership  in  these  groups 
very  much  fit  Wenger’s  (1998)  community  of  practice 
trajectory  and  that  both  formal  and  informal  situated 
learning  was  prevalent. 


SUMMARY  OF  COGNITIVE 
APPRENTICESHIP  RESEARCH 

As  can  be  seen  in  this  review  of  theory  and  research 
on  the  cognitive  apprenticeship  model,  the  professional 
dialog  spans  diverse  fields  of  study,  learner  groups,  and 
settings.  Empirical  studies  have  confirmed  much  of 
what  theories  have  suggested:  (1)  that  the  cognitive 
apprenticeship  model  is  an  accurate  description  of  how 
learning  occurs  naturally  as  part  of  everyday  life  and 
social  interactions,  and  (2)  that  the  instructional  strat¬ 
egies  that  have  been  extracted  from  these  observations 
of  everyday  life  can  be  designed  into  more  formal 
learning  contexts  with  positive  effect.  On  the  whole, 
however,  the  research  is  still  fragmented,  with  bits  and 
pieces  situated  in  different  subfields  of  educational 
research  (e.g.,  teacher  education,  multimedia-based 
education,  adult  education).  Although  many  of  the 
studies  point  back  to  Collins  et  al.  (1989)  as  a  frame¬ 
work,  few  refer  to  each  other.  In  part  this  may  be  due 
to  the  recency  of  this  work  and  publication  cycles. 

FUTURE  STEPS  IN 
RESEARCH  AND  PRACTICE 

Two  areas  in  which  future  research  on  cognitive 
apprenticeships  may  be  particularly  valuable  are  the 
design  of  communities  of  practice  and  technology- 
based  learning  programs.  The  growing  popularity  of 
situated  learning  and  the  desire  to  create  learning  com¬ 
munities  to  support  professional  development  and 
organizational  knowledge  management  have  spurred 
the  intentional  design  of  communities  of  practice. 
Questions  remain  about  how  these  communities  are 
best  designed  and  implemented  or  if  they  even  can  be 
purposely  created  as  opposed  to  naturally  evolved. 

The  potential  impact  of  computer  technologies  on 
cognitive  apprenticeships  has  been  explored  with  two 
main  purposes  in  mind:  using  computers  to  provide 
learning  support  and  using  computers  to  support  learn¬ 
ing-focused  discourse.  In  the  case  of  the  former,  the 
challenges  to  researchers  and  developers  are  twofold: 
(1)  to  develop  guiding  principles  of  providing  com¬ 
puter-supported  cognitive  apprenticeships  that  will 
work  across  proprietary  software  products,  and  (2)  to 
develop  programs  that  are  sufficiently  able  to  address 
learners’  individual  needs  and  provide  appropriate  sup¬ 
ports  at  the  right  moments.  In  the  latter  example,  the 
literature  on  distance  learning  and  online  discourse, 
although  not  explicitly  focused  on  cognitive  appren¬ 
ticeship,  may  provide  a  good  start  for  examining  how 
to  engage  in  modeling  and  coaching  and  how  to 
encourage  articulation,  reflection,  and  exploration  in 
computer- mediated  learning  environments. 
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Finally,  as  research  on  the  cognitive  apprenticeship 
model  matures,  it  would  be  helpful  to  see  a  more 
systematic  and  integrated  program  of  studies  working 
toward  the  development  of  guiding  principles  to  sup¬ 
port  instructional  design,  teaching,  and  learning  based 
on  this  model. 
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ABSTRACT 

Whole-task  models  support  the  development  of  educa¬ 
tional  programs  for  students  who  need  to  learn  and 
transfer  professional  competences  or  complex  cognitive 
skills  to  an  increasingly  varied  set  of  real-world  contexts 
and  settings.  They  are  a  reaction  to  traditional  atomistic 


approaches  in  which  complex  contents  and  tasks  are 
reduced  into  increasingly  simpler  elements  until  reach¬ 
ing  a  level  where  the  distinct  elements  can  be  transferred 
to  the  learners  through  presentation  or  practice.  These 
approaches  work  well  if  there  are  few  interactions 
between  the  elements,  but  they  do  not  work  well  if  the 
elements  are  interrelated  because  the  whole  is  then  more 
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than  the  sum  of  its  parts.  Whole-task  models  basically 
try  to  deal  with  complexity  without  losing  sight  of  the 
relationships  between  elements.  This  chapter  briefly 
discusses  the  history  of  whole-task  models.  They  are 
rooted  in  motor  learning  and  sports,  andragogy  and 
adult  learning,  and  Gestalt  psychology.  The  character¬ 
istics  of  whole-task  models  in  the  field  of  educational 
communications  and  technology  are  also  discussed. 
Elaboration  theory,  goal-based  scenarios,  and  four- 
component  instructional  design  are  presented  as  three 
representative  examples  of  whole-task  models.  We 
present  empirical  evidence  for  the  effectiveness  of  the 
whole-task  approach  and  the  three  example  models. 
We  conclude  with  a  summary  of  findings  and  directions 
for  future  research  on  whole-task  models. 


KEYWORDS 

Competence  development:  A  feature  of  the  holistic 
approach,  indicating  that  educational  programs 
should  be  aimed  at  the  development  of  compe¬ 
tences  rather  than  teaching  different  topics  in  dif¬ 
ferent  courses. 

Integrated  curriculum:  A  curriculum  based  on  a 
whole-task  approach  aimed  at  the  integration  of 
supportive  contents  with  whole  tasks,  knowledge, 
skills,  and  attitudes  and  at  integrating  first-order 
skills  with  higher-order  skills. 

Mathemagenic  methods:  Instructional  methods  that 
explicitly  aim  at  the  transfer  of  learning;  these 
methods  encourage  learners  to  invest  effort  and 
time  in  the  development  of  general  or  abstract  cog¬ 
nitive  schemas. 

Part-task  models:  Instructional  models  that  apply  an 
atomistic  approach  in  which  complex  contents  and 
tasks  are  reduced  into  increasingly  simpler  ele¬ 
ments  until  reaching  a  level  where  the  distinct  ele¬ 
ments  can  be  taught  to  the  learners. 

Whole-task  models:  Instructional  models  that  apply  a 
holistic  approach  in  which  complex  contents  and 
tasks  are  analyzed  in  coherence  and  taught  from 
their  simplest,  yet  still  meaningful,  version  toward 
increasingly  more  complex  versions. 


There  are  contexts  in  which  what  is  happening  in  the 
whole  cannot  be  deduced  from  the  characteristics  of  the 
separate  pieces,  but  conversely  what  happens  to  a  part  of 
the  whole  is,  in  clearcut  cases,  determined  by  the  laws 
of  the  inner  structure  of  its  whole.  (Max  Wertheimer, 
1925) 


INTRODUCTION 

In  the  21st  century,  there  has  been  a  growing  interest 
in  whole-task  models  of  learning  and  instructional 
design.  In  dealing  with  the  learning  of  highly  complex 
contents  and  tasks,  whole-task  models  provide  an 
alternative  for  atomistic ,  part-task  models .  To  deal  with 
complexity,  atomistic  models  analyze  a  learning 
domain  into  smaller  pieces  and  then  teach  the  domain 
piece  by  piece.  Whole-task  models,  in  contrast,  ana¬ 
lyze  a  learning  domain  as  a  coherent,  interconnected 
whole  and  then  teach  it  from  very  simple,  yet  mean¬ 
ingful  wholes  that  are  representative  for  the  whole 
domain  to  increasingly  more  complex  wholes.  They 
aim  to  solve  three  basic  problems  in  educa¬ 
tion—  namely,  fragmentation,  indicating  that  students 
are  often  not  able  to  combine  the  many  pieces  they 
have  learned  into  coherent  wholes;  compartmentaliza- 
tion,  indicating  that  students  have  difficulties  integrat¬ 
ing  acquired  knowledge,  skills,  and  attitudes;  and  low 
transfer  of  learning,  indicating  that  learners  are  often 
not  able  to  apply  what  they  have  learned  to  new  prob¬ 
lems  and  new  situations  (van  Merrienboer,  2006). 

The  game  of  tennis  may  serve  as  an  example  to 
contrast  a  whole-task  with  a  part-task  approach  (Gall- 
wey,  1974).  In  a  part-task  approach,  practice  divides 
the  activity  into  pieces  that  are  not  complete  in  them¬ 
selves;  the  instructor  focuses  on  teaching  the  student 
isolated  components  of  good  tennis  playing  such  as 
the  grip,  the  stance,  and  the  swing.  A  holistic  approach, 
in  contrast,  works  with  a  complete  pattern  of  activity; 
that  is,  practice  takes  place  in  the  context  of  the  whole 
task,  starting  simply  and  gradually  growing  in  com¬ 
plexity  toward  the  activity  being  trained.  As  described 
by  Strauch  (1984),  it  might  start  with  the  student  on 
one  side  of  the  net  holding  a  racket  and  the  instructor 
on  the  other  side  of  the  net.  The  student’s  first  task 
might  be  to  say  “toss”  to  signal  the  instructor  to  throw 
the  ball  and  to  say  “bounce”  when  the  ball  bounces  in 
the  student’s  court.  The  second  task  might  be  to  say 
“toss,”  after  which  the  instructor  begins  a  toss  but  does 
not  complete  it;  the  student  says  “toss  ...  bounce” 
anyway,  anticipating  continuation  of  the  past  pattern. 
The  third  task  might  be  to  say  “toss”  to  signal  the 
instructor  to  throw  the  ball,  to  say  “bounce”  when  the 
ball  bounces,  and  to  say  “hit”  when  the  student  would 
hit  the  ball  but  without  making  an  attempt  to  actually 
hit  it,  and  so  on. 

This  chapter  first  briefly  discusses  the  history  of 
whole-task  models.  They  are  rooted  in  motor  learning 
and  sports  but  also  in  andragogy  and  adult  learning, 
as  well  as  in  Gestalt  psychology.  Second,  the  charac¬ 
teristics  of  whole-task  models  in  the  field  of  educa¬ 
tional  communications  and  technology,  which  only 
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became  popular  in  the  early  1990s,  are  discussed. 
Third,  elaboration  theory,  goal-based  scenarios,  and 
four-component  instructional  design  are  presented  as 
three  good  examples  of  whole-task  models.  The  fourth 
section  provides  empirical  evidence  for  the  effective¬ 
ness  of  the  whole-task  approach  and  three  example 
models.  In  the  fifth  and  final  section,  conclusions  are 
made  and  directions  for  future  research  on  whole-task 
models  are  suggested. 

BRIEF  HISTORY  OF 
WHOLE-TASK  MODEFS 

In  the  field  of  educational  communications  and  tech¬ 
nology,  interest  in  whole-task  models  only  became 
evident  in  the  late  1980s  and  early  1990s,  but  it  has  a 
much  older  history  that  is  rooted,  among  others,  in 
motor  learning,  andragogy,  and  psychology.  In  motor 
learning  and  sports,  there  is  a  history  of  comparing 
part-task  and  whole-task  sequencing  techniques  for 
practice.  In  andragogy  and  adult  learning,  there  is  a 
history  of  holistic  education  and  approaching  the  stu¬ 
dent  as  a  whole  person  rather  than  as  a  learner  per  se . 
And,  in  psychology,  an  atomistic  approach  to  the  trans¬ 
fer  of  learning  rooted  in  associationism  has  competed 
for  a  long  time  with  a  holistic  approach  rooted  in 
German  Gestalt  psychology. 

Motor  Learning  and  Sports 

The  field  of  (psycho)motor  learning  has  a  long-lasting 
tradition  of  comparing  whole-task  and  part-task 
approaches  to  sequence  training  (for  reviews,  see 
Schmidt,  1991;  Wightman  and  Lintern,  1985).  In  a 
whole-task  sequence,  the  learner  is  taught  whole 
meaningful  tasks  requiring  the  simultaneous  coordina¬ 
tion  of  component  skills,  and  the  tasks  become  more 
and  more  complex  during  the  training.  In  a  part-task 
sequence,  in  contrast,  the  learner  is  taught  only  one  or 
a  very  limited  number  of  isolated  component  skills 
simultaneously,  and  new  component  skills  to  practice 
are  gradually  added.  If  a  skill  consists  of  the  compo¬ 
nent  skills  A,  B,  and  C,  a  further  distinction  can  be 
made  between  a  part-task  sequence  with  forward 
chaining  (practice  A,  then  B,  then  C);  a  part-task 
sequence  with  backward  chaining  (practice  C  given 
the  results  of  AB,  then  B  given  the  results  of  A,  then 
A);  a  part-task  sequence  with  forward  chaining  and 
snowballing  (practice  A,  then  AB,  then  ABC);  and  a 
part-task  sequence  with  backward  chaining  and  snow¬ 
balling  (practice  C  given  the  results  of  AB,  then  BC 
given  the  results  of  A,  then  ABC).  For  example,  if  A, 
B,  and  C  refer  to  driving  off,  maneuvering,  and  park¬ 


ing,  the  training  schedules  involve,  in  this  order:  (1) 
driving  off,  maneuvering,  and  parking;  (2)  parking 
after  the  instructor  does  the  driving  off  and  maneuver¬ 
ing,  maneuvering  after  the  instructor  does  the  driving 
off,  and  driving  off;  (3)  driving  off,  driving  off  and 
maneuvering,  and  driving  off  and  maneuvering  and 
parking;  and,  finally,  (4)  parking  after  the  instructor 
does  the  driving  off  and  maneuvering,  maneuvering 
and  parking  after  the  instructor  does  the  driving  off, 
and,  finally,  driving  off,  maneuvering,  and  parking. 

Furthermore,  whole-task  and  part-task  sequencing 
may  be  combined  in  two  ways  — namely,  whole-part 
sequencing  and  part- whole  sequencing.  In  whole-part 
sequencing,  a  sequence  of  simple  to  complex  versions 
of  whole  tasks  is  developed  first.  If  it  turns  out  that 
the  first  whole  task  is  still  too  difficult  to  start  the 
training  with,  part-task  sequencing  techniques  are  used 
to  divide  this  whole  task  and,  if  desired,  subsequent 
whole  tasks  in  parts.  A  whole-task  sequence,  for  exam¬ 
ple,  may  pertain  to  driving  a  car  in  a  training  area,  in 
a  rural  area,  and  in  a  city.  If  driving  the  car  in  the 
training  area  is  still  regarded  as  too  difficult  to  start 
the  training  with,  one  might  start  with  the  component 
skill  of  driving  off  in  the  training  area.  In  part- whole 
sequencing,  a  sequence  of  parts  is  developed  first.  If 
the  first  part  is  too  difficult  to  start  the  training  with, 
whole-task  sequencing  is  used  to  sequence  this  part 
from  simple  to  complex.  For  example,  if  the  part-task 
sequence  relates  to  driving  off,  maneuvering,  and  park¬ 
ing,  and  driving  off  is  still  regarded  as  too  difficult  to 
start  the  training  with,  one  might  start  with  driving  off 
an  automatic  car  and  only  then  a  stick-shift  car. 

Already  in  the  1960s,  Briggs  and  Naylor  (Briggs 
and  Naylor,  1962;  Naylor  and  Briggs,  1963)  found  that 
part-task  and  part-whole  sequencing  are  most  suitable 
for  complex  skills  if  little  coordination  of  component 
skills  is  required  (i.e.,  low  task  organization)  and  if  each 
of  the  separate  component  skills  is  already  complex  of 
itself  (i.e.,  high  task  complexity).  But,  for  tasks  with 
high  task  organization,  whole-task  and  whole-part 
approaches  are  typically  more  effective.  This  finding  is 
not  only  true  for  complex  motor  skills  but  also  for  many 
professional  real-life  tasks.  Since  the  1960s,  over¬ 
whelming  evidence  has  been  obtained  showing  that 
breaking  a  complex  task  down  into  a  set  of  distinct  parts 
and  then  teaching  or  training  those  parts  without  taking 
their  interactions  and  required  coordination  into  account 
does  not  work  because  learners  ultimately  are  not  able 
to  integrate  and  coordinate  the  separate  parts  in  transfer 
situations  (Clark  and  Estes,  1999;  Perkins  and  Grotzer, 
1997;  Spector  and  Anderson,  2000).  Performing  a  par¬ 
ticular  component  skill  in  isolation  is  simply  different 
from  performing  it  in  the  context  of  a  whole  task.  It 
seems  to  lead  to  different  mental  representations  (Elio, 
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1986),  and  automaticity  of  a  component  skill  that  is 
developed  as  a  function  of  extensive  part-task  practice 
is  often  not  preserved  in  the  context  of  whole-task  per¬ 
formance  (Schneider  and  Detweiler,  1988). 

Whole-task  models  were  developed  in  sports  and 
in  professions  (Dreyfus,  1982;  Feldenkrais,  1982; 
Strauch,  1984).  One  key  issue  in  those  models  is  how 
to  deal  with  task  complexity.  Most  holistic  approaches 
introduce  some  notion  of  modeling  to  attack  this  prob¬ 
lem.  A  powerful  two-step  approach  to  modeling  first 
develops  simple -to-complex  models  of  reality  or  real- 
life  tasks,  and  then  models  these  models  from  a  ped¬ 
agogical  perspective  to  ensure  that  they  are  presented 
in  such  a  way  that  students  can  actually  learn  from 
them  (Achtenhagen,  2001).  Thus,  in  this  view,  instruc¬ 
tion  should  ideally  begin  with  a  simplified  but  whole 
model  of  real-life  task  performance,  which  is  then 
conveyed  to  the  learners  according  to  sound  pedagog¬ 
ical  principles,  including,  for  example,  the  provision 
of  learner  guidance  and  support. 

Concluding,  part-task  models  have  been  found  to 
be  very  effective  to  reduce  task  difficulty,  but  they 
hinder  integration  of  knowledge,  skills,  and  attitudes 
and  limit  the  opportunities  to  learn  to  coordinate  com¬ 
ponent  skills.  Whole-task  models  are  better  suited  to 
learning  to  coordinate  component  skills  and  are  pre¬ 
ferred  for  tasks  with  a  high  level  of  organization.  To 
deal  with  task  complexity,  simplification  of  the  whole 
task  and  giving  learners  support  and  guidance  are  use¬ 
ful  approaches. 

Andragogy  and  Adult  Learning 

Whereas  researchers  in  the  field  of  motor  learning  and 
sports  stress  the  idea  of  the  whole  task ,  researchers  in 
the  field  of  andragogy  and  adult  learning  mainly  empha¬ 
size  the  idea  of  the  whole  person  in  his  or  her  context. 
Holistic  education  is  defined  as  (Rinke,  1985,  p.  67): 

A  functional,  integrated  and  generalized  model  of  edu¬ 
cation  that  focuses  on  the  whole  teaching-learning  situ¬ 
ation,  and  varies  the  teaching-learning  strategy  to  meet 
the  needs  of  the  learner,  the  teacher  and  the  situation  in 
an  effort  to  attain  educational  outcomes  greater  than  the 
sum  of  their  parts. 

This  approach  is  rooted  in  Holism  (Smuts,  1926)  and 
related  to,  for  example,  the  contingency  approach  in 
management,  which  maintains  that  managers  should 
vary  their  leadership  style  in  accordance  with  the  situ¬ 
ation  to  improve  managerial  effectiveness  (Graen  and 
Hui,  2001)  and  holistic  medicine  (Graham-Pole,  2001), 
which  avoids  the  piecemeal  treatment  of  isolated  symp¬ 
toms  and  regards  the  patient  as  a  whole  person  who  is 
co-responsible  for  his  or  her  own  health  care. 


A  first  characteristic  of  this  holistic  approach  is  the 
focus  on  the  whole  person  and  his  or  her  meaningful, 
situated  behaviors  in  real  settings.  To  make  this  point 
clear,  suppose  you  have  to  undergo  surgery.  Would  you 
prefer  a  surgeon  with  great  technical  skills  but  with  no 
knowledge  of  the  human  body?  Or  would  you  prefer 
a  surgeon  with  great  knowledge  of  the  human  body 
but  with  two  left  hands?  Or,  perhaps  you  would  want 
a  surgeon  with  great  technical  skills  but  who  has  a 
horrible  bedside  manner  and  a  hostile  attitude  toward 
his  patients?  Or,  finally,  would  you  want  a  surgeon 
who  has  all  of  the  knowledge,  skills,  and  attitudes  that 
he  learned  35  years  ago  but  has  not  kept  them  up  to 
date?  These  questions  clearly  indicate  that  it  makes 
little  sense  to  distinguish  domains  of  learning  (e.g., 
conceptual  knowledge,  skills,  attitudes),  as  is  often 
done  in  formal  educational  programs.  In  a  holistic 
approach,  this  compartmentalization  of  behaviors  is 
replaced  by  a  focus  on  whole  and  meaningful  behav¬ 
iors  in  realistic  settings,  and  the  learning  of  distinct 
pieces  of  knowledge  is  replaced  by  a  model  of  personal 
development  and  growth. 

A  second  characteristic  pertains  to  the  co-respon¬ 
sibility  of  the  learner:  The  educator  and  the  learner 
always  work  together  to  approach  each  learning  task 
or  learning  opportunity  with  its  unique  characteristics 
(e.g.,  context,  features  of  the  task,  personality  charac¬ 
teristics)  in  the  best  possible  way.  Because  no  one 
knows  a  learner’s  desires,  needs,  and  capabilities  better 
than  the  learner,  co-responsibility  is  a  prerequisite  to 
maximizing  the  effectiveness  of  the  learning  process. 
Furthermore,  learners  are  also  expected  to  assume 
responsibility  for  becoming  lifelong  learners  who  are 
able  to  realize  their  full  potential.  Thus,  whereas  the 
educator  is  responsible  for  diagnosing  the  learner’s 
current  level  of  competence,  including  readiness  to 
learn  and  degree  of  dependence,  and  for  making  every 
effort  to  move  the  learner  along  the  learning  contin¬ 
uum,  the  holistic  educator  does  not  and  cannot  take 
full  responsibility  for  the  actual  learning  process.  This 
view  is  currently  found  in  several  forms  of  on-demand 
education,  in  which  learners  select  their  own  learning 
tasks  (van  Merrienboer  and  Kirschner,  2007),  and  in 
resource -based  learning,  in  which  learners  are  required 
to  track  their  own  learning  resources  (see  Chapter  40 
in  this  Handbook). 

A  third  and  last  characteristic  pertains  to  the  sys¬ 
temic  character  of  educational  systems,  meaning  that 
the  performance  or  function  of  each  element  directly 
or  indirectly  has  an  impact  on,  or  is  impacted  by,  one 
or  more  of  the  other  elements  in  the  system.  To  deal 
with  that,  a  holistic  approach  takes  an  integrated  per¬ 
spective  and  repeatedly  adds  small  increments  of  inno¬ 
vation  and  uses  multiple  strategies  to  capitalize  on  their 
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synergy.  Poindexter  (2003)  compared  it  to  a  weight- 
loss  program:  People  who  try  one  diet,  one  pill,  or  one 
exercise  usually  do  not  achieve  their  goals.  Only  when 
a  holistic  approach  to  health  is  taken,  including  incre¬ 
mental  changes  in  eating,  drinking,  life  style,  and  exer¬ 
cise,  do  weight  loss  and  fitness  occur.  In  educational 
systems,  the  best  example  of  a  failing  nonholistic 
approach  can  be  found  in  the  use  of  new  instructional 
methods  that  focus  on  deep  processing,  understanding, 
and  higher-order  skills  without  changing  the  assess¬ 
ment  system.  Such  isolated  changes  are  doomed  to 
fail,  because  the  test  is  at  least  as  important  to  deter¬ 
mine  learning  behaviors  as  the  applied  instructional 
methods  (i.e.,  the  tail  wags  the  dog;  see  Pollio  and 
Back,  2000).  To  reach  desired  results,  there  should  be 
constructive  alignment  of  changes  throughout  the 
whole  educational  system  (Biggs,  1996). 

To  conclude,  in  the  field  of  andragogy  and  adult 
learning,  whole-task  models  are  mostly  found  in  a 
holistic  approach.  Apart  from  a  focus  on  whole  tasks, 
this  approach  stresses  development  and  growth  of  the 
whole  person,  co-responsibility  of  the  learner  and  the 
teacher,  and  an  integrated  approach  to  systemic 
change. 

Psychology  and  Transfer  of  Learning 

In  the  field  of  experimental  psychology,  the  distinction 
between  part-task  models  and  whole-task  models 
becomes  most  evident  in  research  on  transfer  of  learn¬ 
ing— that  is,  the  ability  to  apply  what  has  been  learned 
to  solving  new  problems  in  new  situations  (for  reviews, 
see  Adams,  1987;  Annett,  1989;  Annett  and  Sparrow, 
1985;  Ellis,  1965;  Osgood,  1949;  Royer,  1979).  In  the 
beginning  of  the  20th  century,  two  approaches  evolved 
that  are  still  of  importance  to  the  issue  of  transfer  of 
learning:  the  associationist  approach,  which  is  repre¬ 
sentative  of  a  part-task  model,  and  the  Gestalt 
approach,  which  is  representative  of  a  whole-task 
model. 

Within  the  associationist  tradition,  the  identical 
elements  theory  of  Thorndike  and  Woodworth  (1901) 
claimed  that  transfer  from  one  task  to  another  task 
would  only  occur  when  both  tasks  shared  the  same 
parts,  called  identical  elements.  In  general,  it  is 
assumed  that  the  greater  the  number  of  identical  parts, 
the  greater  the  amount  of  transfer.  This  construction 
indicates  that  transfer  is  functionally  related  to  the 
similarity  and  difference  relationships  between  stimuli 
and  responses  in  an  original  task  and  a  transfer  task. 
With  respect  to  the  transfer  of  learning,  one  current 
cognitive  view  tackles  transfer  problems  by  carefully 
analyzing  the  stimulus  and  response  properties  of  the 
learning  elements.  These  stimulus-response  pairs  are 


subsumed  in  the  concept  of  production  systems,  in 
which  the  productions  may  also  be  seen  as  the  identical 
parts.  Transfer  is  predicted  in  so  far  as  the  performance 
of  two  tasks  can  be  described  by  identical  productions 
(Singley  and  Anderson,  1985, 1988).  This  is  clearly  in 
the  tradition  of  the  associationist  approach,  because  it 
is  assumed  that  events  that  share  the  same  parts  will 
be  recognized  by  the  learner  as  being  similar  and  that 
the  responses  learned  for  the  first  event  can  be  trans¬ 
ferred  to  the  second  event. 

Unlike  the  associationists  who  consider  the  con¬ 
cept  of  identical  elements  as  the  determining  factor  for 
transfer  of  training.  Gestalt  psychologists  rely  on  men¬ 
tal  structures  that  act  as  a  “gestalt”— a  whole  that  is 
more  than  the  sum  of  its  parts.  They  consider  the 
thinking  process  as  reorganizing  or  relating  one  aspect 
of  a  problem  situation  to  another,  which  may  result  in 
structural  understanding  (Ash,  1998;  Mandler  and 
Mandler,  1964).  This  involves  restructuring  the  ele¬ 
ments  of  a  whole,  meaningful  problem  situation  in  a 
new  way  so  a  problem  can  be  solved.  Gestalt  psychol¬ 
ogists  hold  that  transfer  from  one  task  to  another  is 
achieved  by  arranging  learning  situations  so  a  learner 
can  gain  insight  into  the  problem  to  be  solved.  This 
type  of  learning  is  thought  to  be  permanent,  and  reor¬ 
ganized  knowledge  may  yield  transfer  to  new  situa¬ 
tions.  Some  current  cognitive  views  also  make  strong 
assumptions  about  the  nature  of  underlying  memory 
representations.  The  central  assumption  is  that  learners 
are  conceptualized  as  active  constructors  of  knowledge 
rather  than  as  passive  recipients  of  information;  they 
actively  seek  to  make  sense  of  the  environment  by 
imposing  structure  and  order  on  stimuli  encountered 
through  direct  perception  and  experience.  Memory  is 
conceptualized  as  a  highly  structured  storage  system 
in  which  information  is  both  stored  and  retrieved  in  a 
systematic  manner.  The  critical  step  in  transfer  is  then 
the  retrieval  and  reorganization  of  a  relevant  cognitive 
schema  when  a  particular  problem  is  encountered. 

It  may  seem  impossible  to  reconcile  a  psycholog¬ 
ical  whole-task  model,  which  explains  transfer  of 
learning  as  a  process  of  interpreting  cognitive  schemas 
to  reorganize  whole  meaningful  problem  situations, 
with  a  part-task  model,  which  explains  transfer  as  a 
process  of  applying  parts  (i.e.,  identical  elements  or 
productions)  that  were  acquired  during  learning  tasks 
to  new  transfer  tasks.  But,  the  fundamental  distinction 
between  controlled  and  automatic  cognitive  processing 
(Schneider  and  Shiffrin,  1977;  Shiffrin  and  Schneider, 
1977)  creates  the  opportunity  to  combine  both 
approaches  in  a  model  that  distinguishes  two  categories 
of  learning  processes:  schema  construction  and  schema 
automation.  For  task  aspects  related  to  problem  solving 
and  reasoning,  the  learner  is  then  confronted  with  a 
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varied  sequence  of  whole-task  situations  promoting 
the  construction  of  general  cognitive  schemas  that 
allow  for  controlled  processing,  such  as  reorganizing 
a  new  situation  in  such  a  way  that  it  can  be  understood 
in  terms  of  the  available  schemas  (whole-task  prac¬ 
tice).  With  regard  to  the  to-be-automated  task  aspects, 
however,  the  learner  may  in  addition  be  confronted 
with  a  repetitive  sequence  of  practice  items  promoting 
the  construction  of  stimulus-response  pairs  or  produc¬ 
tions  that  allow  for  automatic  processing,  such  as  per¬ 
forming  routine  aspects  of  new  transfer  tasks  (part-task 
practice).  The  reader  is  referred  to  van  Merrienboer 
(1997)  for  an  elaborate  discussion. 

WHOLE-TASK  MODELS  IN 
EDUCATIONAL  COMMUNICATIONS 
AND  TECHNOLOGY 

Part-task  models  dominated  the  field  of  educational 
communications  and  technology  until  the  late  1980s. 
From  then  on,  constructivist  views  of  learning  and 
instruction  have  had  a  major  impact  on  the  thoughts 
and  actions  of  many  researchers  in  the  instructional 
design  field.  Within  this  context,  some  researchers 
started  to  work  on  instructional  design  models  building 
on  the  research  traditions  discussed  in  the  previous 
section.  These  models  share  three  characteristics.  First, 
whole  meaningful  tasks  are  seen  as  the  driving  force 
for  learning;  easy-to-difficult  sequencing  techniques 
and  learner  support  and  guidance,  which  may  be  faded 
as  learners  acquire  more  expertise  (i.e.,  scaffolded), 
are  studied  as  methods  to  deal  with  task  complexity. 
Second,  there  is  a  focus  on  the  development  of  the 
whole  person  (i.e.,  learner-centered)  rather  than  the 
acquisition  of  isolated  pieces  of  knowledge,  and  the 
learner  is  co-responsible  for  a  process  of  competence 
development.  Third,  there  is  a  renewed  interest  in  the 
study  of  instructional  methods  that  explicitly  aim  at 
transfer  of  learning. 

The  Integrated  Curriculum 

In  the  field  of  educational  communications  and  tech¬ 
nology,  the  traditional  approach  has  for  a  long  time 
been  very  similar  to  the  atomistic  part-task  approach 
in  motor  learning  and  sports.  Rather  than  breaking 
down  a  complex  learning  domain  into  component 
skills,  it  was  described  in  terms  of  distinct  learning 
goals  or  objectives.  In  a  traditional  objectives-driven 
approach  (Gagne  and  Briggs,  1979;  Landa,  1983; 
Scandura,  1983),  the  learner  is  taught  only  one  or  a 
very  limited  number  of  objectives  at  the  same  time. 
New  objectives  are  gradually  added  to  practice  until 


all  objectives  have  been  treated.  The  basic  assumption 
is  that  the  teaching  of  different  types  of  objectives  (e.g., 
remembering  a  fact,  applying  a  procedure,  understand¬ 
ing  a  concept)  requires  different  instructional  methods, 
which  is  typical  for  outcomes-based  models  of  instruc¬ 
tional  design  (see  Chapter  32  in  this  Handbook).  To 
sequence  the  objectives,  some  kind  of  learning  hierar¬ 
chy  is  often  used  (Gagne,  1968,  1985).  Such  a  hierar¬ 
chy  closely  resembles  a  hierarchy  of  component  skills; 
it  has  the  most  complex  cognitive  skill  at  its  top  and 
all  of  its  prerequisite  skills  below  it.  Sequencing  takes 
place  from  the  base  of  the  hierarchy  to  its  top. 

In  the  early  1990s,  Gagne  and  Merrill  (1990) 
identified  the  need  to  use  learning  goals  that  require 
an  integration  of  multiple  objectives.  They  proposed 
the  term  enterprise  to  denote  a  real-life  learning 
activity  (e.g.,  denoting,  manifesting,  discovering)  in 
which  the  learner  is  engaged  to  reach  such  multiple 
objectives,  and  they  stress  the  importance  of  enter¬ 
prises  to  reach  better  transfer  of  learning.  Their 
enterprises  are  good  examples  of  what  are  referred 
to  as  whole  tasks  in  this  chapter,  because  they  attempt 
to  deal  with  complexity  without  losing  sight  of  the 
separate  elements  and  the  interconnections  between 
those  elements.  Van  Merrienboer  et  al.  (1992)  dis¬ 
cussed  the  requirements  that  had  to  be  met  when 
using  whole  learning  tasks.  They  focus  on  the  impor¬ 
tance  of  sequencing  whole,  meaningful  tasks  from 
simple  to  complex  and  on  the  necessity  to  give  learn¬ 
ers  support  and  guidance  when  they  start  to  work  on 
tasks  at  a  higher  level  of  difficulty. 

Competence  Development 

Atomistic  instructional  design  models  usually  focus 
on  one  particular  domain  of  learning,  such  as  the  cog¬ 
nitive,  the  affective,  or  the  psychomotor  domain.  A 
further  distinction  (e.g.,  in  the  cognitive  domain)  is  the 
differentiation  between  models  for  declarative  learn¬ 
ing ,  with  an  emphasis  on  instructional  methods  for  the 
construction  of  conceptual  knowledge,  and  models  for 
procedural  learning ,  with  an  emphasis  on  methods  for 
the  acquisition  of  procedural  skills  (Andrews  and 
Goodson,  1980).  In  contrast,  a  holistic  approach  aims 
at  the  integration  of  declarative  learning,  procedural 
learning  (including  perceptual  and  psychomotor 
skills),  and  affective  learning  (including  the  predispo¬ 
sition  to  keep  all  of  these  aspects  up  to  date)  and  so 
facilitates  the  development  of  an  integrated  knowledge 
base  that  increases  the  chance  that  transfer  of  learning 
occurs.  Many  superordinate  terms  that  encompass 
knowledge,  skills,  and  attitudes  have  been  proposed  in 
the  literature,  including  expertise,  complex  skills ,  and 
(professional)  competences. 
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In  current  whole-task  models,  final  attainment  lev¬ 
els  are  often  described  in  terms  of  competences.  On 
the  basis  of  a  comprehensive,  analytical  study  on  the 
concept  of  competence,  van  Merrienboer  et  al.  (2002) 
concluded  that  three  dimensions  are  basic  to  the  use 
of  this  term.  The  first  dimension,  integrativity,  indicates 
that  competence  always  combines  knowledge,  skills, 
and  attitudes  as  well  as  aptitudes  of  the  task  performer. 
The  second  dimension,  specificity ,  indicates  that  a  com¬ 
petence  is  always  bound  to  a  context  that  can  be  highly 
specific  (e.g.,  a  profession)  or  more  general  (e.g.,  a 
career).  The  third  dimension,  durability ,  indicates  that 
a  competence  is  more  or  less  stable  in  spite  of  changes 
in  tools,  working  methods,  and  technologies. 

Whole-task  educational  programs  or  integrated 
curricula  often  aim  at  the  simultaneous  development 
of  first-order,  professional  skills  as  well  as  higher 
order,  general  skills  such  as  self-directed  learn¬ 
ing-including  reflection  on  and  assessment  of  one’s 
own  performance,  self-monitoring  of  task  perfor¬ 
mance,  and  planning  one’s  own  learning  trajectories. 
In  such  programs,  learners  are  able  to  select  their  own 
learning  tasks,  so  each  individual  learner  has  his  or  her 
own  curriculum  rather  than  having  a  one-and-the-same 
curriculum  for  all  learners.  In  such  a  form  of  on- 
demand  education,  support  and  guidance  given  by  the 
teacher  are  not  limited  to  performing  the  learning  tasks 
but  are  also  directed  toward  orienting,  monitoring, 
assessing,  and  planning  the  learning  tasks. 

Mathemagenic  Methods 

A  logical  result  of  using  distinct  learning  objectives  as 
the  basis  for  instructional  design  is  the  application  of 
instructional  methods  that  minimize  the  number  of 
practice  items  required,  the  time  spent  on  the  task,  and 
the  learners’  investment  of  effort  made  to  reach  those 
objectives.  Designing  and  producing  practice  items 
costs  time  and  money,  which  are  often  scarce,  and  the 
learner  does  not  have  unlimited  time  or  motivation  to 
study.  Consider  the  situation  where  students  must  learn 
to  diagnose  three  different  types  of  errors  (el,  e2,  and 
e3)  in  a  complex  technical  system.  If  a  minimum  of 
three  practice  items  is  required  to  learn  to  diagnose 
each  error,  one  may  first  train  the  students  to  diagnose 
error  1 ,  then  to  diagnose  error  2,  and  finally  to  diagnose 
error  3.  This  leads  to  the  following  training  blueprint: 

el,  el,  el,  e2,  e2,  e2,  e3,  e3,  e3 

Although  this  practice  schedule  will  probably  be  most 
efficient  for  reaching  the  three  objectives,  thus  mini¬ 
mizing  the  required  time  on  task  and  student  invest¬ 
ment  of  effort,  it  also  yields  a  low  transfer  of  learning. 


The  reason  for  this  is  that  the  chosen  instructional 
method  invites  students  to  construct  highly  specific 
knowledge  for  diagnosing  each  distinct  error,  which 
only  allows  them  to  perform  in  the  way  specified  in 
the  objectives  but  not  to  show  performances  that  go 
beyond  the  given  objectives.  If  a  designer  is  aiming  at 
a  transfer  of  learning,  and  the  objective  is  to  train 
students  to  diagnose  as  many  errors  as  possible  in  a 
technical  system,  it  is  far  better  to  train  the  students 
to  diagnose  the  three  errors  in  a  random  order.  This 
leads,  for  example,  to  the  following  training  blueprint: 

e3,  e2,  e2,  el,  e3,  e3,  el,  e2,  el 

This  sequence  is  probably  less  efficient  for  reaching 
the  three  isolated  objectives  than  the  previous  one 
because  it  may  increase  the  necessary  time  on  task  or 
the  investment  of  effort  by  the  learners.  It  might  even 
require  four  instead  of  three  practice  items  to  reach  the 
same  level  of  performance  for  each  separate  objective. 
But,  in  the  end  it  yields  higher  transfer  of  learning!  The 
reason  for  this  increase  of  transfer  is  that  this  instruc¬ 
tional  method  invites  students  to  construct  knowledge 
that  is  general  and  abstract  rather  than  entirely  bound 
to  the  three  concrete,  specific  errors.  This  better  allows 
them  to  diagnose  new  not  earlier  encountered  errors. 
This  phenomenon— where  the  methods  that  work  the 
best  for  reaching  isolated,  specific  objectives  are  often 
not  the  methods  that  work  best  for  achieving  integrated 
objectives  and  increasing  the  transfer  of  learning— is 
known  as  the  transfer  paradox  (van  Merrienboer  et  al., 
1997, 2006).  A  whole-task  approach  takes  this  paradox 
into  account  and  is  always  directed  toward  reaching 
multiple,  integrated  objectives  that  go  beyond  a  limited 
list  of  highly  specific  objectives.  Whole-task 
approaches  are  characterized  by  the  use  of  mathema¬ 
genic  instructional  methods  that  are  accompanied  with 
a  germane  cognitive  load  (see  Chapter  31  in  this  Hand¬ 
book)  and  give  rise  to  meaningful  learning  and  transfer. 


EXAMPLES  OF  WHOLE-TASK  MODELS 

Based  on  a  study  of  a  variety  of  modern  design  theories 
and  models,  Merrill  (2002a)  suggested  five  first  prin¬ 
ciples  of  instruction,  stating  that  learning  is  promoted 
when:  (1)  learners  are  engaged  in  solving  real-world 
problems,  (2)  new  knowledge  is  applied  by  the  learner, 
(3)  new  knowledge  is  integrated  into  the  learner’s 
world,  (4)  existing  knowledge  is  activated  as  a  foun¬ 
dation  for  new  knowledge,  and  (5)  new  knowledge  is 
demonstrated  to  the  learner.  The  characteristics  of 
whole-task  models  are  clearly  reflected  in  the  first 
three  principles. 
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TABLE  35.1 

Overview  of  Elaboration  Sequences  in  Elaboration  Theory 


Learning  goal: 
Sequence: 


Conceptual 

Learning  many  related  concepts 
Teach  broader,  inclusive  concepts 
before  narrow,  detailed  concepts 


Theoretical 

Learning  many  related  principles 
Teach  broader,  inclusive  principles 
before  narrow,  detailed  principles 


Procedural 

Learning  procedural  or  heuristic  tasks 
Teach  simpler  versions  of  the  whole 
task  before  complex  versions 


All  Sequences 

Instructional  approach:  Topical  or  spiral  sequencing 

Integrate  knowledge,  skills,  and  attitudes 
Group  wholes  into  learning  episodes 

Give  learners  some  control  over  contents/instructional  method 


The  first  principle  emphasizes  that  students  learn 
better  when  they  are  involved  in  solving  increasingly 
more  complex  real-world  problems.  This  closely 
resembles  a  whole-task  approach  as  well  as  the  idea 
that  tasks  should  be  ordered  from  simple  to  complex, 
while  the  support  and  guidance  given  to  learners 
decrease  as  they  acquire  more  expertise.  The  second 
principle  acknowledges  the  importance  of  applying 
newly  acquired  competences  in  real-life  situations  and 
rehects  the  importance  of  mathemagenic  instructional 
methods  that  facilitate  the  transfer  of  learning.  The 
third  principle  stresses  the  importance  of  integration. 
It  is  in  agreement  with  the  idea  of  competence  growth 
in  an  integrated  curriculum  in  which  learners  are  co- 
responsible  for  their  learning,  as  opposed  to  the  one¬ 
way  teaching  of  distinct  pieces  of  knowledge,  skills, 
and  attitudes. 

In  this  section,  we  briefly  describe  three  example 
models.  It  is  not  our  intention  to  provide  an  exhaustive 
overview  of  models  but  only  to  discuss  a  small  number 
of  models  that  are  representative  of  the  family  of 
whole -task  models.  First,  a  description  is  given  of 
elaboration  theory.  This  forerunner  of  current  whole- 
task  models  stresses  the  notion  that  working  from  sim¬ 
ple  to  complex  is  a  sine  qua  non  for  a  whole-task 
approach.  Second,  a  description  is  given  of  goal-based 
scenarios.  This  theory  focuses  on  the  importance  of 
real-world  applications  and  the  transfer  of  learning. 
Finally,  four-component  instructional  design  is  dis¬ 
cussed  as  an  example  of  a  theory  trying  to  implement 
all  basic  principles  of  the  whole-task  approach. 

Elaboration  Theory 

Reigeluth's  elaboration  theory  (Reigeluth,  1987, 1999; 
Reigeluth  and  Stein,  1983;  Reigeluth  et  al.,  1980;  Van 
Patten  et  al.,  1986)  can  be  seen  as  a  forerunner  of  the 
whole-task  approach  in  educational  communications 


and  technology.  The  basic  principle  of  this  theory  is 
that  instruction  should  be  organized  from  the  simplest 
representation  of  the  learning  domain  or  task  (i.e.,  the 
epitome ,  which  contains  the  most  fundamental  and 
representative  ideas  at  a  concrete  level)  to  increasingly 
more  complex  and  elaborated  representations.  Origi¬ 
nally,  the  theory  focused  on  the  sequencing  of  instruc¬ 
tional  content  in  conceptual  and  theoretical  domains. 

The  conceptual  elaboration  sequence  (see  left  col¬ 
umn  in  Table  35.1)  emphasizes  the  superordinate, 
coordinate,  and  subordinate  relationships  among  con¬ 
cepts.  The  concept  of  dog,  for  example,  is  subordinate 
to  pet,  coordinate  to  cat,  and  superordinate  to  poodle. 
In  a  process  of  conceptual  analysis,  a  conceptual 
knowledge  structure  or  taxonomy  is  made  of  the  learn¬ 
ing  content.  This  structure  is  translated  to  an  elabora¬ 
tion  sequence  in  which  the  broadest,  most  inclusive 
concepts  are  taught  first,  including  the  supporting  con¬ 
tent  (i.e.,  relevant  knowledge,  skills,  and  attitudes) 
related  to  them,  and  subsequently  the  ever  more  nar¬ 
row,  detailed  concepts  are  taught  together  with  related 
supporting  content.  Typical  approaches  to  sequencing 
are  topical  and  spiral.  In  a  topical  approach,  the  content 
is  presented  in  a  vertical  manner;  for  example,  a  stu¬ 
dent  first  studies  the  sequence  pet-dog-poodle,  then 
pet-cat-tabby,  and  so  forth.  A  spiral  approach,  in  con¬ 
trast,  rehects  a  horizontal  method;  for  example,  a  stu¬ 
dent  first  studies  the  sequence  pet-...,  then 
dog-cat-...,  then  poodle-tabby-...  and  so  on. 

The  theoretical  elaboration  sequence  (see  middle 
column  in  Table  35.1)  focuses  on  interrelated  sets  of 
principles.  An  introductory  course  on  psychology,  for 
example,  focuses  on  principles  of  human  anatomy  and 
physiology, elementary  statistics, genetics,  culture,  and 
so  forth.  In  a  process  of  theoretical  analysis,  a  structure 
is  made  of  the  learning  content.  Such  a  structure  differs 
from  a  causal  model  because  it  shows  principles  that 
are  elaborations  of  other  principles,  while  a  causal 
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model  shows  principles  that  combine  with  other  prin¬ 
ciples.  The  theoretical  elaboration  sequence  that  is 
derived  from  the  theoretical  structure  first  teaches,  top¬ 
ically  or  spirally,  the  broadest,  most  inclusive  and  most 
general  principles  along  with  the  supporting  content, 
and  then  proceeds  to  teach  ever  more  narrow,  less 
inclusive,  more  detailed,  and  more  precise  principles 
and  supporting  content. 

Elaboration  theory  clearly  reflects  some  basic 
principles  of  whole-task  models.  The  topical  and 
spiral  approach  to  sequencing  works  from  simple  to 
complex  wholes.  The  combination  of  organizing 
content  (conceptual,  theoretical)  and  supporting  con¬ 
tents  aims  at  the  integration  of  knowledge,  skills,  and 
attitudes.  The  concept  of  learning  episodes  is  used 
to  denote  instructional  units  that  allow  for  review 
and  synthesis  without  breaking  up  the  idea  of  a 
meaningful  whole.  And,  finally,  elaboration  theory 
suggests  giving  learners  some  control  over  both  con¬ 
tent  and  instructional  methods,  which  resembles  the 
principle  of  shared  responsibility.  Later  versions  of 
the  theory  pay  more  attention  to  procedural  organiz¬ 
ing  contents,  with  a  focus  on  solution  steps  or  heu¬ 
ristic  tasks  that  focus  on  problem-solving  principles, 
guidelines,  and  causal  models.  In  a  process  of  task 
analysis,  a  flowchart  is  made  that  depicts  the  steps 
(for  a  procedural  task)  or  the  principles,  guidelines, 
and  causal  models  (for  a  heuristic  task)  experts 
would  use  to  decide  what  to  do  when.  Based  on  this 
flowchart,  a  simplifying  conditions  sequence  is  built 
(see  right  column  in  Table  35.1).  It  begins  with  the 
simplest  version  of  the  task  that  is  still  representative 
of  the  whole  task  and  ends  with  the  most  complex 
version  of  this  task.  This  approach  is  also  typical  for 
the  whole-task,  learning-by-doing  models  discussed 
in  the  next  two  sections. 

Goal-Based  Scenarios 

Goal-based  scenarios  (Schank,  1993/1994;  Schank  et 
al.,  1993/1994)  are  the  backbone  of  learning  in 
Schank’s  learning-by-doing  paradigm  (Schank  et  al., 
1999).  These  goal-based  scenarios  represent  “a  learn- 
by-doing  simulation  in  which  students  pursue  a  goal 
by  practicing  target  skills  and  using  relevant  content 
knowledge  to  help  them  achieve  their  goal”  (Schank 
et  al.,  1999,  p.  165).  A  goal-based  scenario  consists  of 
the  following  seven  essential  components: 

•  Goal.  Two  categories  of  learning  goals  are 
distinguished:  process  knowledge  and  con¬ 
tent  knowledge.  Process  knowledge  reflects 
the  skills  and  attitudes  necessary  to  solve  a 
problem,  while  content  knowledge  considers 


the  knowledge  required  to  achieve  the  goal. 

A  goal-based  scenario  best  appeals  to  both 
categories. 

•  Mission.  The  mission  of  a  goal-based  sce¬ 
nario  is  closely  related  to  the  goal  and  rep¬ 
resents  the  actual  assignment  the  student  has 
to  carry  out.  The  mission  must  be  as  realistic 
and  motivating  as  possible  for  the  students. 

It  should  resemble  a  real-life  task  that  a  real 
person  would  plausibly  have  to  achieve  for 
an  important  reason. 

•  Cover  story.  The  cover  story  forms  the 
incentive  for  the  students  to  become  engaged 
in  practice.  It  should  elicit  students  to  prac¬ 
tice  the  skills  and  attitudes  and  seek  the 
information  and  construct  the  knowledge 
that  are  reflected  in  the  learning  goals. 

•  Role.  The  role  defines  the  perspective  the 
student  will  take  in  the  cover  story.  This  role 
should  ensure  that  the  student  will  achieve 
his  or  her  learning  goals,  and  it  should  be  as 
realistic  and  appealing  as  possible  to  moti¬ 
vate  them. 

•  Scenario  operations.  The  scenario  opera¬ 
tions  are  the  activities  a  student  carries  out 
to  accomplish  the  learning  goal.  Each  goal- 
based  scenario  should  elicit  numerous  sce¬ 
nario  operations  that  contain  decision  points 
so  students  can  infer  the  consequences  of 
their  actions. 

•  Resources.  Well-organized  and  easily  acces¬ 
sible  resources  contain  all  the  information 
(i.e.,  stories)  that  students  need  to  achieve 
the  learning  goal. 

•  Feedback.  The  situated,  just-in-time  feedback 
can  be  derived  from  the  consequences  of  cer¬ 
tain  actions,  it  could  be  given  by  coaches,  or 
it  can  be  found  in  the  resources  as  domain 
experts’  stories  about  similar  experiences. 

Elaboration  theory  and  goal-based  scenarios 
clearly  resemble  each  other.  Like  the  learning  episodes 
in  elaboration  theory,  goal-based  scenarios  provide  an 
opportunity  to  integrate  knowledge,  skills,  and  atti¬ 
tudes  in  meaningful  wholes.  In  addition,  both  theories 
stress  the  importance  of  some  learner  control  over 
contents  and  strategies.  In  goal-based  scenarios,  teach¬ 
ers  may  design  a  diverse  set  of  goals  to  help  learners 
with  different  prior  knowledge  and  interests  acquire  the 
same  knowledge,  skills,  and  attitudes,  or  learners  may 
sometimes  even  be  permitted  to  set  their  own  subgoals. 
Compared  to  elaboration  theory,  however,  goal-based 
scenarios  pay  far  less  attention  to  the  sequencing  of 
instruction.  In  contrast,  there  is  a  stronger  focus  on  the 
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Figure  35.1  A  schematic  overview  of  the  4C/ID  model  and  its  main  elements. 


performance  of  real-life  tasks  in  authentic  contexts  to 
facilitate  the  transfer  of  learning.  This  fits  the  basic 
idea  that  more  general  goals  (i.e.,  integrated  objec¬ 
tives)  should  drive  the  learning  process, because  highly 
specific  learning  objectives  invite  learners  to  apply 
strategies  that  do  not  allow  for  the  transfer  of  learning 
(for  the  role  of  goals  in  reaching  the  transfer  of  learn¬ 
ing,  see  Machin,  2002). 

Four-Component  Instructional  Design 

Van  Merrienboer’s  four-component  instructional 
design  model  (4C/ID  model)  (van  Merrienboer,  1997; 
van  Merrienboer  et  al.,  2002, 2003)  claims  that  whole- 
task  environments  for  complex  learning  can  always  be 
described  in  four  components: 

•  Learning  tasks— Meaningful  whole-task 
experiences  that  are  based  on  real-life  tasks. 
Ideally,  the  learning  tasks  ask  the  learners  to 
integrate  and  coordinate  many  if  not  all 
aspects  of  real-life  task  performance,  includ¬ 
ing  problem  solving  and  reasoning  aspects, 
which  are  different  across  tasks,  and  routine 
aspects,  which  are  consistent  across  tasks. 

•  Supportive  information — Information  that  is 
supportive  to  the  learning  and  performance 
of  problem  solving  and  reasoning  for  learn¬ 
ing  tasks.  It  describes  how  the  task  domain 
is  organized  and  how  problems  in  this 
domain  can  best  be  approached.  It  builds  a 
bridge  between  what  learners  already  know 


and  what  may  be  helpful  to  know  so  they 
can  work  fruitfully  on  the  learning  tasks. 

•  Procedural  information — Information  that 
is  prerequisite  to  the  learning  and  perfor¬ 
mance  of  routine  aspects  of  learning  tasks. 
This  information  provides  an  algorithmic 
specification  of  how  to  perform  those  routine 
aspects.  It  is  best  organized  in  small  infor¬ 
mation  units  and  presented  to  learners  pre¬ 
cisely  when  they  need  it  during  their  work 
on  the  learning  tasks. 

•  Part-task  practice.  Additional  exercises  for 
routine  aspects  of  learning  tasks  for  which  a 
very  high  level  of  automaticity  is  required 
after  the  instruction.  Part-task  practice  is 
only  necessary  if  the  learning  tasks  do  not 
provide  enough  repetition  for  a  particular 
routine  aspect  to  reach  the  required  high 
level  of  automaticity. 

The  4C/ID  model  shares  it  focus  on  authentic 
learning  tasks  with  goal -based  scenarios.  The  tasks 
are  based  on  real-life  tasks  and  ask  learners  to  com¬ 
bine  knowledge,  skills,  and  attitudes.  As  in  elabora¬ 
tion  theory,  ample  attention  is  paid  to  sequencing  the 
learning  tasks.  To  do  so,  learning  tasks  are  organized 
into  so-called  task  classes,  which  are  ordered  from 
simple  to  complex  (see  Figure  35.1;  task  classes  are 
the  dotted  rectangles  around  a  set  of  learning  tasks). 
The  first  task  class  contains  the  simplest  learning 
tasks  that  an  expert  might  encounter  in  the  real  world 
yet  are  still  whole,  meaningful  tasks  that  allow  the 
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learners  to  quickly  develop  a  holistic  vision  of  the 
task.  Subsequent  task  classes  contain  increasingly 
more  complex  and  difficult  learning  tasks.  Learning 
tasks  within  a  particular  task  class  are  equivalent  in 
the  sense  that  they  can  be  performed  on  the  basis  of 
the  same  body  of  knowledge;  each  subsequent,  more 
complex  task  class  requires  more  knowledge  or  more 
elaborated  knowledge  for  effective  performance  than 
the  preceding,  simpler  task  classes. 

An  additional  element  in  the  4C/ID  model  pertains 
to  the  support  and  guidance  that  are  given  to  learners 
(in  Figure  35.1,  this  is  indicated  by  the  filled-in  cir¬ 
cles).  When  learners  begin  to  work  on  more  difficult 
learning  tasks  in  a  new  task  class,  they  first  receive 
sizeable  support  and  guidance  from  the  teacher  or 
instructional  materials.  Support  and  guidance  gradu¬ 
ally  decrease  in  a  process  of  scaffolding  as  learners 
acquire  more  expertise.  When  learners  are  able  to  inde¬ 
pendently  perform  learning  tasks  without  any  support 
or  guidance  up  to  the  given  standards  (empty  circles 
in  Figure  35.1),  they  are  ready  to  continue  to  a  next 
task  class  with  more  difficult  tasks. 

Also  characteristic  of  the  model  is  its  focus  on 
the  transfer  of  learning.  Supportive  information  (L- 
shaped,  light-gray  shapes  in  Figure  35.1)  relates  to 
the  idea  that  transfer  is  explained  by  general  or 
abstract  information  that  may  be  interpreted  by  a  task 
performer  to  solve  a  new  problem  situation  (i.e.,  the 
Gestalt  approach).  Conceptual  models  (what  is  this?), 
structural  models  (how  is  this  built?),  causal  models 
(how  does  this  work?),  and  cognitive  strategies  (how 
should  I  approach  this  task?)  provide  this  kind  of 
information.  Procedural  information  (dark-gray  rect¬ 
angles  with  upward-pointing  arrows  in  Figure  35.1) 
and  part-task  practice  (small  series  of  circles  in  Fig¬ 
ure  35.1)  relate  to  the  idea  that  transfer  may  be 
explained  by  the  application  of  knowledge  elements 
that  are  shared  between  the  practice  and  the  transfer 
situation  (i.e.,  the  associationist  approach).  Proce¬ 
dural  information  tells  the  learner,  precisely  when  the 
information  is  needed,  how  to  perform  routine 
aspects  of  the  learning  tasks.  Part-task  practice  may 
provide  the  additional  practice  required  to  develop 
knowledge  elements  that  allow  the  learner  to  perform 
routine  aspects  at  a  high  level  of  automaticity.  Finally, 
like  elaboration  theory  and  goal -based  scenarios,  the 
4C/ID  model  acknowledges  the  importance  of  a  cer¬ 
tain  degree  of  learner  control  over  and  responsibility 
for  the  learning  process.  In  new  versions  of  the  model 
(van  Merrienboer  and  Kirschner,  2007),  learners  are 
able  to  select  their  own  learning  tasks ,  as  well  as  the 
supportive  information  and  the  part-task  practice  rel¬ 
evant  for  those  tasks. 


EMPIRICAL  EVIDENCE  FOR 
THE  EFFECTIVENESS  OF 
WHOLE-TASK  MODELS 

The  previous  sections  presented  a  brief  history  of 
whole-task  models,  discussed  the  main  characteristics 
of  whole-task  models  in  the  field  of  educational  com¬ 
munications  and  technology,  and  described  three 
example  models.  The  next  sections  discuss,  in  order, 
general  research  on  the  effectiveness  of  the  whole-task 
approach  and  specific  research  on  the  effectiveness  of 
elaboration  theory,  goal -based  scenarios,  and  four- 
component  instructional  design. 

Whole-Task  Practice 

In  a  review  study,  Wightman  and  Lintern  (1985)  inves¬ 
tigated  the  effectiveness  of  segmentation,  fraction¬ 
ation,  and  simplification  techniques  for  part-task  train¬ 
ing.  They  concluded  that  whole-task  training  was 
generally  more  beneficial  for  learning  than  part-task 
training.  Only  segmentation  in  the  form  of  backward 
chaining  with  snowballing  (i.e.,  first  the  final  segment 
of  a  task  is  practiced  and  prior  segments  are  succes¬ 
sively  added  during  training)  was  as  beneficial  for 
transfer  performance  as  whole-task  training.  This  con¬ 
clusion  was  confirmed  by  the  results  of  a  study  by 
Wightman  and  Sistrunk  (1987)  that  showed  that  learn¬ 
ers  who  received  whole-task  carrier-landing  training 
outperformed  learners  who  received  part-task  training 
using  simplification,  but  they  did  not  outperform  learn¬ 
ers  who  received  part-task,  backward-chaining  train¬ 
ing.  Moreover,  Goettl  and  Shute  (1996)  found  that  a 
segmentation  with  backward-chaining  approach  to 
training  is  only  as  efficient  as  whole-task  training  if  it 
is  composed  of  critical  component  tasks.  Otherwise, 
whole-task  training  leads  to  higher  transfer  test  per¬ 
formance. 

In  two  laboratory  experiments,  Detweiler  and 
Lundy  (1995)  studied  the  effects  of  single-task  and 
dual-task  practice  of  consistent  word-category  map¬ 
ping  and  spatial-pattern  visual  search  tasks  on  dual¬ 
task  transfer  test  performance.  They  found  that  dual¬ 
task  practice  yielded  higher  transfer  test  performance 
than  single-task  practice.  Based  on  their  findings,  they 
concluded  that  target  tasks  that  must  be  performed 
together  should  also  be  practiced  together.  Peck  and 
Detweiler  (2000)  compared  four  approaches  to  practic¬ 
ing  concurrent  tasks  related  to  adjusting  the  navigation 
and  peripheral  systems  of  a  fictitious  submarine:  (1)  a 
part-task  approach,  (2)  a  part-task  chaining  approach 
(i.e.,  snowballing),  (3)  a  concurrent-chaining  approach, 
and  (4)  a  whole-task  approach.  In  this  experiment,  part- 
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task  practice  consisted  of  single-task  trials  that 
required  an  adjustment  of  either  the  navigation  system 
or  the  peripheral  system.  In  part-task  chaining  practice, 
single-task  practice  gradually  became  multi-task  prac¬ 
tice.  Concurrent-chaining  practice  referred  to  increas¬ 
ingly  complex  concurrent-task  practice,  and  in  whole- 
task  practice  concurrent-task  practice  was  required 
from  the  start.  The  results  of  this  study  demonstrated 
the  beneficial  effects  on  transfer  test  performance  of 
both  the  concurrent-chaining  approach  and  the  whole- 
task  approach. 

To  summarize,  whole-task  training  seems  to  be  the 
preferred  method  of  instruction  when  it  comes  to  com¬ 
plex  skill  acquisition.  Sometimes,  however,  it  may  be 
desirable  to  include  some  (additional)  part-task  prac¬ 
tice  in  the  training— for  example  ,  when  the  whole  task 
is  dangerous  or  highly  demanding.  Roessingh  et  al. 
(2002)  developed  and  tested  a  model  to  determine  the 
optimal  training  time  schedule  for  such  a  combined 
training.  They  concluded  that,  if  one  part  task  has  to 
be  included  in  the  training,  then  more  than  50%  of  the 
total  training  time  has  to  be  devoted  to  whole-task 
practice  to  maximize  performance.  This  indicates  that 
even  in  cases  where  whole-task  practice  is  not  the  only 
relevant  type  of  training  it  still  should  take  up  the 
largest  part  of  it.  An  example  of  the  successful  appli¬ 
cation  of  combined  training  can  be  found  in  an  exper¬ 
iment  of  Pollock  et  al.  (2002).  They  compared  a  mixed 
instructional  approach  (i.e.,  first  part-task  practice  then 
whole-task  practice)  to  a  whole-task  approach  (i.e., 
only  whole-task  practice)  to  teach  electrical  safety  tests 
and  found  that  novices  in  the  domain  demonstrated 
superior  performance  on  a  knowledge  test  and  practi¬ 
cal  task  after  receiving  the  mixed  instruction.  Such  a 
combined  approach,  in  which  part-task  practice  is  used 
to  supplement  preponderant  whole-task  practice,  is 
also  found  in  four-component  instructional  design. 
First,  though,  research  on  elaboration  theory  and  goal- 
based  scenarios  are  discussed. 

Elaboration  Theory 

Beissner  and  Reigeluth  (1989, 1994)  studied  whether 
it  was  possible  to  design  a  course  on  physical-therapy 
treatment  with  combined  sequencing  techniques  (i.e., 
multiple  strands)  as  prescribed  by  elaboration  theory. 
They  carefully  described  the  four-step  design  process. 
In  the  first  step,  a  procedural  elaboration  sequence 
was  designed  using  the  simplifying  conditions 
approach.  Subsequently,  and  independent  of  the  pro¬ 
cedural  sequence,  a  prescriptive  theoretical  elabora¬ 
tion  sequence  (i.e.,  understanding  why  principles 
work)  and  a  descriptive  theoretical  elaboration 
sequence  (i.e.,  understanding  how  principles  work) 


were  designed  as  steps  two  and  three.  Finally,  these 
three  steps  were  integrated  into  a  single  course 
sequence.  Beissner  and  Reigeluth  concluded  that  it  is 
possible  to  have  parallel  elaboration  sequences  in  one 
course,  yielding  learning  episodes  (whole  tasks)  that 
integrate  relevant  knowledge,  skills,  and  attitudes. 
The  effectiveness  of  this  multiple-strand  technique, 
however,  was  not  investigated. 

English  and  Reigeluth  (1996)  conducted  a  study 
that  used  a  mixed-method  approach  to  determine  the 
strengths  and  weaknesses  of  elaboration  theory.  They 
revised  three  chapters  of  a  book  on  electrical  circuit 
analysis  according  to  multiple-strand  sequencing 
(combining  procedural  and  theoretical  sequences) .  The 
qualitative  data  (i.e.,  impromptu  and  debriefing  ques¬ 
tions)  obtained  in  this  study  indicated  that  the 
sequences  could  be  improved  in  the  following  ways: 
( 1 )  Every  step  in  the  elaboration  sequence  should  high¬ 
light  the  important  relationships  and  relate  each  step 
to  previous  steps  and  knowledge,  (2)  students  should 
be  informed  that  the  first  step  is  entry  level  and  will 
be  followed  by  steps  that  are  more  complex  at  the  start 
of  the  sequence,  and  (3)  learners  should  be  given  some 
learner  control  over  the  learning  materials  (note  that 
this  recommendation  is  incorporated  in  the  1999  ver¬ 
sion  of  elaboration  theory  discussed  earlier).  Further¬ 
more,  the  quantitative  data  (pre-  and  post-test  results) 
showed  that  the  instructional  material  was  effective  for 
both  low-ability  and  high-ability  students.  An  explor¬ 
ative  comparison  of  student  performance  after  the 
experimental  course  and  the  regular  course  on  this 
topic  indicated  that  students  who  participated  in  the 
experimental  course  performed  better  than  students  in 
the  regular  course. 

Goal-Based  Scenarios 

Bell  et  al.  (1993/1994)  evaluated  a  goal-based  sce¬ 
nario  involving  sickle  cell  disease  known  as  the 
Sickle  Cell  Counselor.  The  Sickle  Cell  Counselor  was 
installed  in  the  Museum  of  Science  and  Industry  in 
Chicago  to  evaluate  its  usefulness.  The  aim  of  the 
installation  was  to  give  museum  visitors,  in  the  course 
of  a  brief  interaction,  an  understanding  of  sickle  cell 
disease  and  the  basic  underlying  inheritance  mecha¬ 
nisms.  Based  on  the  fact  that  in  a  period  of  25  days, 
933  individuals  spent  on  the  average  more  than  7 
minutes  on  the  installation,  the  authors  concluded  that 
the  Sickle  Cell  Counselor  succeeded  in  attracting  and 
holding  onto  the  visitors’  attention.  The  effectiveness 
of  the  Sickle  Cell  Counselor  was  evaluated  in  a  sub¬ 
sequent  study.  Three  groups  were  compared:  a  group 
that  used  the  Sickle  Cell  Counselor,  a  group  that  read 
a  pamphlet  that  conveyed  the  same  information,  and 
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a  control  group.  The  group  that  used  the  Sickle  Cell 
Counselor  outperformed  both  the  pamphlet  group  and 
the  control  group  in  role-playing  performance  and  a 
paper- and-pencil  test. 

Another  goal-based  scenario.  Architecting  Busi¬ 
ness  Change,  developed  for  Andersen  Consulting  in  St. 
Charles,  Illinois,  was  evaluated  by  Kantor  et  al.  (2000). 
This  goal-based  scenario  addressed  eight  skills,  and 
Kantor  and  colleagues  conducted  a  needs  assessment 
to  find  out  the  minimal  proficiency  level  for  each  skill. 
The  proficiency  level  ranged  from  little/no  ability  to 
expert  and  was  based  on  ratings  made  by  supervisors. 
The  goal-based  scenario  was  evaluated  against  those 
minimal  proficiency  levels.  After  working  with  the 
goal -based  scenario,  students  were  asked  to  rate  their 
proficiency  level  before  and  after  completing  the  sce¬ 
nario.  A  comparison  of  the  minimal  proficiency  levels 
and  those  reported  after  completion  indicated  that  stu¬ 
dents  rated  their  proficiency  level  higher  than  mini¬ 
mally  required  for  five  out  of  eight  of  the  trained  skills. 
In  addition,  a  comparison  of  the  proficiency  levels 
before  and  after  completing  the  scenario  indicated  that 
all  students  rated  their  proficiency  level  higher  after 
than  before  working  with  the  goal-based  scenario. 

Four-Component  Instructional  Design  (4C/ID) 

In  two  studies,  Hoogveld  et  al.  (2001,  2003)  studied 
the  effectiveness  of  the  4C/ID  model  as  an  instruc¬ 
tional  systems  design  (ISD)  approach  to  designing 
competence-based  education  (for  applications  in  the 
medical  domain,  see  Janssen-Noordman  et  al.,  2006). 
In  the  first  study,  two  groups  of  teachers  were  com¬ 
pared:  One  group  was  trained  to  use  the  4C/ID  model 
to  design  instruction,  and  the  other  group  was  trained 
to  optimize  its  own  design  approach.  After  the  training 
phase,  the  design  quality  of  their  educational  products 
was  measured  by  experts,  and  it  was  found  that  teach¬ 
ers  trained  to  use  the  4C/ID  model  developed  qualita¬ 
tively  better  designs  than  the  other  teachers.  The  sec¬ 
ond  study  investigated  whether  teams  or  individuals 
benefited  more  from  a  4C/ID  approach  to  designing 
competence -based  education.  It  was  found  that  low 
achievers  benefited  more  from  the  4C/ID  model  when 
they  were  working  in  teams,  but  high  achievers  worked 
as  well  in  teams  as  individually. 

Other  researchers  studied  the  effectiveness  of 
4C/ID-based  instruction  from  the  learners’  perspec¬ 
tive.  Nadolski  et  al.  (2005)  focused  on  segmenting 
complex  whole  learning  tasks  in  the  area  of  law  into 
phases.  They  varied  the  number  of  phases  (one,  four, 
or  nine)  of  the  whole  task  to  determine  the  optimal 
balance  between  the  advantages  of  whole-task  practice 
and  the  disadvantages  of  cognitive  overload  caused  by 


whole  tasks  that  are  too  complex  for  learners.  The 
results  of  this  study  showed  that  learners  who  carried 
out  the  learning  task  in  four  phases  were  most  effective 
during  practice  as  measured  by  the  coherence  and  con¬ 
tent  of  their  practice  products.  Learners  who  carried 
out  the  learning  tasks  in  one  or  four  phases  were  most 
efficient  during  practice  as  measured  by  a  combination 
of  practice-product  quality  and  invested  mental  effort. 
No  effects  were  found  on  transfer  test  performance. 
These  results  were  confirmed  in  a  follow-up  study 
(Nadolski  et  al.,  2006)  in  which  learners  who  received 
learning  tasks  that  consisted  of  four  phases  outper¬ 
formed  learners  who  received  tasks  consisting  of  eight 
phases.  The  results  indicate  that  whole  learning  tasks 
should  only  be  segmented  if  this  is  the  only  possible 
way  to  diminish  their  complexity. 

Using  a  model  closely  resembling  the  4C/ID 
approach,  Merrill  (2002b)  carried  out  a  study  at 
Thompson/NETg  to  validate  his  first  principles  of 
instruction.  Three  instructional  scenarios  for  a  course 
in  Excel  were  developed:  (1)  a  whole-task  scenario 
group,  (2)  an  e-learning  group  that  received  the  exist¬ 
ing  commercial  version  of  the  NETg  Excel  course, 
and  (3)  a  control  group  that  only  received  three 
authentic  test  tasks.  Statistical  differences  were  found 
between  the  three  groups  on  test  task  performance. 
The  whole-task  scenario  group  scored  an  average  of 
89%,  the  e-learning  group  scored  an  average  of  68%, 
and  the  control  group  scored  only  an  average  of  34% 
correct  on  the  test  tasks.  In  addition,  the  whole-task 
scenario  group  required  significantly  less  time  to 
complete  the  three  test  tasks  compared  to  the  e-leam- 
ing  group. 

A  study  by  Lim  and  Reiser  (2006)  compared  the 
effects  of  4C/ID  whole-task  training  and  part-task 
training  on  the  acquisition  and  transfer  of  a  complex 
cognitive  skill  (preparing  a  grade  book  in  Excel)  for 
novices  and  advanced  learners .  They  found  that  both 
novice  and  advanced  learners  achieved  better  whole- 
task  performance  and  better  transfer  performance  if 
they  received  the  4C/ID  whole-task  training.  The 
superiority  of  a  4C/ID  approach  over  other 
approaches  was  confirmed  in  another  classroom  study 
(Sarfo  and  Elen,  2005,  2006)  that  compared  three 
groups  who  had  to  learn  how  to  design  a  single  build¬ 
ing  plan  based  on  local  conditions.  The  control  group 
was  taught  according  to  an  approach  that  was  applied 
in  technical  schools  in  Ghana;  the  experimental 
groups  were  taught  according  to  the  4C/ID  approach, 
either  with  or  without  technology-enhanced  learning. 
Although,  the  groups  performed  equally  well  on  a 
pretest  and  both  showed  learning  gains  on  a  post-test, 
both  experimental  groups  outperformed  the  control 
group  on  the  post-test. 
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DISCUSSION 

This  chapter  has  discussed  the  historical  roots  of 
whole-task  models  and  the  main  features.  It  has  also 
provided  three  examples  of  whole-task  models  in  the 
field  of  educational  communications  and  technology 
and  has  presented  research  findings  concerning  the 
effectiveness  of  whole-task  models.  Overall,  research 
findings  show  that  whole -task  models  are  particularly 
effective  to  teach  complex  skills  and  professional  com¬ 
petences;  however,  this  research  also  highlights  two 
limitations  of  whole-task  models.  First,  whole-task 
models  should  only  be  used  to  teach  content  and  tasks 
that  are  characterized  by  a  high  degree  of  coordina¬ 
tion— that  is,  many  interrelationships  between  knowl¬ 
edge  elements  and  component  skills.  If  coordination 
is  low,  part-task  models  may  be  equally  effective  or 
even  more  effective  than  whole-task  models.  Second, 
whole-task  practice  should  not  be  seen  as  incompatible 
with  part-task  practice.  Even  in  a  dominant  whole-task 
approach,  it  may  be  desirable  to  offer  additional  part- 
task  practice  for  routine  aspects  of  a  complex  task. 
Furthermore,  if  it  proves  to  be  impossible  to  develop 
a  version  of  the  whole  task  that  is  simple  enough  to 
begin  the  training  with,  it  may  be  necessary  to  work 
from  parts  that  are  as  large  as  possible  within  a  whole- 
part  approach.  In  a  purely  holistic  model,  however,  this 
would  be  viewed  as  a  very  last  resort. 

Whole-task  models  are  relatively  new  in  the  field 
of  educational  communications  and  technology,  and 
there  is  a  clear  need  for  additional  research.  On  the 
most  basic  level,  much  is  still  unknown  regarding  the 
design  of  optimal  whole  learning  tasks,  sequencing 
techniques,  and  ways  to  scaffold  learners  who  are 
working  on  the  tasks.  Another  set  of  research  questions 
pertains  to  learner  control  and  self-directed  learning. 
In  a  whole-task  model,  self-directed  learners  should 
select  their  own  learning  tasks,  based  on  self-assess¬ 
ments  of  their  performance  and  information  regarding 
the  tasks  (e.g.,  difficulty,  available  support  and  guid¬ 
ance).  Future  research  should  investigate  how  learners 
are  best  guided  in  their  process  of  task  selection  (e.g., 
through  the  use  of  portfolios  or  coaching)  and  encour¬ 
aged  to  develop  their  self-directed  learning  skills. 
Finally,  the  issue  of  transfer  of  learning  is  central  to 
whole-task  models.  Here,  an  interesting  line  of 
research  pertains  to  the  importance  of  real-life,  authen¬ 
tic  whole  tasks.  Although  some  researchers  argue  that 
it  is  critical  to  perform  those  tasks  in  an  authentic 
environment  (see,  for  example.  Brown  et  al.,  1989), 
others  claims  that  especially  early  in  the  learning  pro¬ 
cess  it  is  not  the  fidelity  of  the  environment  but  the 
psychological  fidelity  of  the  task  itself  that  is  most 
important. 


It  is  clear  that  whole-task  models  today  hold  a 
prominent  position  in  the  field  of  educational  commu¬ 
nications  and  technology  and,  at  least  in  Western 
Europe,  in  vocational  education  and  training  and  in 
higher  professional  education.  In  our  opinion,  this  is 
an  inevitable  reaction  to  societal  and  technological 
developments  as  well  as  students’  and  employers’ 
uncompromising  views  regarding  the  value  of  educa¬ 
tion.  Due  to  new  technologies,  routine  tasks  have  been 
taken  over  by  machines  and  the  complex  cognitive 
tasks  that  must  be  performed  by  humans  are  becoming 
increasingly  important.  Moreover,  both  the  nature  of 
and  the  skills  required  for  currently  available  jobs  are 
rapidly  changing  while  the  information  relevant  to  car¬ 
rying  out  these  jobs  quickly  becomes  obsolete.  This 
imposes  higher  demands  on  the  workforce,  as  employ¬ 
ers  stress  the  importance  of  problem  solving,  reason¬ 
ing,  and  self-directed  learning  to  ensure  that  employ¬ 
ees  can  and  will  flexibly  adjust  to  rapid  changes  in 
their  environment.  Whole-task  models  in  education 
aim  to  reach  precisely  those  goals. 
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ABSTRACT 

In  this  chapter,  we  discuss  the  possible  roles  of  models 
in  learning,  with  computer  models  (simulations)  as  our 
focus.  In  learning  from  models,  students’  learning  pro¬ 
cesses  center  around  the  exploration  of  a  model  by 
changing  values  of  input  variables  and  observing  result¬ 
ing  values  of  output  variables.  In  this  process,  they 
experience  rules  of  the  simulated  domain  or  discover 
aspects  of  these  rules.  Models  can  also  play  a  role  in 
the  learning  process  when  we  ask  students  to  construct 
models.  In  learning  by  modeling,  students  are  required 
to  construct  an  external  model  that  can  be  simulated  to 
reproduce  phenomena  observed  in  a  real  system. 
Finally,  both  ways  of  using  models  can  be  combined 
in  what  we  refer  to  as  model-based  inquiry  learning. 
Here,  students  encounter  a  computer  model  that  they 
can  explore  by  changing  values  of  the  input  variables 
and  by  observing  values  of  the  output  variables  and 
then  they  reconstruct  the  model,  including  its  internal 
functioning,  so  both  models  will  behave  similarly. 


KEYWORDS 

Inquiry  learning:  “An  approach  to  learning  that 
involves  a  process  of  exploring  the  natural  or  mate¬ 
rial  world,  and  that  leads  to  asking  questions,  mak¬ 
ing  discoveries,  and  rigorously  testing  those  discov¬ 
eries  in  the  search  for  new  understanding”  (NSF, 

2000,  p.  2). 

Model:  Structured  representation  of  a  system  in  terms 
of  variables  or  concepts  and  their  (quantitative  or 
qualitative)  relations  that  can  be  used  for  predicting 
system  behavior  by  means  of  simulations. 

Modeling:  The  process  of  creating  simulations  as  a 
means  for  learning. 

Simulation:  Computer-based  model  of  a  natural  process 
or  phenomenon  that  reacts  to  changes  in  values  of 
input  variables  by  displaying  the  resulting  values  of 
output  variables. 
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INTRODUCTION 

In  many  domains,  especially  in  science,  learning 
involves  the  acquisition  and  construction  of  models 
(Lehrer  and  Schauble,  2006).  Models  are  defined  as 
“a  set  of  representations,  rules,  and  reasoning  struc¬ 
tures  that  allow  one  to  generate  predictions  and  expla¬ 
nations”  (Schwarz  and  White,  2005,  p.  166).  Models 
can  be  seen  as  structured  representations  of  (parts  of) 
domains  in  terms  of  variables  or  concepts  and  their 
interconnections.  Each  scientific  domain  has  a  set  of 
externally  represented  domain  models  that  are  gener¬ 
ally  agreed  upon  by  researchers  working  in  these 
domains.  Individuals  have  personal  models  that  may 
be  externally  represented  or  that  may  be  mental  models 
(Gentner  and  Stevens,  1983).  Scientific  practice  can 
be  seen  as  a  process  of  constantly  adapting,  refining, 
or  changing  models,  under  the  influence  of  observa¬ 
tions  or  of  constraints  set  by  the  properties  of  the 
models  themselves.  In  a  similar  vein,  learning  science 
consists  of  creating  and  adapting  mental  models  with 
the  aim  of  moving  the  mental  model  toward  an  expert 
or  theoretical  domain  model  (Clement,  2000;  Snyder, 
2000).  Such  adaptation  of  mental  models  may  evolve 
with  gradual  modifications  or  involve  more  radical 
changes  in  the  nature  of  the  mental  model  (Chi,  1992). 

In  its  influential  report,  the  American  Association 
for  the  Advancement  of  Science  (AAAS,  1989)  stated 
that  students  need  time  to  explore,  make  observations, 
take  wrong  turns,  test  ideas,  rework  tasks,  build  things, 
calibrate  instruments,  collect  things,  construct  physical 
and  mathematical  models,  learn  required  mathematics 
and  other  relevant  concepts ,  read  materials ,  discuss  and 
debate  ideas,  wrestle  with  unfamiliar  and  counterintu¬ 
itive  ideas,  and  explore  alternative  perspectives. 
According  to  this  description,  learning  resembles  cre¬ 
ating  and  adapting  mental  models  by  using  scientific 
inquiry.  In  2000,  the  National  Science  Foundation 
defined  inquiry  learning  as  “an  approach  to  learning 
that  involves  a  process  of  exploring  the  natural  or 
material  world,  and  that  leads  to  asking  questions, 
making  discoveries,  and  rigorously  testing  those  dis¬ 
coveries  in  the  search  for  new  understanding”  (NSF, 

2000,  p.  2). 

In  model-facilitated  learning,  the  natural  or  mate¬ 
rial  world  in  the  above  definition  is  replaced  by  a 
model.  These  models  can  take  many  forms  (e.g.,  a 
simplified  sketch  or  a  concept  map;  see  Gobert,  2000); 
however,  in  this  chapter,  we  speak  of  learning  from 
models  only  when  students  can  interact  with  the  model, 
which  means  that  they  can  manipulate  input  to  the 
model  with  a  reaction  of  the  model  as  a  result.  As  a 
further  specification,  we  focus  on  computer  models 
(simulations)  (de  Jong,  1991)  in  this  chapter.  This 


restriction  means  that  the  models  we  discuss  are  exe¬ 
cutable;  that  is,  they  use  some  computational  algorithm 
to  generate  output  (i.e.,  a  change  in  the  values  describ¬ 
ing  the  model’s  state)  on  the  basis  of  students’  input 
(Hestenes,  1987).  This  process  is  called  simulation.  In 
learning  from  models,  students’  learning  processes 
center  around  the  exploration  of  a  model  by  changing 
values  of  input  variables  and  observing  resulting  val¬ 
ues  of  output  variables.  In  the  process,  they  experience 
rules  of  the  simulated  domain  or  discover  aspects  of 
these  rules  (de  Jong,  2006a).  Models  can  also  play  a 
role  in  the  learning  process  when  we  ask  students  to 
construct  models.  In  this  learning  by  modeling ,  stu¬ 
dents  are  required  to  construct  an  external  model,  with 
the  objective  of  making  the  model  behave  as  much  like 
the  real  system  as  possible  (Penner,  2001).  Finally, 
both  ways  of  using  models  can  be  combined  in  what 
we  refer  to  as  model-based  inquiry  learning.  Here, 
students  receive  a  model  that  they  can  explore  by 
changing  values  of  the  input  variables  and  observing 
values  of  the  output  variables.  They  then  have  to  recon¬ 
struct  this  model,  including  its  internal  functioning,  in 
such  a  way  that  both  models  will  behave  similarly 
(Fohner  et  al.,  2005;  van  Joolingen  et  al.,  2005). 

In  this  chapter,  we  discuss  three  approaches  to 
using  models  in  education:  one  in  which  students  try 
to  grasp  the  properties  of  an  existing  model  ( learning 
from  models),  one  in  which  students  learn  from  creat¬ 
ing  models  ( learning  by  modeling) ,  and  a  way  of  learn¬ 
ing  in  which  these  two  forms  are  combined.  In  doing 
so,  we  concentrate  on  learning  in  the  science  domains. 

LEARNING  FROM 
COMPUTER  MODELS 

In  learning  from  computer  (simulation)  models,  stu¬ 
dents  try  to  build  a  mental  model  based  on  the  behavior 
of  a  given  model  with  which  they  can  experiment. 
There  is  a  large  variety  of  models  and  of  possible  ways 
to  interact  with  them  (van  Joolingen  and  de  Jong, 
1991),  but  students  basically  interact  with  a  computer 
model  through  a  model  interface  that  allows  them  to 
change  values  of  variables  in  the  model  and  that  dis¬ 
plays  the  computed  results  of  their  manipulations  in 
one  way  or  another. 

Computer  technology  supporting  learning  from 
computer  models  began  to  be  developed  in  the  late 
1970s  and  1980s.  Of  course,  many  simulations  existed 
that  were  used  more  or  less  directly  in  an  educational 
context,  but  only  a  few  systems  were  specifically 
geared  toward  education.  Many  of  these  systems  pri¬ 
marily  concerned  either  operational  models  or  a  com¬ 
bination  of  operational  and  conceptual  models. 
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SOPHIE,  for  example,  was  an  environment  for  teach¬ 
ing  electronic  troubleshooting  skills,  but  it  was  also 
designed  to  give  students  insight  into  electronic  laws, 
circuit  causality,  and  the  functional  organization  of 
particular  devices  (Brown  et  al.,  1982).  QUEST  also 
focused  on  electronic  circuit  troubleshooting  (White 
and  Frederiksen,  1989).  QUEST  used  model  progres¬ 
sion;  circuits  became  increasingly  more  complex  as 
students  progressed  through  QUEST,  and  they  could 
view  circuits  from  different  perspectives  (e.g.,  a  func¬ 
tional  or  a  behavioral  perspective) .  Another  system  that 
combined  the  learning  of  operational  and  conceptual 
knowledge  was  STEAMER.  This  system  simulated  a 
complex  steam  propulsion  system  for  large  ships  (Hol- 
lan  et  al.,  1984).  Systems  such  as  MACH-III  for  com¬ 
plex  radar  devices  (Kurland  and  Tenney,  1988)  and 
IMTS  (Towne  et  al.,  1990)  also  focused  on  trouble¬ 
shooting.  Smithtown  was  one  of  the  first  educational 
simulations  that  targeted  a  conceptual  domain  (eco¬ 
nomic  laws)  and  that  included  several  support  mech¬ 
anisms  for  students  (Shute  and  Glaser,  1990).  In 
Smithtown,  students  could  explore  simulated  markets. 
They  could  change  such  variables  as  labor  costs  and 
population  income  and  observe  the  effects  on,  for 
example,  prices.  A  further  example  of  early  conceptual 
simulations  for  education  was  ARK  (Alternate  Reality 
Kit),  a  set  of  simulations  on  different  physics  topics 
(e.g.,  collisions)  that  provided  students  with  direct 
manipulation  interfaces  (Scanlon  and  Smith,  1988; 
Smith,  1986). 

Although  scaffolds  were  already  present  to  some 
degree  in  the  systems  cited  here,  research  has  empha¬ 
sized  the  awareness  that  learning  from  models  can  only 
be  successful  if  the  student  is  sufficiently  scaffolded. 
Unscaffolded  inquiry  is  generally  seen  as  not  fruitful 
(Mayer,  2004).  Cognitive  scaffolds  can  be  integrated 
with  the  simulation  software  and  aim  at  one  or  more 
of  the  inquiry  processes  mentioned  above.  Overviews 
of  systems  that  contain  cognitive  scaffolds  have  been 
presented  by  de  Jong  and  van  Joolingen  (1998),  Quin¬ 
tana  et  al.  (2004),  Linn  et  al.  (2004),  and  recently  de 
Jong  (2006b). 

Identifying  the  basis  of  adequate  scaffolding 
requires  a  detailed  insight  into  the  learning  processes 
associated  with  learning  from  models  (de  Jong  and  van 
Joolingen,  1998).  The  overall  learning  process  that  is 
associated  with  learning  from  models  is  a  process  of 
scientific  discovery  or  inquiry.  The  National  Research 
Council  in  1996  defined  inquiry  as  a  multifaceted 
activity  involving  making  observations,  posing  ques¬ 
tions,  examining  various  sources  of  information,  plan¬ 
ning  investigations,  reviewing  what  is  known,  using 
tools  to  gather  and  interpret  data,  proposing  explana¬ 
tions  and  predictions,  and  communicating  findings; 


inquiry  requires  the  identification  of  explicit  assump¬ 
tions,  the  use  of  critical  and  logical  thinking,  and  the 
creation  and  consideration  of  alternative  explanations 
(NRC,  1996).  This  description  lists  a  large  set  of  pro¬ 
cesses  that  constitute  inquiry  learning.  De  Jong 
(2006b)  presented  a  number  of  processes  that  encom¬ 
pass  the  processes  mentioned  in  the  NRC  definition: 
orientation,  hypothesis  generation,  experimentation 
(i.e .,  experiment  design,  prediction,  data  interpreta¬ 
tion),  drawing  a  conclusion,  and  making  an  evaluation. 
In  orientation,  the  general  research  issue  is  determined 
and  the  student  makes  a  broad  analysis  of  the  domain; 
in  hypothesis  generation,  a  specific  statement  (or  a  set 
of  statements,  for  example,  in  the  form  of  a  model) 
about  the  domain  is  chosen  for  consideration;  in  exper¬ 
imentation,  a  test  to  investigate  the  validity  of  this 
hypothesis  or  model  is  designed  and  performed,  pre¬ 
dictions  are  made,  and  outcomes  of  the  experiments 
are  interpreted;  in  conclusion,  a  conclusion  about  the 
validity  of  the  hypothesis  is  drawn  or  new  ideas  are 
formed;  and,  finally,  in  evaluation,  a  reflection  on  the 
learning  process  and  the  domain  knowledge  acquired 
is  made.  A  central  and  developing  product  in  the 
inquiry  learning  process  is  the  student’s  mental  model 
of  the  domain  (White  and  Frederiksen,  1998). 

Figure  36.1  presents  a  diagrammatic  attempt  to 
depict  the  development  of  a  student’s  mental  model 
throughout  the  inquiry  process.  In  this  figure,  the  men¬ 
tal  model  in  orientation  has  loose  ends,  relations  are 
not  yet  defined,  and  variables  are  missing.  When  a 
student  generates  a  hypothesis,  a  relation  between  vari¬ 
ables  is  selected,  and  an  idea  (still  uncertain)  about 
this  relation  is  formed.  Of  course,  the  ideas  that  are 
formed  in  the  hypothesis  phase  are  not  necessarily 
constrained  to  single  hypotheses  but  may  refer  to 
broader  parts  of  a  model  (see  the  next  section).  In 
experimentation,  a  move  to  more  manipulable  vari¬ 
ables  is  made.  When  designing  an  experiment,  the 
conceptual  variables  are  operationalized  in  variables 
that  can  be  manipulated.  In  prediction ,  the  hypothesis 
that  was  stated  is  translated  into  observable  variables. 
In  data  interpretation ,  the  outcomes  of  the  experiment 
are  known,  and  an  understanding  of  the  data  must  be 
reached.  Stating  a  conclusion  involves  returning  to  a 
more  theoretical  level,  in  which  the  data  that  were 
interpreted  are  related  to  the  hypothesis  or  mental 
model  under  consideration  and  decisions  on  the  valid¬ 
ity  of  the  original  ideas  are  made. 

In  Figure  36.1,  the  process  of  experimentation  is 
at  the  level  of  manipulable  (operationalized)  variables, 
whereas  the  domain  view  in  the  processes  of  orienta¬ 
tion,  hypotheses,  and  conclusion  is  at  the  level  of  the¬ 
ory.  Ideally,  a  student’s  view  of  the  domain  should  go 
from  orientation  through  hypotheses  to  conclusion. 
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Orientation 


Hypotheses 


Figure  36.1  An  overview  of  the  student’s  mental  model  of  inquiry  processes  (ovals  are  variables,  lines  represent  relations).  (From 
de  Jong,  T.,  in  Dealing  with  Complexity  in  Learning  Environments,  Elen,  J.  and  Clark,  R.E.,  Eds.,  Elsevier,  London,  2006,  pp. 
107-128.  With  permission.) 


resulting  in  a  correct  and  complete  mental  model  of 
the  domain.  In  practice,  however,  after  going  through 
these  learning  processes  a  student’s  mental  model  will 
often  still  have  some  open  ends  (an  orientation  char¬ 
acter),  unresolved  issues  (a  hypothesis  aspect),  and 
some  firm  ideas  (conclusions,  but  still  some  of  these 
may  be  faulty) .  This  emphasizes  the  iterative  character 
of  the  inquiry  learning  process. 

The  processes  mentioned  above  directly  yield 
knowledge  (as  is  reflected  in  the  developing  view  of 
the  domain),  de  Jong  and  Njoo  (1992)  refer  to  these 
processes  as  transformative  inquiry  processes,  reflect¬ 
ing  the  transformation  of  information  into  knowledge. 
Because  inquiry  learning  is  a  complex  endeavor  with 
a  number  of  activities  and  iterations,  de  Jong  and  Njoo 
(1992)  added  the  concept  of  regulation  of  learning, 
comprised  of  processes  aimed  at  planning  and  moni¬ 
toring  the  learning  process.  Together,  transformative 
and  regulative  processes  form  the  main  inquiry  learn¬ 
ing  processes  (de  Jong  and  van  Joolingen,  1998). 

Evaluation  takes  a  special  place,  located  somewhere 
between  transformative  and  regulative  processes.  In 
evaluation  (or  reflection),  students  examine  the  inquiry 
process  and  its  results  and  try  to  take  a  step  back  to 
learn  from  their  experiences.  This  reflection  may  con¬ 
cern  the  inquiry  process  itself  (successful  and  less  suc¬ 
cessful  actions)  as  well  as  the  domain  under  investiga¬ 
tion  (e.g.,  general  domain  characteristics).  As  is  the  case 
with  all  inquiry  processes,  evaluation  activities  can 
occur  at  any  point  in  the  cycle,  not  just  during  evalua¬ 
tion.  Evaluation  activities  can  influence  the  inquiry  pro¬ 
cess  itself  and  thus  have  a  regulative  character. 


Smaller  scale  evaluations  of  inquiry  learning  often 
concentrate  on  assessing  the  effects  of  different  types 
of  scaffolding.  This  work  shows  that  the  effectiveness 
of  inquiry  learning  can  be  greatly  improved  by  offering 
students  adequate  scaffolds  (de  Jong,  2006a,b).  Large- 
scale  evaluations  of  technology-based  inquiry  environ¬ 
ments  comparing  them  to  more  traditional  modes  of 
instruction  are  not  very  frequent,  but  a  few  of  these 
large-scale  evaluations  do  exist.  Smithtown,  a  support¬ 
ive  simulation  environment  in  the  area  of  economics, 
was  evaluated  in  a  pilot  study  with  30  students  and  in 
a  large-scale  evaluation  with  a  total  of  530  students. 
Results  showed  that  after  5  hours  of  working  with 
Smithtown,  students  reached  a  degree  of  micro-eco¬ 
nomics  understanding  that  would  have  required 
approximately  1 1  hours  of  traditional  teaching  (Shute 
and  Glaser,  1990).  The  Jasper  project  offers  another 
classic  example  of  a  large-scale  evaluation.  The 
domain  in  this  project  is  mathematics,  and  students 
learn  in  real  contexts  in  an  inquiry  type  of  setting. 
Although  Jasper  is  not  a  pure  inquiry  environment,  the 
learning  has  many  characteristics  of  inquiry,  as  stu¬ 
dents  collect  and  try  to  interpret  data.  Evaluation  data 
involving  over  700  students  showed  that  students  who 
followed  the  Jasper  series  outperformed  a  control 
group  that  received  traditional  training  on  a  series  of 
assessments  (Cognition  and  Technology  Group  at 
Vanderbilt,  1992). 

White  and  Frederiksen  (1998)  described  the  Think- 
erTools  Inquiry  Curriculum,  a  simulation-based  learn¬ 
ing  environment  on  the  physics  topic  of  force  and 
motion.  The  ThinkerTools  software  guides  students 
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through  a  number  of  inquiry  stages  that  include  exper¬ 
imenting  with  the  simulation,  constructing  physics 
laws,  critiquing  each  other’s  laws,  and  reflecting  on 
the  inquiry  process.  Thinker  Tools  was  implemented  in 
12  classes  with  approximately  30  students  each.  Stu¬ 
dents  worked  daily  with  Thinker  Tools  over  a  period 
of  a  little  more  than  10  weeks.  A  comparison  of  the 
Thinker  Tools  students  with  students  in  a  traditional 
curriculum  showed  that  the  ThinkerTools  students  per¬ 
formed  significantly  better  on  a  (short)  conceptual  test 
(68%  vs.  50%  correct).  Even  the  students  who  scored 
low  on  a  test  for  general  basic  skills  from  the  Think¬ 
erTools  curriculum  had  a  higher  average  conceptual 
physics  score  (58%)  than  the  students  who  followed 
the  traditional  curriculum. 

Hickey  et  al.  (2003)  assessed  the  effects  of  the 
introduction  of  a  simulation-based  inquiry  environ¬ 
ment  (GenScope)  on  the  biology  topic  of  genetics.  In 
GenScope  students  can  manipulate  genetic  informa¬ 
tion  at  different  levels:  DNA,  chromosomes,  cells, 
organisms,  pedigrees,  and  populations.  Students,  for 
example,  can  change  the  chromosomes  (e.g.,  for  pres¬ 
ence  or  absence  of  wings  or  horns)  of  virtual  dragons, 
breed  these  dragons,  and  observe  the  effects  on  the 
genotype  and  phenotype  of  the  offspring.  A  large-scale 
evaluation  was  conducted  involving  31  classes  (23 
experimental,  8  comparison)  taught  by  13  teachers  and 
a  few  hundred  students  in  total.  Overall,  the  evaluation 
results  showed  better  performance  by  the  GenScope 
classes  compared  to  the  traditional  classes  on  tests 
measuring  genetic  reasoning.  A  follow-up  study  with 
two  experimental  classes  and  one  comparison  class 
also  showed  significantly  higher  gains  for  the  two 
experimental  classes  on  a  reasoning  test,  with  a  higher 
gain  for  students  from  the  one  of  these  two  groups  in 
which  more  investigation  exercises  were  offered. 

Another  recent  example  is  the  River  City  project. 
The  River  City  project  software  is  intended  to  teach 
biology  topics  and  inquiry  skills.  It  is  a  virtual  environ¬ 
ment  in  which  students  move  around  with  avatars. 
River  City  contains  simulations,  databases,  and  multi- 
media  information.  Students  have  to  perform  a  full 
investigation  following  all  of  the  inquiry  processes 
listed  above  and  end  their  investigation  by  writing  a 
letter  to  the  mayor  of  the  city.  Preliminary  results  of  a 
large  evaluation  (involving  around  2000  students)  of 
the  River  City  project  showed  that,  compared  to  a  con¬ 
trol  group  who  followed  a  paper-based  inquiry  based 
curriculum,  the  technology-based  approach  led  to  a 
higher  increase  in  biology  knowledge  (32  to  34%  vs. 
17%)  and  better  achievement  on  tests  for  inquiry  skills 
(Ketelhut  et  al.,  2006).  Linn  et  al.  (2006)  evaluated 
modules  created  in  the  Technology-Enhanced  Learning 
in  Science  (TELS)  center.  These  modules  are  inquiry 


based  and  contain  simulations  (e.g.,  on  the  functioning 
of  airbags).  Over  a  sample  of  4328  students  and  6 
different  TELS  modules,  an  overall  effect  size  of  0.32 
in  favor  of  the  TELS  subjects  over  students  following 
a  traditional  course  was  observed  on  items  that  mea¬ 
sured  how  well  students’  knowledge  was  integrated. 


LEARNING  BY  CREATING 
COMPUTER  MODELS 

Apart  from  observing  simulations  based  on  formal 
models,  students  can  also  learn  from  constructing 
these  models  themselves  (Alessi,  2000).  This  approach 
is  in  line  with  the  basic  ideas  behind  constructionism 
(Harel  and  Papert,  1991;  Kafai,  2006;  Kafai  and 
Resnick,  1996),  of  which  the  main  focus  is  “knowledge 
construction  that  takes  place  when  students  are 
engaged  in  building  objects”  (Kafai  and  Resnick, 
1996,  p.  2).  Objects  that  are  constructed  can  be  phys¬ 
ical  objects  and  artifacts  (Crismond,  2001),  drawings 
(Hmelo  et  al.,  2000),  concept  maps  (Novak,  1990), 
computer  programs  (Mayer  and  Fay,  1987),  instruction 
(Vreman-de  Olde  and  de  Jong,  2006),  and  more.  In 
this  section,  we  focus  on  constructing  executable  mod¬ 
els,  the  same  kind  of  models  that  are  explored  in  the 
situations  described  in  the  previous  section;  instead  of 
exploring  these  models,  the  students’  task  becomes 
one  of  constructing  them. 

Science  has  always  used  models  to  understand  a 
domain.  Simulation  as  a  tool  to  predict  a  model’s 
behavior  was  one  of  the  first  applications  of  computers 
as  they  became  available  shortly  after  World  War  II. 
The  use  of  constructing  models  in  the  process  of  learn¬ 
ing  science  goes  back  to  the  early  1980s,  when  Jon 
Ogborn  created  the  Dynamical  Modelling  System 
(DMS)  (Ogborn  and  Wong,  1984).  In  this  system,  stu¬ 
dents  could  create  a  model  of  a  dynamical  system  by 
entering  equations  that  described  an  initial  state  and 
the  change  of  that  state  over  time.  Even  before  these 
attempts,  Jay  Forrester  had  developed  his  ideas  on 
system  dynamics,  a  way  of  representing  processes  in 
business  organizations,  which  soon  acquired  a  wider 
use  as  a  versatile  tool  to  model  any  kind  of  system 
(Forrester,  1961).  An  example  of  a  system  dynamics 
model  is  provided  in  Figure  36.2.  This  model  uses  the 
system  dynamics  notation  introduced  by  Forrester 
(1961).  The  water  level  is  represented  by  a  stock  (rect¬ 
angle)  and  the  outflow  as  a  flow  (the  thick  arrow  point¬ 
ing  to  the  cloud.  The  thin  arrows  indicate  relations 
between  the  variables. 

At  first,  system  dynamics  models  were  created  as 
drawings  that  were  used  as  a  tool  for  reasoning.  Later 
these  models  were  used  as  a  guideline  to  create  computer 
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Figure  36.2  System  dynamics  model  of  a  leaking  water  bucket. 

programs,  and  eventually  systems  such  as  STELLA 
(Steed,  1992)  were  introduced  that  allowed  direct  sim¬ 
ulation  of  system  dynamics  models.  The  educational 
value  of  these  systems  was  immediately  recognized, 
and  other  systems  following  the  same  basic  system 
dynamics  ideas  such  as  Model-It  (Jackson  et  al.,  1996) 
and  Co-Lab  (van  Joolingen  et  al.,  2005)  were  created. 
These  newer  systems  improved  on  user-friendliness  by 
offering  alternative  ways  of  specifying  the  model,  but 
they  adhere  to  the  same  basic  principle:  The  student 
specifies  a  model  drawn  as  a  graphical  structure  that 
can  be  executed  (simulated),  yielding  outcomes  that 
are  the  consequences  of  the  ideas  expressed  in  the 
model.  Through  all  of  these  developments  we  see  an 
evolution  toward  tools  that  make  it  easier  for  students 
to  create  formal  models. 

A  modeling  activity  starts  from  a  scientific  prob¬ 
lem.  Students  generally  find  it  very  difficult  to  generate 
an  adequate  research  question,  and  they  often  need 
help  in  arriving  at  a  good  research  question  (White  and 
Frederiksen,  1998);  therefore,  students  are  often  pro¬ 
vided  with  an  assignment  that  asks  them  to  model  a 
certain  phenomenon  (van  Joolingen  et  al.,  2005; 
White,  1993).  The  overall  goal  of  a  student  is  to  create 
a  model  in  such  a  way  that  the  behavior  of  the  model 
mimics  the  behavior  of  a  theoretical  model  or  the 
behavior  of  a  real  phenomenon.  Hestenes  (1987) 
described  a  (formal)  model  as  a  mathematical  entity 
that  consists  of  named  objects  and  agents,  variables 
to  define  the  properties  of  these  objects,  equations  that 
describe  the  development  of  variable  values  over  time, 
and  an  interpretation  that  links  the  modeling  concepts 
to  objects  in  the  real  world.  This  characterizes  the 
model  as  a  computational  (runnable,  executable)  entity 
that  can  be  used  for  simulation.  More  recently,  the 
modeling  literature  has  also  included  qualitative  mod¬ 
els  in  which  the  development  of  variable  values  is 
defined  in  terms  of  (qualitative)  relations  rather  than 
equations  (Dimitracopoulou  et  al.,  1999;  Jackson  et 


al.,  1996;  Papaevripidou  et  al.,  2007;  Schwarz  and 
White,  2005;  van  Joolingen  et  al.,  2005),  but  this  does 
not  essentially  change  Hestenes’  conceptualization. 

Hestenes’  (1987)  conceptualization  suggests  that 
to  construct  a  model  students  need  to  iterate  through 
three  types  of  processes:  orientation,  in  which  the 
objects  and  variables  are  identified  and  defined;  spec¬ 
ification,  in  which  the  relations  and  equations  between 
variables  are  specified;  and  evaluation ,  in  which  the 
outcomes  of  the  model  are  interpreted  in  terms  of  the 
real  world  and  matched  to  expectations.  In  orientation, 
the  student  identifies  objects  and  variables  and  makes 
an  initial  sketch  of  the  model;  in  specification,  the 
relations  between  the  variables  are  specified  in  a  qual¬ 
itative  or  quantitative  form  that  allows  computation, 
and  additional  variables  may  be  introduced.  In  evalu¬ 
ation ,  the  model  structure  is  assessed,  model  output  is 
evaluated  against  outcome  expectations,  and  the  model 
output  is  compared  with  observations. 

Initial  evidence  suggests  that  learning  by  modeling 
has  positive  effects  on  the  understanding  of  dynamic 
systems.  Kurtz  dos  Santos  et  al.  (1997)  reported  trans¬ 
fer  from  a  modeled  domain  to  a  new  one.  Schecker 
(1998)  found  that  after  a  mechanics  course  using 
STELLA,  five  out  of  ten  pairs  of  students  were  able 
to  construct  a  qualitative  causal  reasoning  chain  on  a 
new  subject.  Mandinach  (1988)  found  that  modeling 
led  to  better  conceptual  understanding  of  the  content 
and  the  solution  and  an  increase  in  problem-solving 
abilities.  Mandinach  and  Cline  (1996)  noted  a  marked 
improvement  in  students’  inquiry  skills  as  an  effect  of 
modeling.  Schwarz  and  White  (2005)  found  that  stu¬ 
dents  who  had  received  a  modeling  facility  as  part  of 
a  ThinkerTools  (White  and  Frederiksen,  1998)  envi¬ 
ronment  improved  on  an  inquiry  post-test  and  on  far 
transfer  problems.  Papaevripidou  et  al.  (2007)  found 
that  students  who  used  a  modeling  approach  with  a 
modeling  tool  acquired  better  modeling  skills  than  stu¬ 
dents  who  used  a  more  traditional  worksheet  and  were 
also  able  to  model  the  domain  in  an  increasingly 
sophisticated  way. 

Apart  from  these  first  results,  evidence  to  support 
these  claims  of  learning  by  modeling  is  still  scarce, 
especially  when  it  comes  to  experimental  studies  (Loh- 
ner,  2005).  Research  is  limited  to  qualitative  studies 
that  provide  mainly  anecdotal  evidence,  often  with 
only  two  (Resnick,  1994;  Wilensky  and  Reisman, 
2006)  or  even  one  (Buckley,  2000;  Ploger  and  Lay, 
1992)  subject.  Spector  (2001)  attributes  this  lack  of 
focus  on  quantitative  evidence  to  the  fact  that  most 
researchers  in  this  field  believe  that  the  standard  mea¬ 
sures  of  learning  outcomes  are  not  adequate  for  a  seri¬ 
ous  evaluation  of  learning  in  these  environments. 
Although  this  may  be  true,  it  indicates  a  mission  for 
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the  field  to  try  to  implement  instruments  that  actually 
assess  the  knowledge  that  is  acquired  through  learning 
by  modeling. 

An  instrument  to  measure  system  dynamics  think¬ 
ing,  operationalized  as  the  ability  to  interpret  data  in 
terms  of  a  model  and  to  distinguish  a  value  and  its 
rate  of  change,  has  been  developed  by  Booth 
Sweeney  and  Sterman  (2000).  The  focus  of  this 
instrument  is  limited  to  some  basic  skills  in  system- 
dynamics-based  modeling.  Van  Borkulo  and  van  Jool- 
ingen  have  developed  an  instrument  that  aims  to  cover 
the  complete  range  of  knowledge  types  addressed  by 
learning  based  on  the  creation  of  models.  In  their 
overview,  the  different  learning  outcomes  are  opera¬ 
tionalized  into  four  categories  of  test  items,  related 
to  the  kind  of  reasoning  process  for  which  the  knowl¬ 
edge  is  used.  Reproducing  factual  domain  knowledge 
is  the  first  category,  relating  to  the  idea  that  in  mod¬ 
eling  one  acquires  knowledge  about  the  domain. 
Model-based  reasoning  appears  as  applying  a  model 
to  given  situations,  more  specifically  as  predicting 
and  explaining  model  behavior,  by  performing  a  men¬ 
tal  simulation  of  the  model.  Learning  about  modeling 
is  reflected  in  two  categories:  evaluating  a  model  — 
that  is  to  say,  determining  its  correctness  or  suitability 
for  a  given  goal— and  creating  a  model  or  parts  of  it. 
These  four  categories  can  be  evaluated  at  two  levels: 
the  node  level  of  individual  relations  in  a  model  and 
the  structure  level,  in  which  the  effects  of  multiple 
interacting  relations  are  at  stake.  Moreover,  the  cat¬ 
egories  of  apply,  evaluate,  and  create  can  be  consid¬ 
ered  at  both  a  domain-general  and  a  domain-specific 
level.  Initial  tests  with  this  instrument  show  that  it 
can  detect  various  aspects  of  model-based  reasoning. 
Such  instruments  should  eventually  lead  to  system¬ 
atically  collected  evidence  of  the  benefits  of  learning 
by  modeling  as  well  as  more  detailed  knowledge  on 
supporting  modeling  processes. 


MODEL-BASED  INQUIRY  LEARNING 

Much  of  the  modeling  literature  sees  modeling  as  a 
stand-alone  activity.  In  most  of  the  activities  described, 
the  modeling  process  takes  place  in  the  absence  of  data 
that  are  to  be  modeled.  As  such,  modeling  remains  a 
purely  theoretical  activity.  Lohner  et  al.  (2003)  as  well 
as  Schwarz  and  White  (2005)  presented  work  in  which 
models  are  used  to  describe  data  generated  from  a 
given  simulation.  Modeling  thus  becomes  an  inte¬ 
grated  part  of  the  inquiry  process.  In  this  section,  a 
short  description  of  a  specific  learning  environment, 
Co-Lab  (van  Joolingen  et  al.,  2005),  is  presented.  Co- 
Lab  offers  an  environment  in  which  students  can  work 


on  scientific  inquiry  tasks  collaboratively  in  small 
groups  and  in  which  they  are  offered  a  modeling  tool. 
In  Co-Lab,  students  have  the  opportunity  to  explore 
existing  models,  to  create  formal  models  with  a  dedi¬ 
cated  modeling  language  based  on  system  dynamics, 
and  to  compare  their  own  model  outcomes  with  the 
data  generated  by  a  given  simulation  or  collected  from 
an  experiment. 

A  typical  Co-Lab  task  is  to  construct  a  model  of 
a  phenomenon  that  is  found  within  the  environment, 
either  as  a  simulation  or  as  a  remote  laboratory  that 
can  be  controlled  from  a  distance.  In  one  Co-Lab  envi¬ 
ronment,  for  example  ,  students  can  connect  to  a  small 
greenhouse  that  contains  a  plant,  along  with  sensors 
that  measure  the  levels  of  C02,  02,  and  H20,  as  well 
as  the  temperature  and  the  intensity  of  the  light.  The 
goal  for  this  environment  is  to  construct  a  model  that 
describes  the  rate  of  photosynthesis  as  a  function  of 
the  amount  of  available  light.  To  accomplish  this,  stu¬ 
dents  can  use  the  data  obtained  from  the  sensors  and 
manipulate  the  intensity  of  light  by  repositioning  a 
lamp  (specially  made  for  use  in  greenhouses)  and 
determining  when  it  should  be  on  or  off.  They  can  thus 
create  graphs  that  yield  the  photosynthesis  rate  for 
each  level  of  lighting.  Combining  these  results  allows 
them  to  model  the  photosynthesis  rate  as  a  function  of 
the  light  level. 

A  Co-Lab  environment  is  divided  into  different 
buildings,  with  each  building  consisting  of  a  number 
of  floors .  A  building  represents  a  domain  (in  this  case, 
greenhouse  effect),  and  a  floor  represents  a  subdo¬ 
main  (e.g.,  photosynthesis)  or  a  specific  level  of  dif¬ 
ficulty,  similar  to  the  idea  of  model  progression  also 
found  in  SimQuest  (van  Joolingen  and  de  Jong,  2003) 
and  in  earlier  work  by  White  and  Frederiksen  (1990). 
Each  floor  is  composed  of  four  rooms:  the  hall,  a  lab 
room,  a  theory  room,  and  a  meeting  room.  The  pho¬ 
tosynthesis  scenario  in  this  Co-Lab  environment 
starts  in  the  hall,  the  default  entry  room  for  all  Co- 
Lab  environments.  In  the  hall,  students  meet  each 
other  and  find  a  mission  statement  that  explains  the 
goal  of  the  floor  in  the  form  of  a  research  problem 
(e.g.,  creating  a  model  that  explains  the  photosynthe¬ 
sis  rate):  they  also  receive  some  background  infor¬ 
mation  they  need  to  get  started.  After  having  read  this 
mission  statement,  they  can  move  to  the  lab,  in  which 
they  find  a  remote  connection  to  the  greenhouse.  They 
can  see  the  greenhouse  through  a  webcam,  and  they 
can  control  it  and  inspect  the  greenhouse  parameters 
using  a  dedicated  interface.  They  can  start  a  measure¬ 
ment  series  and  plot  the  development  of  the  data  in 
a  graph.  Data  obtained  this  way  can  be  stored  as 
datasets  in  an  object  repository.  In  the  theory  room, 
students  find  a  system  dynamics  modeling  tool  that 
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Figure  36.3  Example  of  a  modeling  tool.  (Courtesy  of  Co-Lab.) 


allows  for  both  qualitative  (relations  such  as  “if  A 
increases  then  B  increases”)  and  quantitative  (equa¬ 
tions)  modeling.  In  the  theory  room,  students  can 
inspect  the  datasets  they  have  stored  in  the  repository 
(which  is  shared  across  rooms)  and  use  these  as  ref¬ 
erence  for  their  model.  This  can  be  done  by  plotting 
model  output  and  observed  data  in  one  graph  and 
comparing  the  two,  or  by  using  the  observed  data  as 
an  element  in  the  model.  Finally,  students  can  plan 
and  monitor  their  work  in  the  meeting  room.  They 
can  review  important  steps  in  the  inquiry  and  mod¬ 
eling  processes,  such  as  planning  experiments  and 
evaluating  models,  using  a  process  coordinator  (Man- 
love  et  al.,  2006).  They  can  make  notes  that  record 
the  history  of  their  learning  process  and  can  eventu¬ 
ally  be  used  as  the  basic  ingredients  for  a  report  that 
they  write  to  conclude  the  activity. 

Co-Lab’s  main  characteristic  is  that  it  combines 
learning  from  models  and  learning  by  modeling  in  one 
environment.  These  activities  take  place  in  the  lab  and 
theory  room,  respectively.  In  doing  so,  the  environment 
offers  opportunities  to  make  the  learning  process  more 
transparent.  Hypotheses  become  visible  as  models  or 
parts  of  models,  their  predictions  can  be  made  visible 
as  model  output,  and  the  validity  of  models  can  be 
assessed  with  reference  to  the  data  collected  from  the 
domain  model  present  in  the  lab.  It  is  also  possible  to 
assess  student’s  models  based  on  a  structural  compar¬ 
ison  of  the  model  with  a  reference  model  (Bravo  et  al., 
2006),  indicating  that  the  domain  model  may  operate 
not  only  as  a  source  of  data  but  also  as  a  resource  for 
tutoring. 


Figure  36.3  shows  an  example  from  the  Co-Lab 
learning  environment.  The  editor  displays  a  model  in 
the  system  dynamics  formalism  that  is  also  used  by 
STELLA  and  PowerS im.  The  graph  shows  the  result 
of  running  this  model  (of  warming  of  the  Earth  under 
the  influence  of  solar  radiation).  In  this  example,  the 
student  has  created  a  model  of  a  physics  topic  (a  black 
sphere  problem)  and  has  run  the  model  to  inspect  its 
behavior.  The  model  is  expressed  in  terms  of  a  graphic 
representation  linking  different  kinds  of  variables,  as 
well  as  equations  or  relations  that  detail  the  behavior 
of  the  model.  The  results  can  be  expressed  as  graphs 
(as  in  Figure  36.3),  tables,  and  animations. 

Co-Lab  has  been  evaluated  in  a  number  of  exper¬ 
imental  studies  focusing  on  specific  aspects  of  the 
environment.  Sins  et  al.  (2007)  found  that  learners 
with  task-oriented  motivation  performed  more  deep 
learning  processes  such  as  changing  and  running  the 
model  with  a  reference  to  prior  knowledge,  which  in 
turn  led  to  better  models.  They  believe  that  the  mode 
of  communication  between  collaborators  (online  chat 
vs.  face  to  face)  influences  the  modeling  process. 
Chatting  learners  used  the  modeling  tool  not  only  as 
a  place  to  construct  the  model  but  also  as  a  means  of 
communication,  resulting  in  many  more  small 
changes  to  the  model  they  were  constructing.  Man- 
love  et  al.  (2006)  found  that  providing  learners  with 
regulative  support  in  the  form  of  a  so-called  process 
coordinator  led  to  better  performance  on  the  model¬ 
ing  task.  This  finding  highlights  the  need  for  instruc¬ 
tional  support  in  this  kind  of  complex  learning  envi¬ 
ronment. 
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CONCLUSIONS 

In  this  chapter,  we  have  discussed  three  modes  of 
learning  in  which  (computer)  models  play  a  pivotal 
role.  One  is  learning  from  models  in  which  students 
gather  knowledge  about  a  model  underlying  a  simula¬ 
tion  through  inquiry  learning.  The  second  one  is  learn¬ 
ing  by  modeling,  in  which  students  learn  by  creating 
models.  Finally,  we  presented  an  example  of  a  system 
in  which  both  ways  of  learning  are  combined,  yielding 
an  integrated  process  of  inquiry. 

Learning  from  models  and  learning  by  modeling 
share  a  number  of  characteristics,  but  there  are  also 
differences.  Both  ways  of  learning  generate  knowledge 
of  the  domain  that  is  involved  in  the  model  (e.g.,  a 
physics  topic  such  as  motion).  Penner  (2001)  asserts 
that  the  main  difference  between  learning  from  models 
in  simulation-based  environments  and  learning  by  mod¬ 
eling  is  that  in  the  first  case  the  underlying  model  stays 
hidden  to  the  students  (they  have  no  direct  access  to  this 
model),  whereas  in  learning  by  modeling  the  exact  char¬ 
acteristics  of  the  model  are  central.  As  a  consequence, 
a  more  intuitive  type  of  knowledge  is  more  likely  to 
evolve  in  learning  from  models  (Swaak  et  al.,  1998), 
whereas  in  learning  by  modeling  more  explicit  concep¬ 
tual  knowledge  is  likely  to  be  acquired  (White  and  Fre- 
deriksen,  2005).  In  both  approaches,  more  general,  pro¬ 
cess-directed  knowledge  is  supposed  to  be  acquired. 
Lohner  (2005),  for  example,  identified  learning  about 
modeling  and  the  modeling  process  as  an  important 
learning  outcome  of  learning  by  modeling.  Learning 
about  modeling  is  seen  as  important  because  science 
and  technology  have  become  increasingly  important  in 
society;  reasoning  with  models,  including  model  con¬ 
struction  as  well  as  awareness  of  the  limitations  of  sci¬ 
entific  models,  is  therefore  seen  as  an  important  part  of 
the  science  curriculum  (Halloun,  1996).  From  model¬ 
ing-based  curricula,  students  should  improve  their  mod¬ 
eling  skills— that  is,  show  effective  modeling  processes 
and  also  obtain  a  better  understanding  of  the  epistemol¬ 
ogy  of  modeling  (Hogan,  1999;  Hogan  and  Thomas, 
2001).  Model-based  reasoning  skills  reflect  the  ability 
to  use  a  model  instrumentally  to  predict  or  explain 
behavior  that  can  be  observed  in  a  modeled  system.  This 
means,  for  example,  being  able  to  predict  the  develop¬ 
ment  of  the  temperature  of  the  atmosphere  under  the 
influence  of  an  increasing  C02  concentration  from  a 
model  of  climate  change  (given  or  self-constructed). 
This  requires  mental  simulation  of  the  model— that  is, 
reasoning  from  the  relations  given  to  projected  values 
of  variables  or  in  the  reverse  direction,  from  observed 
values  of  variables  to  relations  that  explain  these  obser¬ 
vations.  Knowledge  about  performing  sound  scientific 
investigations  is  acquired  in  inquiry  learning.  This 


includes  more  general  skills  such  as  knowing  how  to 
follow  an  inquiry  cycle  (White  and  Frederiksen,  2005) 
as  well  as  more  specific  knowledge  of  experimentation 
heuristics  (Veermans  et  al.,  2006)  or  strategies  on  how 
to  cope  with  anomalous  data  (Lin,  2007).  For  a  more 
complete  overview,  see  Zachos  et  al.  (2000).  These 
inquiry  skills  are  seen  as  important  for  students  to 
become  self-directed  researchers. 

The  learning  processes  involved  also  show  simi¬ 
larities.  The  processes  of  scientific  inquiry  as  we  have 
identified  them  (orientation,  hypothesis  generation, 
experimentation,  and  conclusion)  strongly  resemble 
the  modeling  processes  (orientation,  specification,  and 
evaluation).  There  are,  however,  two  basic  differences. 
First,  a  hypothesis  (or  set  of  hypotheses)  as  present  in 
learning  from  models  does  not  have  to  form  a  runnable 
model.  Second,  in  learning  by  modeling  experimenta¬ 
tion  is  not  necessary  to  gather  data  for  creating  a 
model;  instead,  students  can  gather  their  information 
from  many  sources  and  use  this  as  input  for  creating 
their  model.  One  of  the  assumptions  underlying  the 
combination  of  learning  from  models  and  learning  by 
modeling  (as  in  the  Co-Lab  environment)  is  that  both 
approaches  can  reinforce  each  other.  Evidence  for  this 
claim  can  be  found  in  Schwarz  and  White  (2005).  They 
found  that  students  who  received  a  modeling  facility 
in  ThinkerTools  also  improved  on  a  test  of  inquiry 
skills.  A  comparison  of  the  modeling-enhanced 
Thinker-Tools  curriculum  with  a  traditional  Thinker- 
Tools  curriculum  showed  no  overall  differences  in  an 
inquiry  skills  test  except  for  a  subscale  measuring  the 
students’  ability  to  formulate  conclusions.  A  correla¬ 
tional  analysis  of  students  who  followed  the  Thinker- 
Tools  curriculum  with  a  modeling  facility  showed  that 
at  the  pretest  there  were  no  significant  correlations 
among  knowledge  of  modeling,  inquiry,  and  physics. 
At  the  post-test,  however,  these  three  tests  correlated 
significantly,  indicating  that  development  in  each  of 
these  three  knowledge  areas  is  mutually  reinforcing. 

Whatever  approach  is  chosen,  it  is  clear  that  stu¬ 
dents  cannot  perform  inquiry,  modeling,  or  a  combi¬ 
nation  of  the  two  without  scaffolding  (Klahr  and 
Nigam,  2004;  Mayer,  2004).  In  inquiry  research,  a 
large  set  of  cognitive  tools  for  inquiry  has  now  been 
developed  (for  recent  overviews,  see  de  Jong,  2006a,b; 
Linn  et  al.,  2004;  Quintana  et  al.,  2004).  Comparable 
scaffolds  for  modeling  are  only  now  about  to  emerge 
(Bravo  et  al.,  2006).  Integrated  environments  such  as 
Co-Lab,  in  which  different  approaches  are  combined 
and  in  which  learning  processes  are  scaffolded  by  col¬ 
laboration  and  by  an  extensive  set  of  cognitive  tools, 
may  provide  students  with  learning  opportunities  that 
help  them  gain  both  (intuitive  and  formal)  domain  and 
general  process  knowledge. 
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ABSTRACT 

Adaptive  instruction  embodies  all  instructional  forms 
that  accommodate  the  needs  and  abilities  of  different 
learners.  This  chapter  summarizes  five  approaches  to 
adaptive  instruction:  (1)  macro-adaptive  instruction; 
(2)  aptitude-treatment  interactions  (ATI-based);  (3) 
micro-adaptive  instruction,  including  intelligent  tutor¬ 
ing  systems  (ITSs);  (4)  the  adaptive/adaptable  hyper¬ 
media/Web-based  system  (AHS);  and  (5)  specific  ped¬ 
agogy-centered  systems.  These  approaches  are 
presented  in  historical  order,  beginning  with  macro- 
adaptive  systems.  For  each  approach,  its  characteristics 
and  representative  systems  are  discussed.  Although 
each  has  its  own  distinctive  properties,  some  similar¬ 
ities  can  be  found  among  the  approaches.  Due  to  the 
development  of  information  and  communication  tech¬ 
nology  (ICT),  the  structural  functions  of  recent  adap¬ 
tive  systems  are  significantly  more  powerful  than  ear¬ 
lier  ones.  New  Web-based  systems  have  functions  that 
simultaneously  provide  customized  learning  experi¬ 
ences  to  masses  of  individual  learners.  The  challenges 
now  facing  researchers  and  developers  are  to  optimally 
integrate  many  different  theories,  principles,  and  strat¬ 
egies  of  learning  and  instruction  with  system  functions 
and  to  prove  empirically  the  effects  and  value  of  these 
systems  in  real-world  environments. 


KEYWORDS 

Adaptive  hypermedia  systems  (AHSs):  Combining 
micro-adaptive  systems  and  hypermedia  systems  to 
provide  adaptive/adaptable,  hybrid  features  by  pre¬ 
senting  learners  with  choices,  along  with  guidance. 

Adaptive  instructional  systems:  Any  forms  of  educa¬ 
tional  intervention  aimed  at  accommodating  indi¬ 
vidual  learner  differences. 

Aptitude-treatment  interactions  (ATIs):  Adapting  spe¬ 
cific  instructional  procedures  and  strategies  to  spe¬ 
cific  learner  characteristics  (or  aptitudes). 

Macro-adaptive  systems:  Allowing  different  alterna¬ 
tives  for  choosing  instructional  goals,  curriculum 
content,  and  delivery  systems, by  grouping  students. 

Micro-adaptive  systems:  Diagnosing  the  learner’s  spe¬ 
cific  learning  needs  during  instruction  and  provid¬ 
ing  instructional  prescriptions  for  the  needs. 


DEFINITION 

A  central  and  persistent  issue  in  educational  technol¬ 
ogy  is  the  planning  and  provision  of  instructional  envi¬ 
ronments  and  conditions  that  fit  and  support  individu¬ 


ally  different  educational  goals  and  learning  abilities 
(Park,  1996).  In  general,  instructional  approaches  and 
techniques  that  are  geared  to  meet  the  needs  of  indi¬ 
vidually  different  students  in  developing  knowledge 
and  skills  required  to  learn  a  task  are  called  adaptive 
instruction  (Como  and  Snow,  1986).  Accordingly,  any 
form  of  instruction  is  adaptive,  whether  it  is  delivered 
by  teachers  or  in  a  technology-based  format,  if  it 
accommodates  different  student  learning  needs  and 
abilities. 

Adaptive  instruction  has  a  long  history  and  has 
been  implemented  in  various  forms  and  settings,  from 
group-based,  classroom  instruction  to  Web-based, 
open  space  instruction.  The  development  of  computer 
technology  has  provided  a  powerful  tool  for  develop¬ 
ing  and  implementing  sophisticated  instructional  sys¬ 
tems  from  diagnostic  assessment  tools  to  tutoring  sys¬ 
tems  generating  individually  tailored  instructional 
prescription.  Recent  advances  in  information  and  com¬ 
munication  technology  (ICT)  allow  for  the  delivery  of 
individually  customized  information  and  instruction  to 
mass  audiences  simultaneously.  This  mass  individual¬ 
ization  has  been  increasingly  popular  and  important  in 
education  and  training  communities  (De  Bra  et  al., 
2004;  Karagiannidis  et  al.,  2001;  van  Merrienboer, 
2005). 


HISTORY  OF  ADAPTIVE 
INSTRUCTIONAL  SYSTEMS 

The  long  history  of  efforts  in  adapting  instruction  to 
an  individual  student’s  needs  and  abilities  has  been 
documented  by  many  researchers  (Corno  and  Snow, 
1986;  Federico  et  al.,  1980;  Glaser,  1977;  Reiser,  1987; 
Tobias,  1989;  Wang  and  Lindvall,  1984).  Since  at  least 
the  fourth  century  B.C.,  adapting  has  been  viewed  as 
a  primary  requirement  for  successful  instruction 
(Corno  and  Snow,  1986),  and  adaptive  tutoring  was 
the  common  instructional  method  until  the  mid- 1800s 
(Reiser,  1987).  Even  after  graded  systems  were 
adopted,  the  importance  of  adapting  instruction  to  indi¬ 
vidual  needs  was  continuously  emphasized.  Dewey, 
for  example,  in  his  1902  essay.  Child  and  Curriculum , 
deplored  the  then  current  emphasis  on  a  single  kind  of 
curriculum  development  that  produced  a  uniform, 
inflexible  sequence  of  instruction  (Dewey,  1902/1964). 
Thorndike  (1911)  argued  for  a  specialization  of 
instruction  that  acknowledged  differences  among 
pupils  within  a  single  class  as  well  as  specialization 
of  the  curriculum  for  different  classes.  Since  then, 
various  approaches  and  methods  have  been  proposed 
to  provide  adaptive  instruction  to  individually  different 
students  (for  early  systems,  see  Reiser,  1987). 
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Since  Cronbach  (1957)  declared  that  a  united  dis¬ 
cipline  of  psychology  would  be  interested  in  not  only 
organisms  but  also  interactions  between  organisms 
and  treatment  variables,  numerous  studies  have  been 
conducted  to  investigate  what  kinds  of  student  char¬ 
acteristics  should  be  considered  in  adapting  instruc¬ 
tion  and  how  instructional  methods  and  procedures 
should  be  adapted  to  those  characteristics  (Cronbach, 
1967;  Cronbach  and  Snow,  1977;  Federico  et  al., 
1980;  Snow  and  Swanson,  1992).  Whereas  early 
instructional  systems  considered  only  one  or  two 
variables,  newer  adaptive  systems  using  computer 
technology  implement  models  that  have  multiple  lay¬ 
ers  ,  each  with  many  variables .  Each  adaptive  system 
lies  along  this  complexity  continuum,  but  they  can 
be  clustered  into  several  types  according  to  their 
approaches. 

DIFFERENT  APPROACHES  TO 
ADAPTIVE  INSTRUCTIONAL  SYSTEMS 

Efforts  to  develop  and  implement  adaptive  instruction 
have  taken  five  different  approaches.  Depending  on 
available  resources  and  constraints  in  the  given  situ¬ 
ation,  adaptive  instruction  can  be  designed  using  one 
or  more  of  these  approaches.  The  first  approach  is  to 
adapt  instruction  on  a  macro-level  by  allowing  alter¬ 
natives  in  instructional  goals,  depth  of  curriculum 
content,  delivery  systems,  etc.  Most  adaptive  instruc¬ 
tional  systems  developed  as  alternatives  to  the  tradi¬ 
tional  lock-step  group  instruction  in  school  environ¬ 
ments  have  taken  this  approach.  The  second  approach 
is  to  adapt  specific  instructional  procedures  and  strat¬ 
egies  to  specific  student  characteristics.  Because  this 
approach  requires  the  identification  of  the  most  rel¬ 
evant  learner  characteristics  (or  aptitudes)  for  instruc¬ 
tion  and  the  selection  of  instructional  strategies  that 
best  facilitate  the  learning  process  of  the  students 
with  those  characteristics,  it  is  based  on  apti¬ 
tude-treatment  interaction  (ATI).  The  third  approach 
is  to  adapt  instruction  on  a  micro-level  by  diagnosing 
the  student’s  specific  learning  needs  during  instruc¬ 
tion  and  providing  instructional  prescriptions  for  the 
needs.  Intelligent  tutoring  systems  (ITSs)  are  an 
example  of  this  approach.  The  fourth  approach  is 
adaptive  hypermedia  and  Web-based  systems  (AHSs). 
Although  this  approach  can  be  considered  as  an 
extension  of  ITS,  it  has  several  new  features.  First, 
most  AHSs  apply  adaptive/adaptable  (hybrid)  fea¬ 
tures  by  allowing  users  to  initiate  their  choices ,  along 
with  guidance  (Cristea  and  Garzotto,  2004).  Second, 
whereas  previous  adaptive  systems  are  closed  corpus 
systems  confined  to  the  program,  most  AHS  applica¬ 


tions  are  Web-based,  open  corpus  systems  which 
allows  the  possibility  of  utilizing  other  Web 
resources.  The  fifth  is  a  group  of  systems  that  were 
developed  based  on  specific  pedagogical  approaches. 
The  pedagogical  approaches  applied  in  these  systems 
include  constructivism,  motivation  theory,  social 
learning  theory,  and  metacognition. 

Although  they  are  presented  as  five  different 
approaches,  some  overlaps  exist  between  categories. 
Further,  an  adaptive  instructional  system  may  contain 
characteristics  of  more  than  one  approach.  Also, 
although  this  review  is  intended  to  be  comprehensive, 
it  is  certainly  not  complete  .  Several  relevant  endeavors 
in  the  held— for  example,  applications  of  cognitive 
load  theory  to  adaptive  instruction— are  not  included 
in  this  review  because  these  systems  are  in  the  early 
stages  of  development. 


MACRO-ADAPTIVE  INSTRUCTION 

Initial  attempts  at  homogeneous  grouping  had  minimal 
effect  because  the  groups  seldom  received  different 
kinds  of  instructional  treatments  (Tennyson,  1975).  In 
the  early  1900s,  a  number  of  adaptive  systems  were 
developed  to  better  accommodate  different  student 
abilities,  such  as  the  Burke  plan,  Dalton  plan,  and 
Winnetka  plan  (Reiser,  1987).  The  notion  of  mastery 
learning  was  also  fostered  in  the  Dalton  and  Winnetka 
plans. 

Several  macro-adaptive  instructional  systems 
were  developed  in  the  1960s,  1970s,  and  1980s  (for 
a  review,  see  Park  and  Lee,  2003).  Examples  of 
macro-adaptive  instructional  systems  include 
Keller’s  Personalized  System  of  Instruction  (PSI) 
(Keller  and  Sherman,  1974),  the  Program  for  Learn¬ 
ing  in  Accordance  with  Needs  (PLAN)  (Flanagan  et 
al.,  1975),  Mastery  Learning  Systems  developed  by 
Bloom  and  his  associates  (Block,  1980),  Individually 
Guided  Education  (IGE)  (Klausmeier,  1975),  and 
Individually  Prescribed  Instructional  System  (IPI) 
(Glaser,  1977).  Although  many  macro-level  systems 
have  been  criticized  as  being  unsystematic,  they  were 
practiced  in  many  school  classrooms  for  a  long  time, 
and  some  systems  are  still  used.  The  Adaptive  Learn¬ 
ing  Environments  Model  (ALEM)  developed  by 
Wang  and  her  associates  (Wang,  1980;  Wang  and 
Lindvall,  1984;  Wang  et  al.,  1995;  see  also  http:// 
www.nwrel.org/scpd/catalog/ModelDetails.asp? 
ModelID=8)  is  still  implemented  in  many  schools. 
Another  sample  of  macro-level  approaches  is  the 
PLATO  Learning  Management  (PLM)  system;  PLM 
was  a  computer-managed  instructional  (CMI)  system 
with  functions  to  diagnose  student  learning  needs  and 
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prescribe  instructional  activities  appropriate  for  those 
needs.  The  PLM  could  evaluate  each  student’s  per¬ 
formance  on  a  test  and  provide  specific  instructional 
prescriptions  (Hart,  1981). 

As  Glaser  (1977)  pointed  out,  the  development  and 
implementation  of  an  adaptive  instructional  program 
in  an  existing  system  were  complex  and  difficult.  This 
problem  may  be  the  primary  reason  why  most  macro- 
adaptive  instructional  systems  have  not  been  used  as 
successfully  and  widely  as  hoped.  Computer  technol¬ 
ogy,  however,  provides  a  powerful  means  to  overcome 
at  least  some  of  the  problems  encountered  in  the  devel¬ 
opment  and  implementation  of  adaptive  instructional 
systems. 

APTITUDE-TREATMENT 

INTERACTIONS 

Cronbach  (1957)  suggested  that  facilitating  educa¬ 
tional  development  for  a  wide  range  of  students 
requires  a  wide  range  of  environments  suited  to  their 
optimal  learning.  He  proposed  prescribing  one  type 
of  instructional  sequence  for  a  student  with  certain 
characteristics  and  another  type  for  a  student  with 
different  characteristics .  This  strategy  has  been  based 
on  aptitude-treatment  interactions  (ATIs).  Cronbach 
and  Snow  (1977)  defined  aptitude  as  any  individual 
characteristic  that  increases  or  impairs  the  student’s 
probability  of  success  in  a  given  treatment,  and  they 
defined  treatment  as  variations  in  the  pace  or  style  of 
instruction. 

An  Eight-Step  Model  for  Designing 
ATI-Based  Courseware 

Carrier  and  Jonassen  (1988)  proposed  an  eight-step 
model  to  provide  practical  guidance  for  applying  the 
ATI-based  model  to  the  design  of  computer-based 
instructional  (CBI)  courseware.  This  model  is  basi¬ 
cally  a  modified  systems  approach  to  instructional 
development  (Dick  and  Carey,  1985;  Gagne  and 
Briggs,  1979):  (1)  Identify  objectives  for  the  course¬ 
ware,  (2)  specify  task  characteristics,  (3)  identify  an 
initial  pool  of  learner  characteristics,  (4)  select  the 
most  relevant  learner  characteristics,  (5)  analyze 
learners  in  the  target  population,  (6)  select  final  dif¬ 
ferences  (in  the  learner  characteristics),  (7)  determine 
how  to  adapt  instruction,  and  (8)  design  alternative 
treatments.  Carrier  and  Jonassen  (1988)  also  listed 
important  individual  variables  that  influence  learning, 
such  as  prior  knowledge,  cognitive  styles,  and  per¬ 
sonality  variables.  For  instructional  adaptation,  they 
recommended  several  types  of  methods:  (1)  remedial. 


(2)  capitalization/  preferential,  (3)  compensatory,  and 
(4)  challenge.  This  model  seemingly  has  practical 
value;  however,  without  theoretically  coherent  and 
empirically  traceable  links  among  different  learner 
variables  and  without  clearly  defined  types  and  levels 
of  learning  requirements  and  instructional  strategies 
for  different  tasks,  the  mere  application  of  this  model 
is  not  likely  to  produce  better  results  than  those  of 
nonadaptive  instructional  systems. 

Limitations  of  ATI 

Since  Cronbach  (1957)  made  his  proposal,  relatively 
few  studies  have  found  consistent  results  to  support 
the  paradigm.  As  shown  in  several  reviews  of  ATI 
research  (Berliner  and  Cahen,  1973;  Corno  and 
Snow,  1986;  Cronbach  and  Snow,  1977;  Tobias, 
1976),  intellectual  abilities  and  other  aptitude  vari¬ 
ables  were  used  in  many  different  studies  to  investi¬ 
gate  their  interactions  with  a  variety  of  instructional 
treatments;  however,  no  convincing  evidence  was 
found  to  suggest  that  such  individual  differences 
were  useful  variables  for  differentiating  instructional 
treatments  in  a  homogeneous  age  group  (Glaser  and 
Resnick,  1972;  Tobias,  1987). 

The  inconsistent  findings  of  ATI  studies 
prompted  researchers  to  reexamine  the  paradigm  and 
propose  alternative  approaches.  According  to  Tobias 
(1976, 1987, 1989),  a  number  of  limitations  and  prob¬ 
lems  in  the  ATI-based  model  have  been  proposed;  for 
example: 

•  The  abilities  assumed  to  be  most  effective 
for  a  particular  treatment  may  not  be  exclu¬ 
sive  (Cronbach  and  Snow,  1977). 

•  Abilities  required  by  a  treatment  may  shift 
as  the  task  progresses  (Burns,  1980;  Feder¬ 
ico,  1983). 

•  ATIs  validated  for  a  particular  task  and  sub¬ 
ject  area  may  not  be  generalizable  to  other 
areas  (Peterson,  1977;  Peterson  and  Janicki, 
1979;  Peterson  et  al.,  1981). 

•  ATIs  validated  in  laboratory  experiments 
may  not  be  applicable  to  actual  classroom 
situations  (Tobias,  1976,  1987,  1989). 

Another  criticism  is  that  ATI  research  has  tended  to  be 
overly  concerned  with  exploration  of  simple  input/out¬ 
put  relations  between  measured  traits  and  learning  out¬ 
comes  (DiVesta,  1975).  Because  individual  difference 
variables  are  difficult  to  measure,  the  validity  and  reli¬ 
ability  of  the  measures  can  be  a  problem  in  adapting 
instruction  to  the  individual  differences;  however, 
research  in  ATI  has  continued. 
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Recently,  several  studies  have  been  conducted  to 
control  learning  environments  using  computers;  for 
example,  Maki  and  Maki  (2002)  examined  the  inter¬ 
actions  of  student  comprehension  skills  with  course 
format  (online  vs.  lecture  format) .  After  examining  the 
interactions  among  particular  learning  types,  explor¬ 
atory  behavior,  and  two  different  learning  environ¬ 
ments  (rule  application  vs.  rule  induction),  Shute  and 
Towle  (2003)  introduced  an  adaptive  e-learning  model 
utilizing  ATI. 


MICRO-ADAPTIVE  INSTRUCTION 

Researchers  have  attempted  to  establish  micro-adap¬ 
tive  instructional  models  using  on-task  measures  rather 
than  pre-task  measures.  On-task  measures  of  student 
behavior  and  performance,  such  as  response  errors, 
response  latencies,  and  emotional  states,  can  be  valu¬ 
able  sources  for  making  adaptive  instructional  deci¬ 
sions  during  the  instructional  process.  Such  measures 
taken  during  the  course  of  instruction  can  be  applied 
to  the  manipulation  and  optimization  of  instructional 
treatments  and  sequences  on  a  much  more  refined  scale 
(Federico,  1983).  Thus,  micro-adaptive  instructional 
models  using  on-task  measures  are  likely  to  be  more 
sensitive  to  the  student’s  needs. 

A  typical  example  of  micro-adaptive  instruction  is 
one-on-one  tutoring.  The  tutor  selects  the  most  appro¬ 
priate  information  and  tutoring  method  for  the  student 
based  on  his  or  her  judgment  of  the  student’s  learning 
needs  and  abilities  (Bloom,  1984;  Kulik,  1982).  As  the 
one-on-one  tutorial  process  suggests,  two  essential  ele¬ 
ments  of  micro- adaptive  instruction  are  the  ongoing 
diagnosis  of  the  student’s  learning  needs  and  the  pre¬ 
scription  of  instructional  treatments  based  on  the  diag¬ 
nosis  (Hansen  et  al.,  1977;  Holland,  1977;  Landa,  1976; 
Rothen  and  Tennyson,  1978).  Instructional  researchers 
or  developers  have  different  views  about  the  variables, 
indices,  procedures,  and  actions  that  should  be  used  in 
the  diagnostic  and  the  prescriptive  processes  (Atkinson, 
1976;  Rothen  and  Tennyson,  1977). 

Micro-Adaptive  Instructional  Models 

Unlike  macro-adaptive  models,  micro-adaptive  models 
are  dynamic  and  use  the  temporal  nature  of  learner 
abilities  and  characteristics  (e.g.,  current  knowledge, 
motivation  level)  as  a  major  source  of  diagnostic  infor¬ 
mation  on  which  instructional  treatments  are  pre¬ 
scribed.  By  including  more  variables  related  to  instruc¬ 
tion,  a  typical  micro-adaptive  model  provides  a  better 
control  process  than  a  macro-adaptive  model  or  pro¬ 
grammed  instruction  in  responding  to  the  student’s 


learning  needs  (Merrill  and  Boutwell,  1973).  As 
described  by  Suppes  et  al.  (1976),  micro-adaptive 
models  typically  use  a  quantitative  representation  and 
trajectory  methodology.  An  important  feature  of  a 
micro-adaptive  model  is  the  timeliness  and  accuracy 
for  determining  and  adjusting  learning  prescriptions 
during  instruction. 

Most  micro-adaptive  models  are  primarily  devel¬ 
oped  to  adapt  two  instructional  variables:  the  amount 
of  content  to  be  presented  and  the  presentation 
sequence  of  the  content.  Representative  examples  of 
micro-adaptive  instructional  models  are  mathemati¬ 
cal  models,  multiple  regression  models,  Bayesian 
probability  models,  and  structural/algorithmic  mod¬ 
els  (for  a  comprehensive  review,  see  Park  and  Lee, 
2003).  The  Bayesian  probabilistic  model  and  the  mul¬ 
tiple  regression  model  are  designed  to  select  the 
amount  of  instruction  required  to  learn  a  given  task 
using  both  pre-task  and  on-task  information  (Hansen 
et  al.,  1977;  Park  and  Tennyson,  1980,  1986;  Ross 
and  Anand,  1986;  Ross  and  Morrison,  1986;  Rothen 
and  Tennyson,  1977).  The  structural/algorithmic 
approach  emphasizes  that  the  sequence  of  instruction 
should  be  decided  by  the  content  structure  of  the 
learning  task  as  well  as  the  student’s  performance 
history  (Scandura,  1977a,b,  1983). 

Regarding  treatment  variables,  some  studies 
(Hansen  et  al.,  1977;  Ross  and  Morrison,  1988)  indi¬ 
cated  that  only  prior  achievement  among  pre-task  mea¬ 
sures  (e.g.,  anxiety,  locus  of  control)  provides  consis¬ 
tent  and  reliable  information  for  prescribing  the 
amount  of  instruction;  however,  subjects  who  received 
the  amount  of  instruction  selected  based  on  both  pre¬ 
task  measures  and  on-task  measures  needed  less  time 
and  showed  higher  test  scores  than  subjects  who 
received  the  amount  of  instruction  based  on  only  pre¬ 
task  measures  (Park  and  Tennyson,  1980).  The  results 
of  the  response-sensitive  strategies  studied  by  Park  and 
Tennyson  (1980,  1986)  suggest  that  the  predictive 
power  of  pre-task  measures,  including  prior  knowl¬ 
edge,  decreases  while  that  of  on-task  measures 
increases  as  the  instruction  progresses. 

As  reviewed  above,  a  common  characteristic  of 
micro-adaptive  instructional  models  is  response  sensi¬ 
tivity  in  diagnosing  student  learning  needs  and  provid¬ 
ing  instructional  prescriptions.  Response-sensitive 
instruction  has  been  used  for  a  long  time,  from  Crow¬ 
der’s  simple  branching  program  (1959)  and  Atkinson’s 
mathematical  model  of  adaptive  instruction  (1968)  to 
intelligent  tutoring  systems.  Until  the  late  1960s,  tech¬ 
nology  was  not  readily  available  to  implement  the 
response-sensitive  diagnostic  and  prescriptive  proce¬ 
dures  as  a  general  practice  outside  the  experimental 
laboratory  (Hall,  1977). 
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Intelligent  Tutoring  Systems 

Intelligent  tutoring  systems  (ITSs)  were  developed 
with  the  application  of  artificial  intelligence  (AI) 
techniques.  Because  the  goal  of  ITSs  is  to  provide 
adaptive  instruction  by  intelligently  diagnosing  stu¬ 
dents’  learning  needs  and  progress  in  a  response- 
sensitive  manner,  they  are  considered  micro-level 
adaptive  instructional  systems.  ITSs  have  three  main 
components:  a  representation  of  the  content  to  be 
taught  (expert  or  domain  model),  an  inherent  teach¬ 
ing  or  instructional  strategy  (tutor  or  teaching 
model),  and  mechanisms  for  understanding  what  the 
student  does  and  does  not  know  (student  model) 
(Akhras  and  Self,  2002;  Shute  and  Psotka,  1996; 
Wenger,  1987).  AI  methods  for  the  representation  of 
knowledge  (e.g.,  production  rules,  semantic  net¬ 
works ,  scripts  frames)  and  problems  make  it  possible 
for  the  ITS  to  generate  and  present  knowledge  to 
students  based  on  their  performance  on  the  task 
rather  than  selecting  the  knowledge  according  to  the 
predetermined  branching  rules.  The  capacity  to  make 
inferences  about  the  cause  of  a  student’s  misconcep¬ 
tions  and  his  or  her  learning  needs  allows  the  ITS 
to  make  decisions  based  on  qualitative  data,  unlike 
most  micro-adaptive  models  based  on  solely  quan¬ 
titative  data. 

Furthermore,  ITS  techniques  provide  a  powerful 
tool  for  effectively  capturing  human  learning  and 
teaching  processes.  It  has  apparently  contributed  to 
a  better  understanding  of  cognitive  processes 
involved  in  learning  specific  skills  and  knowledge 
(Shute  and  Psotka,  1996).  Some  ITSs  provided 
research  environments  for  investigating  specific 
instructional  strategies  and  tools  for  modeling  human 
tutors  and  simulating  human  learning  and  cognition 
(Koedinger  and  Anderson,  1998;  Seidel  and  Park, 
1994;  Shute  and  Psotka,  1996).  Recently,  ITSs  were 
expanded  to  enhance  metacognition  (Aleven  et  al., 
2001;  White  et  al.,  1999);  however,  it  has  been  noted 
that  ITS  developers  have  failed  to  incorporate  many 
valuable  learning  principles  and  instructional  strate¬ 
gies  developed  by  instructional  researchers  and  edu¬ 
cators  (Park  et  al.,  1987).  Ohlsson  (1987,  1993)  and 
others  criticized  ITS  and  other  computer-based  inter¬ 
active  learning  systems  for  their  limited  range  and 
adaptability  of  teaching  actions  compared  to  rich  tac¬ 
tics  and  strategies  employed  by  human  expert  teach¬ 
ers.  Cooperative  efforts  among  experts  in  different 
domains ,  including  learning/instruction  and  artificial 
intelligence,  are  required  to  develop  more  powerful 
adaptive  systems  using  the  ITS  methods  and  tech¬ 
niques  (Park  and  Seidel,  1989;  Seidel  et  al.,  1989). 


Applying  ATI  to  Micro-Level 
Adaptive  Systems 

To  integrate  the  ATI-based  approach  in  a  micro-adaptive 
model,  Tennyson  and  Christensen  (1988;  also  see  Ten¬ 
nyson  and  Park,  1987)  proposed  a  two-level  model  of 
adaptive  instruction  based  on  the  findings  of  their  own 
research.  First,  this  computer-based  model  allows  the 
computer  tutor  to  establish  conditions  of  instruction 
based  on  learner  aptitude  variables  (cognitive,  affective, 
and  memory  structure)  and  context  structure.  Second, 
the  computer  tutor  provides  moment-to-moment  adjust¬ 
ments  of  instructional  conditions  by  adapting  the 
amount  of  information,  example  formats,  display  time, 
sequence  of  instruction,  instructional  advisement,  and 
embedded  refreshment  and  remediation.  The  micro- 
level  adaptation  is  based  on  the  student’s  on-task  per¬ 
formance,  and  the  procedure  is  response  sensitive  (Park 
and  Tennyson,  1980).  The  amount  of  information  to  be 
presented  and  the  time  to  display  the  information  on  the 
computer  screen  are  determined  through  the  continuous 
decision-making  process  based  on  on-task  performance 
data.  The  selection  and  presentation  of  other  instruc¬ 
tional  strategies  (sequence  of  examples,  advisement, 
and  embedded  refreshment  and  remediation)  are  deter¬ 
mined  based  on  the  evaluation  of  the  on-task  perfor¬ 
mance. 

Evidence  shows  that  some  aptitude  variables  (e.g., 
prior  knowledge,  interest,  intellectual  ability)  are  impor¬ 
tant  predictors  in  selecting  instructional  treatments 
(Tobias,  1994;  Whitener,  1989);  however,  some  studies 
(Park  and  Tennyson,  1980,  1986)  suggest  that  the  pre¬ 
dictive  value  of  aptitude  variables  decreases  as  the  learn¬ 
ing  process  continues.  In  contrast,  the  diagnostic  power 
of  on-task  performance  increases  because  it  reflects  the 
most  updated  and  integrated  reflection  of  aptitude  and 
other  variables  involved  in  the  learning.  The  decrease 
in  the  predictive  power  of  the  premeasured  aptitude 
variables  and  the  increase  in  that  of  on-task  performance 
can  be  represented  as  shown  in  Figure  37.1 . 


Figure  37.1  Predictive  power  of  aptitudes  and  on-task  perfor¬ 
mance. 
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In  the  two-stage  approach,  the  student  is  assigned 
initially  to  the  best  instructional  alternative  based  on 
the  aptitude  measured  prior  to  instruction,  and  then 
response-sensitive  procedures  are  applied  as  the  stu¬ 
dent’s  response  patterns  emerge  to  reflect  his  or  her 
knowledge  or  skills  on  the  given  task.  A  representative 
example  of  this  two-stage  approach  is  the  Bayesian 
adaptive  instructional  model.  As  the  process  for  esti¬ 
mating  student  learning  needs  continues  using  a  Baye¬ 
sian  probability  model,  the  value  of  the  pretest  perfor¬ 
mance  data  becomes  less  important  and  recent 
performance  data  become  more  important.  The  effect- 
ness  of  the  two-stage  approach  has  been  empirically 
supported  (Park  and  Tennyson,  1980,  1986). 

ADAPTIVE  HYPERMEDIA  SYSTEMS 

In  the  early  1990s,  when  researchers  incorporated  the 
concepts  of  hypermedia/hypertext  with  intelligent 
tutoring  systems,  adaptive  hypermedia  systems 
(AHSs)  were  born  (Beaumont,  1994;  Brusilovsky  et 
al.,  1996;  De  Bra  et  al.,  2005;  Fischer  et  al.,  1990; 
Gonschorek  and  Herzog,  1995;  Hohl  et  al.,  1996;  Kay 
and  Kummerfeld,  1994).  An  AHS  utilizes  hyperlinks, 
and  its  technical  approach,  including  a  user-model- 
based  interface,  is  similar  to  an  ITS;  however,  AHSs 
differ  from  both  micro-adaptive  systems  (including 
ITSs)  and  conventional  hypermedia  systems  in  terms 
of  adaptability/adaptivity. 

In  general,  computer-based  instructional  systems 
that  allow  the  user  to  choose  certain  parameters  and 
adapt  the  behavior  of  these  systems  are  referred  to  as 
learner-controlled  instruction  (Williams,  1996)  or 
adaptable  systems  (Cristea  and  Garzotto,  2004;  Fink 
et  al.,  1998;  Opperman,  1994),  whereas  adaptive  sys¬ 
tems  in  which  behaviors  are  automatically  based  on 
the  user’s  needs  are  referred  to  as  program-controlled 
(system-controlled)  instruction ,  or  adaptive  systems* 

Conventional  hypermedia  learning  environments 
are  nonadaptive,  as  they  provide  the  same  content  and 
the  same  set  of  links  to  all  learners;  however,  they  are 
adaptable  because  links  or  tasks  to  be  presented  next 
are  determined  by  the  learner’s  actions.  Adaptability 
without  adaptivity  may  lead  the  learner  along  a  poor 
path  (Steinberg,  1991;  Williams,  1996).  Unlike  con¬ 
ventional  hypermedia  environments,  an  AHS  is  adap¬ 
tive,  providing  content  and  links  based  on  the  individ¬ 
ual  user’s  needs  and  characteristics;  for  example,  the 


*  These  definitions  of  the  words  adaptive  and  adaptable  are  not 
employed  universally;  for  example,  Leutner  (2004)  referred  to  learning 
environments  that  are  macro- adaptive  as  adaptable  and  those  that  are 
micro-adaptive  as  adaptive. 


next  button  in  an  AHS  may  not  be  different  in  appear¬ 
ance  but  it  will  take  different  users  to  different  pages 
(Schwarz  et  al.,  1996). 

Traditional  micro-adaptive  instructional  systems 
are  adaptive  but  not  adaptable.  They  control  the 
instructional  process  and  activities  based  on  an  evalu¬ 
ation  of  student  learning  needs  and  abilities,  but  they 
do  not  allow  students  to  control  the  process  of  adap¬ 
tation.  In  contrast,  most  AHSs  allow  users  to  initiate 
their  own  choices;  for  example,  an  AHS  might  provide 
a  suggested  set  of  most  relevant  links  based  on  the 
system’s  assessment  of  the  user’s  needs  and  character¬ 
istics  (Brusilovsky,  1994). 

Because  an  AHS  includes  both  learner  and  pro¬ 
gram  control,  it  is  also  referred  to  as  an  adaptive/ 
adaptable  hybrid  system  (Cristea  and  Garzotto,  2004). 
Oppermann  developed  and  tested  the  adaptive/adapt¬ 
able  system  Flexcel.  He  concluded  that  adaptivity  and 
adaptability  should  receive  benefits  from  each  other 
(Oppermann,  1994). 

Another  difference  between  AHSs  and  micro- 
adaptive  systems  is  that,  except  for  a  few  early  AHSs 
developed  before  the  advent  of  the  Web ,  AHSs  are  Web 
based.  As  Brusilovsky  (2003)  pointed  out,  when  using 
the  Web  platform  course  instructors  or  developers  can 
use  not  only  the  closed  corpus  course  material  in  the 
system  but  also  open  corpus  Web  sources.  Open  corpus 
Web  sources  were  not  available  for  previous  adaptive 
systems. 

Since  the  mid-1990s,  with  the  advent  of  the  Web, 
AHSs  have  grown  rapidly  (Brusilovsky,  2001).  Now, 
the  Web  is  the  choice  of  all  AHSs,  and  AHSs  are  now 
referred  to  as  adaptive  Web-based  hypermedia  or 
Web-based  adaptive  hypermedia.  Due  to  its  large 
variety  of  users,  the  Web  is  an  ideal  candidate  for 
adaptivity  and  has  served  as  a  platform  for  AHS 
research  and  development  (Brusilovsky,  2000).  Many 
AHSs  have  been  developed  for  education  and  training 
purposes,  such  as  hypermedia,  e-learning,  virtual 
museums,  and  online  information  systems  (Brusi¬ 
lovsky,  2003;  Cristea,  2005).  AHS  research  and 
development  have  been  extended  to  the  development 
of  AHS  authoring  tools,  such  as  Multibook  (Stein- 
acker  et  al.,  1998),  InterBook  (Brusilovsky  et  al., 
1998),  ACE  (Specht  and  Oppermann,  1998),  KBS 
Hyperbook  (Henze  and  Nejdl,  1999,  2001),  AHA! 
(De  Bra  et  al.,  2003),  ADAPTS  (Brusilovsky  and 
Cooper,  2002),  WHURLE  (Moore  et  al.,  2004),  and 
SmexWeb  (Albrecht  et  al.,  2000).  Also,  the  functions 
of  AHSs  have  been  improved,  from  the  application 
of  simple  one-layered,  knowledge -based  user  models 
to  sophisticated  multiple-layered  user  models  (Brown 
et  al.,  2005). 
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AHS  Taxonomies 

In  1997,  the  Adaptive  Hypertext  and  Hypermedia 
Discussion  forum  (Eklund  and  Sinclair,  2000)  defined 
adaptive  hypermedia  systems  as  “all  hypertext  and 
hypermedia  systems  which  reflect  some  features  of 
the  user  in  the  user  model  and  apply  this  model  to 
adapt  various  visible  and  functional  aspects  of  the 
system  to  the  user.”  More  specifically,  an  adaptive 
hypermedia  system  should:  (1)  be  based  on  hyperme¬ 
dia  or  hypertext,  (2)  have  a  user  model,  (3)  have  a 
domain  model,  and  (4)  be  modifiable  (adaptive)  based 
on  information  contained  in  the  user-model  (Eklund 
and  Sinclair,  2000). 

Brusilovsky  (1996)  developed  the  first  taxonomy 
of  AHSs  which  distinguishes  two  areas  of  adaptation: 
(1)  adaptation  of  the  content  of  the  page,  called  con- 
tent-level  adaptation  or  adaptive  presentation ,  and  (2) 
behavior  of  the  links,  called  link-level  adaptation  or 
adaptive  navigation  support.  The  goal  of  adaptive  pre¬ 
sentation  is  to  adapt  the  content  of  a  hypermedia  page 
to  the  learner’s  goals,  knowledge,  and  other  informa¬ 
tion  stored  in  the  user  model.  The  techniques  of  adap¬ 
tive  presentation  are  to  adapt  the  content  of  a  page 
accessed  by  a  particular  user  to  the  user’s  current 
knowledge,  goals,  and  other  characteristics  or  to  pro¬ 
vide  not  only  text  but  also  a  set  of  various  multimedia 
items.  The  goal  of  adaptive  navigation  support  is  to 
assist  learners  in  finding  their  optimal  paths  in  hyper¬ 
space  by  adapting  link  presentation  and  functionality 
to  the  goals,  knowledge,  and  other  characteristics  of 
individual  learners.  Direct  guidance;  sorting;  annota¬ 
tion;  link  hiding,  disabling,  and  removal;  and  link  gen¬ 
eration  are  ways  to  provide  adaptive  links  to  individual 
learners  (Brusilovsky,  2000,  2003,  2004;  Brusilovsky 
and  Pesin,  1994,  1998;  Brusilovsky  and  Rizzo,  2002; 
Brusilovsky  and  Vassileva,  1996;  De  Bra,  2000; 
Kayama  and  Okamoto,  1998). 

Brusilovky’s  taxonomy  was  refined  by  others. 
Cristea  and  Calvi  (2003)  presented  three  layers  of 
adaptation  from  direct  adaptation  techniques  (low 
level),  adaptation  language  (medium  level),  and  adap¬ 
tation  strategies  (high  level).  The  low  level  is  based 
on  Brusilovky’s  taxonomy,  the  medium  level  refers 
primarily  to  goal-  or  domain-oriented  adaptation 
techniques,  and  the  high  level  refers  to  adaptation 
techniques  for  detecting  learner’s  information-pro¬ 
cessing  strategies  and  cognitive  styles  (Brown  et  al., 
2005;  Calvi  and  Cristea,  2002;  Cristea  and  Calvi, 
2003;  Cristea  and  De  Bra,  2002).  My  Online  Teacher 
(MOT)  is  a  representative  sample  of  AHS  authoring 
tools  that  have  integrated  the  three  layers  of  adapta¬ 
tion  techniques  (Brown  et  al.,  2005;  Cristea  and 
Calvi,  2003). 


Limitations  and  Challenges 

As  reviewed  earlier,  two  features  distinguish  AHSs  from 
micro- adaptive  systems:  (1)  adaptivity /adaptability  and 
(2)  open  corpus  learning  environments.  However,  these 
features  impose  new  challenges.  Williams’  comprehen¬ 
sive  review  (1996)  of  learner-controlled  vs.  program- 
controlled  instruction  revealed  that  learner-controlled 
(adaptable)  systems  and  program-controlled  (adaptive) 
systems  have  different  strengths  and  weaknesses. 
According  to  Williams,  each  type  of  system  is  more 
effective  than  the  other  for  some  people  and  under  cer¬ 
tain  conditions  (for  those  conditions,  see  Williams, 
1996).  This  finding  suggests  that  a  system  with  the 
combined  functions  of  adaptability  and  adaptivity 
would  be  more  effective  than  a  system  with  only  adapt¬ 
able  or  adaptive  functions.  AHSs  were  developed  to 
provide  hybrid  systems;  however,  decisions  about  how 
to  design  adaptive/adaptable  systems  are  still  made  on 
an  ad  hoc  basis  without  an  adequate  theoretical  or 
empirical  rationale  for  the  design  of  these  hybrid  sys¬ 
tems  (Avgeriou  et  al.,  2004;  Cristea  and  De  Bra,  2002). 
Adaptivity  has  risen  to  a  new  level;  the  question  now  is 
how  to  balance  adaptivity  and  adaptability  in  a  system. 

A  conceptual  model  for  adaptable/adaptive  sys¬ 
tems  can  be  found  in  Park’s  (1996)  proposal  of  on- 
task  adaptive  learner-control  (see  Figure  37.1),  in 
which  learners  have  more  freedom  in  choosing  learn¬ 
ing  activities  as  the  instruction  progresses.  In  the 
beginning  stage  of  learning,  the  student’s  familiarity 
with  the  subject  knowledge  and  its  learning  require¬ 
ments  would  be  relatively  low,  and  the  student  would 
not  be  able  to  choose  the  best  strategies  for  learning. 
As  the  instruction  and  learning  continue  ,  however,  the 
student’s  familiarity  with  the  subject  and  ability  to 
learn  it  would  increase,  thus  enabling  the  student  to 
make  better  decisions  in  selecting  strategies  for  learn¬ 
ing  the  subject.  This  argument  is  supported  by  research 
(Carrier,  1984;  Ross  and  Rakow,  1981;  Seidel  et  al., 
1978;  Snow,  1980).  An  on-task  adaptive  learner  control 
system  will  decide  not  only  when  is  the  best  time  for 
giving  the  learner-control  option  but  also  what  kind  of 
control  options  (e.g.,  selection  of  contents,  learning 
activities)  should  be  given  based  on  the  student’s  on- 
task  performance. 

Cristea  and  Garzotto  (2004)  identified  a  set  of 
design  variables  or  problem  classes  to  develop  a  tax¬ 
onomy  of  adaptive/adaptable  educational  hypermedia: 
learner  model,  instructional  strategy,  instructional 
view,  detection  mechanism,  and  adaptation  mecha¬ 
nism.  Also,  they  provided  typical  problems  that  a 
developer  might  face  and  design  guidelines  for  each 
of  the  problems.  Although  the  guidelines  are  not  yet 
complete,  the  taxonomy  can  be  used  as  a  starting  point 
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for  establishing  the  design  standards  of  adaptive/adapt¬ 
able  systems,  including  methods  and  procedures  (Cris- 
tea  and  De  Bra,  2002).  Empirical  study  should  be 
conducted  to  validate  and  refine  these  standards. 

Web-based  AHSs  may  or  may  not  utilize  open 
corpus  resources  in  the  Web;  however,  designing  an 
AHS  that  utilizes  the  open  corpus  hyperspace  is  a 
challenging  task.  Brusilovsky  (2003)  pointed  out  that 
simply  providing  external  links  to  multiple  sources  is 
not  considered  adaptive  support  for  students.  An 
important  question  is  how  the  system  provides  adap¬ 
tive  open  resources  for  students.  Many  systems  and 
techniques  have  been  developed  to  expose  users  to 
information  relevant  only  to  them  (Hanani  et  al.,  2001). 
Several  Web-authoring  tools,  such  as  WHURLE 
(Moore  et  al.,  2004)  and  KBS  Hyperlink  (Henze  and 
Nejdl,  2001),  generate  hyperlinks  not  only  inside  of 
the  lesson  but  also  to  external  Web  resources  based  on 
learners’  previous  activities.  The  adaptive  Web  recom¬ 
mendation  (AWR)  system  is  also  another  method  for 
providing  adaptive  navigation  support  technologies  for 
open  corpus  learning  environments  (Brusilovsky, 
2004).  Whereas  typical  AHSs  attempt  to  adapt  to  var¬ 
ious  aspects  of  the  users  and  provide  a  rich  set  of 
adaptive  navigation  support  techniques  in  closed  learn¬ 
ing  environments,  AWRs  focus  on  one  aspect— user 
interest— and  provide  relevant  external  links.  Web 
Knowledge  Sea  was  developed  utilizing  AWR  tech¬ 
niques  (Brusilovsky,  2004;  Brusilovsky  and  Rizzo, 
2002).  Providing  adaptive  navigational  support  is 
another  important  issue  not  only  for  the  development 
of  adaptive  systems  but  also  for  the  development  of 
information  and  communication  technology.  With  the 
steadily  growing  Semantic  Web,  which  provides  a 
generic  structure  for  producing  machine-processable 
Web  content  (see  http://www.w3.org/2001/sw),  adap¬ 
tive  navigational  support  will  become  more  feasible. 

Reusability,  a  long-standing  problem  in  computer- 
based  adaptive  instructional  systems,  continues  to 
plague  those  working  with  AHSs.  Until  recently,  AHSs 
were  domain- specific  applications  in  particular  systems 
only;  therefore,  it  was  difficult  to  reuse  the  technology 
in  one  system  for  the  development  of  another  system. 
Research  is  currently  in  progress  to  develop  adaptation 
technology  for  establishing  uniform  standards  (Brusi¬ 
lovsky,  2003;  Cristea,  2004;  de  Assis  et  al.,  2004).  The 
Minerva  project  is  an  example.  The  goal  of  the  project 
was  to  establish  a  European  platform  of  standards, 
guidelines,  techniques,  and  tools  for  user- model-based 
adaptability  and  adaptation  (see  http://wwwis.win.tue. 
nl/~acristea/HTML/Minerva/).  This  standardization 
effort  is  expected  to  address  the  reusability  issue.  Meta- 
adaptive  models  and  adaptation  standard  techniques  for 
AHSs  have  been  tested  (de  Assis  et  al.,  2004). 


ADAPTIVE  SYSTEMS  SUPPORTING 
SPECIFIC  PEDAGOGICAL 
APPROACHES 

Andriessen  and  Sandberg  (1999)  noted  that  adaptive 
computer-based  instruction  mainly  focuses  on  the 
acquisition  of  conceptual  and  procedural  knowledge, 
identification  of  common  misconceptions  in  specific 
domains,  and  interactions  between  the  program  (or 
tutor)  and  students.  As  mentioned  earlier,  adaptive 
instructional  systems  were  criticized  for  their  limited 
range  of  teaching  actions  compared  to  human  expert 
teachers.  Researchers  began  to  incorporate  more  com¬ 
plex  pedagogical  approaches,  such  as  constructivist 
learning,  contingent  learning  strategies,  motivational 
competence,  metacognitive  strategies,  and  collaborative 
learning  in  adaptive  instructional  systems.  These  five 
adaptive  approaches  are  not  mutually  exclusive,  but  they 
can  be  distinguished  from  one  another  in  terms  of  their 
primary  thrust.  Although  researchers  are  applying  other 
pedagogical  approaches  to  adaptive  instructional  sys¬ 
tems,  these  five  were  selected  to  provide  an  overview 
of  the  impact  of  pedagogical  approaches  on  the  held. 

Constructivist  Adaptive  Systems 

Constructivist  learning  theories  emphasize  active  roles 
for  learners  to  construct  their  own  knowledge  through 
experiences  in  a  learning  context  in  which  the  target 
domain  is  integrated.  To  implement  constructivist  prin¬ 
ciples  into  computer-based  learning  environments, 
researchers  developed  the  concept  of  intelligent  learn¬ 
ing  environments  (ILEs),  which  emphasize  the  facili- 
tatory  role  of  the  system  as  opposed  to  the  tutoring 
role  (Akhras,  2004;  Brusilovsky,  1994). 

Most  adaptive  instructional  systems  utilized  repre¬ 
sentation  of  knowledge,  inference  of  the  learner’s  state 
of  knowledge,  and  planning  of  instructional  steps 
(Akhras  and  Self,  2000).  Akhras  and  Self  argued  that 
“alternative  views  of  learning,  such  as  constructivism, 
may  similarly  benefit  from  a  system  intelligence  in 
which  the  mechanisms  of  knowledge  representation, 
reasoning,  and  decision  making  originate  from  a  for¬ 
mal  interpretation  of  the  values  of  that  view  of  learn¬ 
ing”  (p.  345).  Constructivist  intelligent  systems  shift 
the  focus  from  a  model  of  what  is  learned  to  a  model 
of  how  knowledge  is  learned  (Akhras  and  Self,  2000). 

To  model  this  facilitatory  role,  researchers 
(Akhras,  2004;  Akhras  and  Self,  2000, 2002)  proposed 
four  important  considerations  when  designing  these 
systems:  situation-based  contexts,  learning  interac¬ 
tions  in  situation,  time-extended  processes  of  interac¬ 
tion,  and  affordances  of  learning  situations.  Their 
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approach  was  implemented  in  INCENSE  (INtelligent 
Constructivist  ENvironment  for  Software  Engineering 
learning).  KBS  Hyperbook  is  another  example  of  con¬ 
structivist  AHS  (Henze  and  Nejdl,  1999). 

Contingent  Teaching  Systems 

According  to  Vygotsky’s  theory  (1978),  providing 
immediate  and  appropriately  challenging  activities  and 
contingent  teaching  based  on  a  learner’s  behavior  is 
important  for  that  learner  to  progress  to  the  next  level 
of  learning.  He  believed  minimal  levels  of  guidance 
are  the  best  for  learners .  Recently,  this  theory  has  been 
applied  in  several  different  ways  to  computer-based 
instruction.  As  compared  to  traditional  adaptive 
instruction,  a  contingent  teaching  system  has  no  global 
model  of  the  learner.  A  learner’s  performance  is  local, 
and  the  situation  is  constrained  by  contingencies  in  the 
learner’s  current  activity.  Because  the  tutor’s  actions 
and  reactions  should  be  occurring  in  response  to  the 
learner’s  inputs,  the  theory  promotes  an  active  view  of 
the  learner  and  an  account  of  learning  as  a  collabora¬ 
tive  and  constructive  process  (Wood  and  Wood,  1996). 
Contingent  tutoring  systems  generally  provide  two 
assessment  methods:  model  tracing  and  knowledge 
tracing  (du  Boulay  and  Luckin,  2001).  The  purpose  of 
model  tracing  is  to  keep  track  of  all  of  the  student’s 
actions  as  the  problem  is  solved  and  to  flag  errors  as 
they  occur.  It  also  adapts  the  help  feedback  based  on 
specific  problem-solving  contexts.  The  purpose  of 
knowledge  tracing  is  to  choose  the  next  problem  that 
is  appropriately  challenging  so  the  students  can  move 
in  a  timely  but  effective  manner  through  the  curricu¬ 
lum.  Examples  of  contingent  teaching  systems  include 
SHERLOCK  (Lesgold,  2001;  Lesgold  et  al„  1992), 
QUADRATIC  tutor  (Wood  and  Wood,  1999),  DATA 
(Wood  et  al„  1998),  Ecolab  (Luckin  and  du  Boulay, 
1999),  and  M-Ecolab  (Rebolledo  Mendez  et  al.,  2006). 

Motivation-Based  Adaptive  Systems 

Some  new  adaptive  instructional  systems  take  into 
account  student  motivation.  Proponents  suggest  that  a 
comprehensive  adaptive  instructional  plan  should  con¬ 
sist  of  a  traditional  instructional  plan  combined  with 
a  motivational  plan  (del  Soldato  and  du  Boulay,  1995; 
du  Boulay  and  Luckin,  2001;  Wasson,  1990).  De  Vice¬ 
nte  and  Pain  (2002)  have  developed  a  model  called  the 
motivation  model ,  which  diagnoses  a  student’s  moti¬ 
vational  state  based  on  several  variables  (i.e.,  control, 
challenge,  independence,  fantasy,  confidence,  sensory 
interest,  cognitive  interest,  effort,  satisfaction). 
COSMO  (Lester  et  al.,  1999)  supports  a  pedagogical 
agent  that  can  incorporate  nonverbal  feedback  and  con¬ 


versational  signals  such  as  its  facial  expression,  its  tone 
of  voice,  its  gestures,  and  the  structure  of  its  utterances 
to  indicate  its  own  affective  state  and  to  increase  stu¬ 
dents’  motivation  during  its  interactions  with  learners. 

Metacognition-Based  Adaptive  Systems 

Metacognitive  skills  enable  students  to  assess  their 
own  learning  processes.  As  ICT-based  individualized 
instruction,  including  online  learning  environments, 
becomes  increasingly  prevalent,  metacognitive  and 
self-regulatory  processes  are  becoming  more  impor¬ 
tant  in  the  design  of  systems  (Azevedo,  2005a,b;  Quin¬ 
tana  et  al.,  2005;  Zimmerman  and  Tsikalas,  2005).  In 
contrast,  most  early  tutoring  systems  did  not  promote 
students’  metacognitive  thinking  (Carroll  and  McKen- 
dree,  1987). 

White  et  al.  (1999)  argued  that  metacognitive  pro¬ 
cesses  can  be  easily  understood  and  observed  in  a  multi¬ 
agent  social  system  that  integrates  cognitive  and  social 
aspects  of  cognition  within  a  social  framework.  Based 
on  this  conceptual  framework,  they  developed  the  SCI- 
WISE  program,  which  houses  a  community  of  software 
agents,  such  as  an  Inventor,  an  Analyzer,  and  a  Collab¬ 
orator.  The  agents  provide  strategic  advice  and  guidance 
to  learners  as  they  undertake  research  projects  and  as 
they  reflect  on  and  revise  their  inquiry;  therefore,  stu¬ 
dents  express  their  metacognitive  ideas  as  they  under¬ 
take  complex  sociocognitive  practices.  Through  this 
exercise,  students  develop  explicit  theories  about  the 
social  and  cognitive  processes  required  for  collaborative 
inquiry  and  reflective  learning  (White  et  al.,  1999). 

Another  example  focusing  on  improving  metacog¬ 
nitive  skills  is  the  Geometry  Explanation  Tutor  (GET) 
developed  by  Aleven  et  al.  (2001).  They  argued  that 
self-explanation  is  an  effective  metacognitive  strategy. 
Having  students  explain  examples  or  problem-solving 
steps  helps  them  learn  with  greater  understanding 
(Aleven  et  al.,  2003a,b;  Chi  et  al.,  1989).  Aleven  and 
his  associates  conducted  experimental  studies  to  test 
the  effects  of  GET.  The  results  provided  positive 
effects  on  learning  outcomes. 

The  ability  to  pursue  help  effectively  and  effi¬ 
ciently  when  needed  is  an  important  metacognitive 
skill.  Aleven  and  his  associates  (Aleven  et  al.,  2003c, 
2006)  also  developed  Help-Seeking  Tutor  Agent, 
which  helps  students  become  better  help  seekers.  They 
embedded  it  within  their  adaptive  instructional  system. 
Geometry  Cognitive  Tutor,  and  tested  whether  Help- 
Seeking  Tutor  Agent  helped  students  to  be  better  help 
seekers  and  to  learn  better  as  a  result  (Aleven  et  al., 
2004).  The  results  showed  that  the  more  help-seeking 
errors  students  made  with  the  system,  the  less  they 
learned.  Other  research  (Roll  et  al.,  2006)  revealed 
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that,  although  Help-Seeking  Tutor  Agent  achieved  pos¬ 
itive  effects  because  students  followed  its  advice,  stu¬ 
dents  did  not  internalize  the  help-seeking  principles. 

Collaborative  Learning  Systems 

One  of  new  pedagogical  approaches  incorporated  in 
adaptive  instructional  systems  is  collaborative  learning 
(Soller  et  al.,  2005).  Effective  collaboration  with  peers 
is  a  powerful  learning  experience;  however,  simply 
placing  students  in  a  group  and  assigning  a  task  to  the 
group  does  not  produce  a  valuable  learning  experience 
for  them  (Soller,  2001).  Teachers  (tutors  in  computer- 
based  systems)  should  provide  strategies  for  students 
to  experience  collaborative  learning  effectively  and 
interestingly.  Through  the  use  of  an  intelligent  collab¬ 
orative  system,  Soller  (2001)  identified  five  character¬ 
istics  of  effective  collaborative  learning  behaviors:  (1) 
participation,  (2)  social  grounding,  (3)  performance 
analysis  and  group  processing,  (4)  application  of  active 
learning  conversation  skills,  and  (5)  promotive  inter¬ 
actions.  Soller  and  her  colleagues  (2005)  developed  a 
conceptual  framework,  the  Collaboration  Management 
Cycle,  represented  by  a  feedback  loop  for  the  five- 
phased  behaviors. 

Although  collaborative  learning  systems  are  still 
in  development  (Jermann  et  al.,  2004;  Or-Bach  and 
van  Joolingen,  2004;  Soller  et  al.,  2005),  their  contri¬ 
butions  to  adaptive  instructional  systems  will  be  sig¬ 
nificant;  they  not  only  facilitate  student  group  activities 
but  also  help  educators  and  researchers  better  under¬ 
stand  group  interactions  and  determine  how  to  support 
collaborative  learning  better.  Appropriate  combina¬ 
tions  of  sound  pedagogical  approaches  such  as  the 
ones  described  above  with  the  structural  features  and 
functions  of  adaptive/adaptable  systems  will  further 
improve  the  technical  capacity  and  instructional  effec¬ 
tiveness  of  the  systems. 

CONCLUSION 

Adaptive  instruction  has  a  long  history.  Systematic 
efforts  to  develop  adaptive  instructional  systems  began 
in  the  early  1900s.  Adaptive  instructional  systems  have 
embodied  different  approaches:  macro- adaptive,  ATI- 
based,  micro-adaptive  (including  ITS),  and  adap¬ 
tive/adaptable  hybrid  systems,  as  well  as  other 
approaches  that  incorporate  different  pedagogical  per¬ 
spectives.  Using  interactive  computer  technology,  a 
number  of  different  micro-adaptive  instructional  sys¬ 
tems  have  been  developed;  however,  early  applications 
were  mostly  in  laboratory  environments  because  of  the 
limitations  of  their  functional  capabilities  to  handle 
complex  learning  and  instructional  processes. 


Since  the  1990s,  with  the  advent  of  the  Web  and 
AHSs,  adaptive  instructional  applications  have  moved 
out  of  the  labs  and  into  classrooms  and  workplaces. 
During  the  last  decade,  numerous  AHS  systems  have 
been  developed;  however,  the  dearth  of  empirical  evi¬ 
dence  and  weak  theoretical  foundations  supporting 
their  effectiveness  (Shapiro  and  Niederhauser,  2004) 
remain  as  a  hindrance  for  the  wider  use  of  the  systems 
in  school  education  and  industry  training.  It  is  a  diffi¬ 
cult  and  challenging  task  to  develop  technically  robust, 
theoretically  sound,  and  empirically  valid  systems. 
Research  and  development  on  new  pedagogical 
approaches  in  adaptive  systems  may  provide  stronger 
theoretical  foundations  for  the  development  of  future 
systems.  AHS  technology  in  collaboration  with  the 
Semantic  Web  may  lead  to  standardized  open  adapta¬ 
tion  technology  for  the  development  of  future  adaptive 
systems.  If  more  theoretically  sound  and  technically 
robust  systems  are  developed,  it  becomes  more  likely 
that  the  effects  of  the  systems  will  be  supported  in 
empirical  evaluations. 
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ABSTRACT 

Problem-based  learning  (PBL)  is  perhaps  the  most 
innovative  instructional  method  conceived  in  the  his¬ 
tory  of  education.  PBL  was  originally  designed  to 
respond  to  the  criticism  that  traditional  teaching  and 
learning  methods  fail  to  prepare  medical  students  for 
solving  problems  in  clinical  settings.  Instead  of  requir¬ 
ing  that  students  study  content  knowledge  and  then 
practice  context-free  problems,  PBL  embeds  students’ 
learning  processes  in  real-life  problems.  After  its  suc¬ 
cessful  implementation  in  various  fields  of  medical 
education,  PBL  is  now  being  implemented  throughout 
higher  education  as  well  as  in  K-12  education.  The 
purpose  of  this  chapter  is  to  inform  researchers  and 
practitioners  about  research  findings  and  issues  in  PBL 
that  may  be  used  to  inform  future  studies.  In  this  chap¬ 
ter,  we  review  PBL  research  from  the  past  30  years. 
We  first  describe  the  history  of  development  and  imple¬ 
mentation  of  PBL  in  various  educational  settings  and 
define  the  major  characteristics  of  PBL .  We  then  review 
the  research  on  PBL.  First,  we  examine  the  effective¬ 
ness  of  PBL  in  terms  of  student  learning  outcomes, 
including  basic  domain  knowledge  acquisition  and 
applications,  retention  of  content  and  problem-solving 
skills,  higher  order  thinking,  self-directed  leaming/life- 
long  learning,  and  self-perception.  Second,  we  look  at 
implementation  issues,  such  as  tutoring  issues,  curric¬ 
ulum  design  issues,  and  use  of  technology.  Finally,  we 
provide  recommendations  for  future  research. 


KEYWORDS 

Curriculum  design:  A  process  of  conceiving  a  plan  to 
define  a  set  of  courses  constituting  an  area  of  spe¬ 
cialization  that  supports  the  specified  learning  goal. 

Problem-based  learning:  An  instructional  method  that 
initiates  students’  learning  by  creating  a  need  to 
solve  an  authentic  problem.  During  the  problem¬ 
solving  process,  students  construct  content  knowl¬ 
edge  and  develop  problem-solving  skills  as  well  as 
self-directed  learning  skills  while  working  toward 
a  solution  to  the  problem. 

Problem  solving:  A  process  of  understanding  the  dis¬ 
crepancy  between  current  and  goal  states  of  a  prob¬ 
lem,  generating  and  testing  hypotheses  for  the 


causes  of  the  problem,  devising  solutions  to  the 
problem,  and  executing  the  solution  to  satisfy  the 
goal  state  of  the  problem. 


INTRODUCTION  AND  HISTORY 

Problem-based  learning  (PBL)  is  perhaps  the  most 
innovative  pedagogical  method  ever  implemented  in 
education.  Its  effectiveness  in  facilitating  student  prob¬ 
lem-solving  and  self-directed  learning  skills  has  been 
widely  reported  in  medical  education  (Barrows  and 
Tamblyn,  1980;  Schmidt,  1983).  PBL  has  also  become 
increasingly  popular  across  disciplines  in  higher  edu¬ 
cation  and  K-12  education  settings  (Barrows,  2000; 
Dochy  et  al.,  2003;  Gallagher  et  al.,  1992;  Hmelo- 
Silver,  2004;  Hmelo  et  al.,  2000;  Torp  and  Sage,  2002; 
Williams  and  Hmelo,  1998).  So,  what  is  PBL?  What 
are  the  theoretical  bases  for  this  instructional  method? 
Why  does  it  receive  such  attention  from  researchers 
and  educators  across  disciplines  and  age  levels?  How 
does  it  work?  And  does  it  really  work?  We  begin  this 
chapter  by  introducing  the  origins  of  PBL  and  provid¬ 
ing  a  brief  history  of  PBL  as  background  information, 
followed  by  a  discussion  of  its  conceptual  assump¬ 
tions.  We  then  review  research  on  the  effectiveness  of 
PBL  and  the  various  implementation  issues  emerging 
from  PBL  research  over  the  past  30  years.  Finally,  we 
conclude  the  chapter  with  a  series  of  proposed  research 
issues  in  light  of  previous  experience  and  empirical 
evidence  from  PBL  research  and  implementation,  as 
well  as  potential  research  topics  for  future  studies. 

Brief  History  of  PBL 

Problem-Based  Learning  in  Medical  Education 

Problem-based  learning  was  first  developed  in  medical 
education  in  the  1950s.  The  development  of  PBL  is 
generally  credited  to  the  work  of  medical  educators  at 
McMasters  University  in  Canada  in  the  1970s.  Around 
the  same  time,  other  medical  schools  in  various  coun¬ 
tries,  such  as  Michigan  State  University  in  the  United 
States,  Maastricht  University  in  the  Netherlands,  and 
Newcastle  University  in  Australia  were  also  develop¬ 
ing  problem-based  learning  curricula  (Barrows,  1996). 
PBL  was  conceived  and  implemented  in  response  to 
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students’  unsatisfactory  clinical  performance  (Bar- 
rows,  1996;  Barrows  and  Tamblyn,  1980)  that  resulted 
from  an  emphasis  on  memorization  of  fragmented  bio¬ 
medical  knowledge  in  the  traditional  health  science 
education.  This  emphasis  was  blamed  for  failing  to 
equip  students  with  clinical  problem-solving  and  life¬ 
long  learning  skills  (Albanese  and  Mitchell,  1993;  Bar- 
rows,  1996). 

In  the  1980s,  the  wider  spread  of  PBL  in  the  United 
States  was  accelerated  by  the  GPEP  report  (Report  of 
the  Panel  on  the  General  Professional  Education  of  the 
Physician  and  College  Preparation  for  Medicine)  spon¬ 
sored  by  the  Association  of  American  Medical  Col¬ 
leges  (Muller,  1984).  This  report  made  recommenda¬ 
tions  for  changes  in  medical  education,  such  as 
promoting  independent  learning  and  problem  solving, 
reducing  lecture  hours,  reducing  scheduled  time,  and 
evaluating  the  ability  to  leam  independently  (Barrows, 
1996).  These  recommendations  strongly  supported  the 
implementation  of  PBL  in  medical  education.  During 
this  period  of  time,  some  medical  schools  also  began 
to  develop  alternative,  parallel  problem-based  curric¬ 
ula  (e.g.,  the  Primary  Care  Curriculum  at  the  Univer¬ 
sity  of  New  Mexico,  the  New  Pathways  Program  in 
Medical  School  of  Harvard  University)  for  a  subset  of 
their  students  (Aspy  et  al.,  1993;  Barrows,  1996). 
Later,  a  number  of  medical  schools,  such  as  the  Uni¬ 
versity  of  Hawaii,  Harvard  University,  and  the  Univer¬ 
sity  of  Sherbrooke  in  Canada,  assumed  the  more  ardu¬ 
ous  tasks  of  converting  their  entire  curriculum  to  PBL. 
In  the  1990s,  many  more  medical  schools,  such  as 
Southern  Illinois  University,  Rush,  Bowman  Gray,  and 
Tufts,  adopted  PBL  as  their  primary  instructional 
method  (Aspy  et  al.,  1993;  Barrows,  1994).  Since  its 
first  implementation  several  decades  ago,  PBL  has 
become  a  prominent  pedagogical  method  in  medical 
schools  and  health-science-related  programs  through¬ 
out  the  world,  including  North  America,  the  Nether¬ 
lands,  England,  Germany,  Australia,  New  Zealand,  and 
India. 

Problem-Based  Learning  Outside  the  Medical  Field 

Higher  Education 

The  adoption  of  PBL  in  higher  education  outside  of 
the  medical  field  as  well  as  K-12  settings  gradually 
occurred  throughout  the  1990s.  PBL  has  been  applied 
globally  in  a  variety  of  professional  schools  (Boud  and 
Feletti,  1991;  Gijselaers  et  al.,  1995;  Wilkerson  and 
Gijselaers,  1996),  such  as  architecture  (Donaldson, 
1989;  Maitland,  1998),  business  administration  (Mer- 
chand,  1995),  chemical  engineering  (Woods,  1996), 
engineering  studies  (Cawley,  1989), law  schools  (Boud 


and  Feletti,  1991;  Kurtz  et  al.,  1990;  Pletinckx  and 
Segers,  2001),  leadership  education  (Bridges  and  Hal- 
linger,  1992,  1995,  1996;  Cunningham  and  Cordeiro, 
2003),  nursing  (Barnard  et  al.,  2005;  Higgins,  1994), 
social  work  (Bolzan  and  Heycox,  1998),  and  teacher 
education  (Oberlander  and  Talbert-Johnson,  2004). 
Moreover,  Moust  et  al.  (2005)  reported  that  PBL  is 
also  frequently  integrated  into  a  wider  range  of  disci¬ 
plines,  such  as  biology  (Szeberenyi,  2005),  biochem¬ 
istry  (Osgood  et  al.,  2005),  calculus  (Seltzer  et  al., 
1996),  chemistry  (Barak  and  Dori,  2005),  economics 
(Garland,  1995),  geology  (Smith  and  Hoersch,  1995), 
psychology  (Reynolds,  1997),  science  courses  (Allen 
et  al.,  1996),  physics,  art  history,  educational  psychol¬ 
ogy,  leadership  education,  criminal  justice,  nutrition 
and  dietetics,  and  other  domains  of  post-secondary 
education  (Edens,  2000;  Savin-Baden,  2000;  Savin- 
Baden  and  Wilkie,  2004). 

K-12  Education 

In  introducing  PBL  into  K-12  education,  Barrows  and 
Kelson  (1993)  systematically  developed  PBL  curricula 
and  teacher-training  programs  for  all  high-school  core 
subjects  (see  Illinois  Math  and  Science  Academy, 
http://www.imsa.edu).  Since  then,  PBL  has  been  pro¬ 
moted  by  a  number  of  scholars  and  practitioners  for 
use  in  basic  education  (Arends,  1997;  Glasgow,  1997; 
Jones  et  al.,  1997;  Kain,  2003;  Krynock  and  Robb, 
1999;  Savoie  and  Hughes,  1994;  Stepien  et  al.,  2000; 
Torp  and  Sage,  2002;  Wiggins  and  McTighe,  1998). 
Various  results  of  implementations  of  PBL  in  K-12 
settings  have  been  widely  reported.  First,  PBL  has 
been  shown  to  be  effective  in  conveying  a  variety  of 
content  areas— for  example,  mathematics  (Cognition 
and  Technology  Group  at  Vanderbilt,  1993),  science 
(Kolodner  et  al.,  2003;  Linn  et  al.,  1999),  literature 
(Jacobsen  and  Spiro,  1994),  history  (Wieseman  and 
Cadwell,  2005),  and  microeconomics  (Maxwell  et  al., 
2005).  Second,  PBL  has  been  implemented  effectively 
in  schools  in  urban,  suburban,  and  rural  communities 
(Delisle,  1997;  Fogarty,  1997).  Third,  PBL  can  be  used 
effectively  in  a  wide  variety  of  student  popula¬ 
tions— for  example,  gifted  elementary-,  middle-,  and 
high-school  students  (Dods,  1997;  Gallagher,  1997; 
Gallagher  et  al.,  1995;  Stepien  and  Gallagher,  1993; 
Stepien  et  al.,  1993),  as  well  as  low-income  students 
(Stepien  and  Gallagher,  1993). 

Interest  in  PBL  is  increasing  in  higher  education 
and  K-12  education  as  evidenced  by  the  widespread 
publication  of  books  about  PBL  (such  as  Barrows, 
2000;  Duch  et  al.,  2001;  Evenson  and  Hmelo,  2000; 
Kain,  2003;  Torp  and  Sage,  2002).  As  Internet  servers 
concerned  with  PBL  (see  http://interact.bton.ac. 
uk/pbl/)  reveal,  many  teachers  around  the  world  are 
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using  PBL,  and  the  numbers  are  expected  to  grow.  An 
increasing  number  of  PBL  literature  reviews  (Albanese 
and  Mitchell,  1993;  Dochy  et  al.,  2003;  Gijbels  et  al., 
2005;  Hmelo-Silver,  2004;  Newman,  2003;  Smits  et 
al.,  2002;  Van  den  Bossche  et  al.,  2000;  Vernon  and 
Blake,  1993)  and  PBL  conferences  (e.g.,  PUCP,  2006) 
also  reflect  the  popularity  of  PBL. 


ASSUMPTIONS  AND 
CHARACTERISTICS 

Assumptions 

A  primary  assumption  of  PBL  is  that  when  we  “solve 
the  many  problems  we  face  everyday,  learning  occurs” 
(Barrows  and  Tamblyn,  1980,  p.  1).  Although  such  a 
statement  may  appear  self-evident,  this  assumption  is 
countered  by  the  public  assumption  that  learning 
occurs  only  in  formal  education  settings,  so  once  we 
leave  school  we  cease  to  learn.  Proponents  of  PBL 
believe,  as  did  Karl  Popper  (1994),  that  “Alles  leben 
ist  Problemlosen  [all  life  is  problem  solving].”  If  all 
life  is  problem  solving,  then  all  life  is  replete  with 
learning  opportunities.  As  we  shall  explain  later,  the 
most  consistent  finding  from  PBL  research  is  the  supe¬ 
riority  of  PBL-trained  learners  in  life-long  learning. 

In  addition  to  the  importance  of  life-long  learning, 
PBL  proponents  assume  the  primacy  of  problems  in 
learning;  that  is,  learning  is  initiated  by  an  authentic, 
ill-structured  problem.  In  PBL  classes,  students 
encounter  the  problem  before  learning,  which  is  coun¬ 
tered  by  centuries  of  formal  education  practice,  where 
students  are  expected  to  master  content  before  they 
ever  encounter  a  problem  and  attempt  to  apply  the 
content.  Learning  in  PBL  is  bounded  by  problems. 

Problem-based  learning  is  based  on  constructivist 
assumptions  about  learning,  such  as: 

•  Knowledge  is  individually  constructed  and 
socially  co-constructed  from  interactions 
with  the  environment;  knowledge  cannot  be 
transmitted. 

•  There  are  necessarily  multiple  perspectives 
related  to  every  phenomenon. 

•  Meaning  and  thinking  are  distributed  among 
the  culture  and  community  in  which  we  exist 
and  the  tools  that  we  use. 

•  Knowledge  is  anchored  in  and  indexed  by 
relevant  contexts. 

Concomitantly,  PBL  is  underpinned  by  theories  of 
situated  learning,  which  assume  that  learning  is  most 
effective  when  it  is  embedded  in  authentic  tasks  that 


are  anchored  in  everyday  contexts.  In  everyday  and 
professional  lives,  people  continuously  solve  ill-struc¬ 
tured  problems,  those  that  have  multiple  or  unknown 
goals,  solution  methods,  and  criteria  for  solving  the 
problems.  Because  meaning  is  derived  by  learners 
from  interactions  with  the  contexts  in  which  they  are 
working  or  learning  (ideas  abstracted  from  contexts 
and  presented  as  theories  have  little,  if  any,  meaning 
to  learners),  knowledge  that  is  anchored  in  specific 
contexts  is  more  meaningful,  more  integrated,  better 
retained,  and  more  transferable.  One  reason  for  this 
phenomenon  is  the  ontology  that  students  use  to  rep¬ 
resent  their  understanding  (Jonassen,  2006).  Knowl¬ 
edge  constructed  for  solving  problems  results  in  epis¬ 
temological  (task- related  procedural  knowledge)  and 
phenomenological  (the  world  as  we  consciously  expe¬ 
rience  it)  knowledge  types.  These  are  richer,  more 
meaningful  and  memorable  representations. 

In  addition  to  supporting  more  meaning  by  anchor¬ 
ing  learning  in  authentic  problems,  problems  provide 
a  purpose  for  learning.  Without  an  intention  to  learn, 
which  is  provided  by  problems,  meaningful  learning 
seldom  occurs.  When  studying  course  content,  stu¬ 
dents  who  are  unable  to  articulate  a  clear  purpose  or 
intention  for  learning  seldom  learn  meaningfully. 
When  knowledge  is  evaluated  based  on  its  similarity 
to  an  authority,  students’  epistemological  development 
is  retarded.  They  fail  to  understand  or  accommodate 
multiple  perspectives  and  make  no  effort  to  construct 
their  own  culturally  relevant  understanding. 

Characteristics  of  PBL 

Problem-based  learning  is  an  instructional  methodol¬ 
ogy;  that  is,  it  is  an  instructional  solution  to  learning 
problems.  The  primary  goal  of  PBL  is  to  enhance 
learning  by  requiring  learners  to  solve  problems.  It  is 
a  methodology  with  the  following  characteristics: 

•  It  is  problem  focused,  such  that  learners 
begin  learning  by  addressing  simulations  of 
an  authentic,  ill-structured  problem.  The 
content  and  skills  to  be  learned  are  organized 
around  problems,  rather  than  as  a  hierarchi¬ 
cal  list  of  topics,  so  a  reciprocal  relationship 
exists  between  knowledge  and  the  problem. 
Knowledge  building  is  stimulated  by  the 
problem  and  applied  back  to  the  problem. 

•  It  is  student  centered,  because  faculty  cannot 
dictate  learning. 

•  It  is  self-directed,  such  that  students  individ¬ 
ually  and  collaboratively  assume  responsi¬ 
bility  for  generating  learning  issues  and 
processes  through  self-assessment  and  peer 
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assessment  and  access  their  own  learning 
materials.  Required  assignments  are  rarely 
made. 

•  It  is  self-reflective,  such  that  learners  moni¬ 
tor  their  understanding  and  learn  to  adjust 
strategies  for  learning. 

•  Tutors  are  facilitators  (not  knowledge  dis¬ 
seminators)  who  support  and  model  reason¬ 
ing  processes,  facilitate  group  processes  and 
interpersonal  dynamics,  probe  students’ 
knowledge  deeply,  and  never  interject  con¬ 
tent  or  provide  direct  answers  to  questions. 

The  PBL  learning  process  normally  involves  the  fol¬ 
lowing  steps: 

•  Students  in  groups  of  five  to  eight  encounter 
and  reason  through  the  problem.  They 
attempt  to  define  and  bound  the  problem  and 
set  learning  goals  by  identifying  what  they 
know  already,  what  hypotheses  or  conjec¬ 
tures  they  can  think  of,  what  they  need  to 
leam  to  better  understand  the  dimensions  of 
the  problem,  and  what  learning  activities  are 
required  and  who  will  perform  them. 

•  During  self-directed  study,  individual  stu¬ 
dents  complete  their  learning  assignments. 
They  collect  and  study  resources  and  prepare 
reports  to  the  group. 

•  Students  share  their  learning  with  the  group 
and  revisit  the  problem,  generating  addi¬ 
tional  hypotheses  and  rejecting  others  based 
on  their  learning. 

•  At  the  end  of  the  leaning  period  (usually  one 
week),  students  summarize  and  integrate 
their  learning. 

In  the  following  sections,  we  discuss  PBL  effectiveness 
and  implementation  issues  from  PBL  research  findings. 


RESEARCH  RESULTS 

Throughout  the  past  several  decades,  a  vast  body  of 
research  on  various  aspects  of  PBL  has  contributed  to 
our  knowledge  of  PBL.  Although  PBL  has  gained  pop¬ 
ularity  in  K-12  and  higher  education,  the  majority  of 
PBL  research  continues  to  be  conducted  in  the  medical 
education  field.  Within  that  body  of  research,  some 
issues,  such  as  the  effects  of  PBL  on  student  perfor¬ 
mance,  have  received  more  attention  than  others.  In 
the  following  sections,  we  will  review  PBL  studies  in 
two  major  research  areas:  student  learning  outcomes 
and  implementation  issues. 


Learning  Outcomes 

Basic  Domain  Knowledge 
Acquisition  and  Applications 

Problem-based  learning  is  often  criticized  for  its 
emphasis  on  facilitating  higher  order  thinking  and 
problem-solving  skills  at  the  expense  of  lower  level 
knowledge  acquisition.  This  concern  has  been 
expressed  not  only  by  teachers  (Angeli,  2002)  but  also 
by  students  (Dods,  1997;  Lieux,  2001;  Schultz-Ross 
and  Kline,  1999).  In  some  cases,  the  students  believed 
that  content  was  inadequately  covered,  even  though 
they  understood  the  content  more  thoroughly  (Dods, 
1997)  and  performed  comparably  to  traditional  stu¬ 
dents  on  assessments  (Lieux,  2001). 

Higher  Education  and  K-12  Education 

Compared  to  PBL  research  conducted  within  the 
medical  field,  empirical  studies  conducted  in  non¬ 
medical  disciplines  and  K-12  settings  are  relatively 
scarce.  Polanco  et  al.  (2004)  investigated  the  effect 
of  PBL  on  engineering  students’  academic  achieve¬ 
ment.  They  found  that,  when  compared  to  their  coun¬ 
terparts,  PBL  curriculum  significantly  enhanced 
engineering  students’  performance  on  the  Mechanics 
Baseline  Test,  in  which  the  focus  of  the  test  was  on 
understanding  and  application  of  the  concepts  rather 
than  recall  of  factual  knowledge.  Also,  to  evaluate 
the  validity  of  the  criticism  that  PBL  students  tend 
to  underperform  on  knowledge  acquisition  when 
being  measured  with  standardized  tests,  Gallagher 
and  Stepien  (1996)  embarked  upon  an  investigation 
in  which  they  devised  a  65-item  multiple-choice  test 
intentionally  imitating  typical  final  exams  on  the 
topic  of  American  studies.  The  results  showed  that 
no  significant  difference  existed  in  the  content  acqui¬ 
sition  between  students  who  were  in  the  PBL  course 
and  students  who  were  in  the  non-PBL  course;  in 
fact,  the  PBL  students’  average  gain  was  higher  than 
the  other  three  traditional  classes. 

Zumbach  et  al.  (2004)  also  studied  PBL  effects 
on  fourth  graders  in  a  German  elementary  school. 
They  found  no  significant  difference  on  domain 
knowledge  acquisition  between  students  who  studied 
using  PBL  and  traditional  formats.  Similar  results 
were  also  found  in  student  learning  in  a  Quantity 
Food  Production  and  Service  course  (Lieux,  2001) 
and  diabetes-related  learning  among  adolescents  with 
diabetes  (Schlundt  et  al.,  1999).  Yet,  a  significantly 
lower  gain  score  in  economic  knowledge  was  found 
in  PBL  classes  than  in  lecture-  and  discussion-based 
classes  in  high-school  economics  classes  (Mergen- 
doller  et  al.,  2000). 
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Medical  Fields 

Research  from  medical  education,  on  the  other  hand, 
provides  a  rich  body  of  empirical  evidence  for  evalu¬ 
ating  the  effectiveness  of  PBL.  Blake  et  al.  (2000) 
reported  a  very  successful  implementation  of  PBL  cur¬ 
riculum  at  the  University  of  Missouri-Columbia.  They 
compared  the  performance  of  six  classes  of  medical 
students  from  1995  to  2000  on  the  U.S.  Medical 
Licensing  Examination  (USMLE,  formerly  NBME). 
They  found  that  the  PBL  classes  performed  substan¬ 
tially  better  on  both  basic  science  and  clinical  science 
than  did  the  classes  under  a  traditional  curriculum. 
More  encouragingly,  the  mean  scores  of  the  PBL 
classes  (1998  and  1999)  were  significantly  higher  than 
their  respective  national  mean  scores,  and  the  mean 
scores  of  the  traditional  classes  were  lower  than 
national  mean  scores.  Especially,  the  1996  class  (tradi¬ 
tional  curriculum)  scored  significantly  lower  than  the 
national  mean  score.  Also,  as  measured  by  key  feature 
problems  (KFPs),  Doucet  et  al.  (1998)  found  PBL  stu¬ 
dents  performed  significantly  better  on  applying  knowl¬ 
edge  in  clinical  reasoning  than  did  the  traditional  stu¬ 
dents  in  a  headache  diagnosis  and  management  course. 
Similarly,  PBL  students  performed  significantly  better 
than  their  counterparts  in  their  clerkships  (Distlehorst 
et  al.,  2005)  and  in  podiatric  medicine  (Finch,  1999). 
Schwartz  et  al.  (1997)  compared  PBL  and  traditional 
medical  students  at  the  University  of  Kentucky  and 
found  that  PBL  students  performed  equally  well  or 
better  on  factual  knowledge  tests  and  significantly  bet¬ 
ter  on  the  application  of  the  knowledge  in  an  essay 
exam  and  a  standardized  patient  exam  than  did  lecture- 
based  students.  Also,  Shelton  and  Smith  (1998) 
reported  a  better  pass  rate  for  the  PBL  biomedical  stu¬ 
dents  than  their  counterparts  in  both  year  1  and  year  2 
in  an  undergraduate  analytic  science  theory  class. 

To  summarize  existing  empirical  studies  being 
conducted  on  PBL,  a  number  of  meta-analyses  have 
been  conducted.  Albanese  and  Mitchell  (1993)  exam¬ 
ined  research  from  1972  to  1992,  and  Vernon  and 
Blake  (1993)  examined  research  from  1970  to  1992. 
Both  meta-analyses  concluded  that,  in  general,  the 
PBL  research  findings  were  mixed.  The  two  meta¬ 
analyses  agreed  that  traditional  curriculum  students 
perform  better  on  basic  science  knowledge  acquisition, 
but  PBL  students  perform  better  on  clinical  knowledge 
acquisition  and  reasoning.  Moreover,  their  finding  that 
PBL  students’  knowledge  acquisition  was  not  robust 
was  confirmed  by  another  meta-analysis  of  43  PBL 
studies  conducted  10  years  later  by  Dochy  et  al. 
(2003);  however,  when  comparing  students’  perfor¬ 
mance  on  progress  tests  under  PBL  and  traditional 
curriculum,  Verhoeven  et  al.’s  (1998)  findings  only 


partially  agreed  with  the  findings  of  Albanese  and 
Mitchell  and  Vernon  and  Blake.  They  found  that  the 
traditional  students  obtained  better  scores  on  basic  sci¬ 
ence,  while  PBL  students  performed  better  on  social 
science;  yet,  to  their  surprise,  the  PBL  students  did  not 
outperform  traditional  students  on  clinical  science. 
Two  other  PBL  literature  reviews  conducted  by  Berk- 
son  (1993)  and  Colliver  (2000)  did  not  agree  with  the 
two  seminal  meta-analyses  and  found  no  convincing 
evidence  to  support  the  superiority  of  PBL  in  the 
acquisition  of  either  basic  or  clinical  knowledge.  Nev¬ 
ertheless,  they  concluded  that  PBL  resulted  in  similar 
achievement  as  did  traditional  methods,  which  implied 
that  PBL  would  not  undermine  students’  acquisition 
of  domain  knowledge. 

Even  though  there  is  consensus  that  PBL  curricula 
result  in  better  knowledge  application  and  clinical  rea¬ 
soning  skills  but  perform  less  well  in  basic  or  factual 
knowledge  acquisition  than  traditional  curriculum, 
McParland  et  al.  (2004)  demonstrated  that  undergradu¬ 
ate  PBL  psychiatry  students  significantly  outperformed 
their  counterparts  in  examinations,  which  consisted  of 
multiple-choice  questions.  Equivalent  performance  on 
basic  science  knowledge  acquisition  (or  USMLE  step 
1)  and  knowledge  application  and  clinical  reasoning  (or 
USMLE  step  2)  between  students  learning  under  PBL 
curriculum  and  traditional  curricula  was  reported  in  sev¬ 
eral  studies  ( Alley ne  et  al.,  2002;  Antepohl  and  Herzig, 
1999;  Blue  et  al.,  1998;  Distlehorst  et  al.,  2005;  Prince 
et  al.,  2003;  Tomczak,  1991;  Verhoeven  et  al.,  1998). 

Retention  of  Content 

With  respect  to  students’  retention  of  content,  PBL 
research  revealed  an  interesting  tendency.  In  terms  of 
short-term  retention,  either  no  difference  was  found 
between  PBL  and  traditional  students  (Gallagher  and 
Stepien,  1996)  or  PBL  students  recalled  slightly  less 
(Dochy  et  al.,  2003);  yet,  PBL  students  consistently 
outperformed  traditional  students  on  long-term  reten¬ 
tion  assessments  (Dochy  et  al.,  2003;  Martenson  et  al., 
1985;  Tans  et  al.,  1986,  as  cited  in  Norman  and 
Schmidt,  1992).  In  reviewing  the  studies  that  investi¬ 
gated  the  effects  of  PBL  over  time,  Norman  and 
Schmidt  (1992)  found  some  interesting  results  in  sev¬ 
eral  studies.  Tans  and  associates  found  that  PBL  stu¬ 
dents’  recall  was  up  to  five  times  greater  on  the  con¬ 
cepts  studied  than  traditional  students  6  months  after 
the  course  was  completed.  The  study  by  Martenson  et 
al.  (1985)  showed  that  no  difference  was  found  in  the 
short-term  retention  of  the  content  between  PBL  stu¬ 
dents  and  traditional  students;  however,  the  PBL  stu¬ 
dents’  long-term  retention  rate  (average  25  points  out 
of  40)  was  60%  higher  than  that  of  traditional  students 
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(average  16  points  out  of  40)  2  to  4-1/2  years  after  the 
course  was  completed.  Also,  the  PBL  students  tended 
to  remember  more  about  principles,  whereas  the  tra¬ 
ditional  students  retained  more  rote-memorization 
types  of  knowledge.  Similarly,  Eisensteadt  et  al.  (1990) 
discovered  that  PBL  students  retained  less  than  tradi¬ 
tional  students  in  the  immediate  recall  test.  Nonethe¬ 
less,  their  retention  rate  remained  rather  consistent  2 
years  later,  while  the  traditional  students’  retention  had 
declined  significantly.  Dochy  et  al.’s  (2003)  review  of 
PBL  studies  also  echoed  Norman  and  Schmidt’s  obser¬ 
vation.  Norman  and  Schmidt  (1992),  therefore,  con¬ 
cluded  that  PBL  might  not  improve  students’  initial 
acquisition  of  knowledge;  however,  the  deeper  pro¬ 
cessing  of  information  in  PBL  classes  appears  to  foster 
better  retention  of  knowledge  over  a  longer  period  of 
time. 

Problem-Solving  Skills 

Improving  problem-solving  skills  is  one  of  the  essen¬ 
tial  promises  of  PBL.  The  results  of  PBL  research  by 
and  large  support  this  assumption.  Gallagher  et  al. 
(1992)  conducted  an  experiment  using  an  interdisci¬ 
plinary  PBL  course  called  Science,  Society  and  Future 
(SSF)  on  gifted  high-school  students  with  a  compari¬ 
son  group  of  high-school  students.  They  found  that 
PBL  students  showed  a  significant  increase  in  the  use 
of  the  problem-finding  step  from  pretest  to  post-test, 
which  was  a  critical  problem-solving  technique.  In 
contrast,  in  the  post-test,  the  comparison  group  tended 
to  skip  the  problem-finding  step  and  move  directly 
from  the  fact-finding  step  to  the  implementation  step. 
The  result  suggested  that  PBL  is  effective  in  fostering 
students’  development  of  appropriate  problem-solving 
processes  and  skills. 

Moreover,  PBL  has  shown  a  positive  impact  on 
students’  abilities  to  apply  basic  science  knowledge 
and  transfer  problem-solving  skills  in  real-world  pro¬ 
fessional  or  personal  situations.  Lohman  and  Finkel- 
stein  (1999)  found  that  the  first-year  dental  education 
students  in  a  10-month  PBL  program  improved  signif¬ 
icantly  in  their  near  transfer  of  problem-solving  skills 
by  an  average  of  31 .3%,  and  their  far  transfer  of  prob¬ 
lem-solving  skills  increased  by  an  average  of  23.1%. 
Based  on  their  data,  they  suggested  that  repeated  expo¬ 
sure  to  PBL  was  the  key  for  facilitating  the  develop¬ 
ment  of  problem-solving  skills.  Several  studies  have 
shown  that  PBL  has  very  positive  effects  on  students’ 
transfer  of  problem-solving  skills  to  workplaces;  for 
example.  Woods  (1996)  reported  that  employers 
praised  McMaster  University’s  PBL  chemical  engi¬ 
neering  graduates’  outstanding  problem-solving  skills 
and  job  performance.  Compared  to  other  new  employ¬ 


ees  who  typically  required  1  to  1-1/2  years  of  on-the- 
job  training  to  be  able  to  solve  problems  indepen¬ 
dently,  “  [the  PBL  graduates]  think  for  themselves  and 
solve  problems  upon  graduation”  (Woods,  1996,  p. 
97).  Kuhn’s  (1998)  study  also  illustrated  the  rapid 
development  of  expertise  of  first-year  PBL  residents 
in  the  emergency  room.  A  superior  ability  to  synthesize 
basic  knowledge  and  clinical  experience  (Patel  et  al., 
1991),  in  addition  to  applying  and  transferring  the 
knowledge  and  skills  into  the  workplace,  may  explain 
why  PBL  students  outperformed  traditional  students 
in  NBME/USMLE  Part  2  while  PBL  students  seem¬ 
ingly  possessed  slightly  less  basic  science  knowledge 
than  traditional  students  as  shown  in  their  performance 
in  NBME/USMLE  Part  1 .  Clinical  reasoning  and  solv¬ 
ing  problems  on  the  job  require  more  than  mere  mem¬ 
orization  of  factual  knowledge.  Norman  and  Schmidt 
(1992)  pointed  out  that  no  evidence  exists  to  confirm 
PBL  advantages  in  general  problem-solving  skills  that 
are  content  free,  which,  again,  supports  the  effective¬ 
ness  of  authentic,  contextualized  learning  in  PBL. 

Higher  Order  Thinking 

Higher  order  thinking  is  an  important  cognitive  skill 
required  for  developing  sophisticated  problem-solving 
skills  and  executing  complex  ill-structured  problem¬ 
solving  processes.  To  be  an  effective  problem  solver, 
students  need  to  possess  analytical,  critical  thinking, 
and  metacognitive  skills.  Articulating  problem  spaces 
requires  analytical  skills  (Newell  and  Simon,  1972), 
evaluating  information  involves  critical  thinking  skills, 
and  reflecting  on  one’s  own  problem-solving  process 
requires  metacognitive  skills.  Shepherd  (1998) 
reported  that  fourth-  and  fifth-grade  students  gained  a 
significantly  greater  increase  in  critical  thinking  skills 
measured  by  the  Cornell  Critical  Thinking  Test 
(CCTT)  than  did  the  comparison  group  after  partici¬ 
pating  in  a  9-week  PBL  course  (the  Probe  Method). 
Schlundt  et  al.  (1999)  also  observed  an  improvement 
of  self-efficacy  in  insulin  administration  management, 
problem-solving  skills,  and  flexibilities  in  choosing 
coping  strategies  to  overcome  the  difficulty  of  dietary 
adherence  among  adolescent  diabetic  patients  who 
received  a  2-week  PBL  summer  program.  They  con¬ 
cluded  that,  instead  of  just  teaching  the  facts,  the  PBL 
course  helped  the  patients  rationalize  the  self-care 
guidelines  and  consider  more  alternatives  to  seek  bet¬ 
ter  solutions  and  strategies  to  cope  with  the  difficult 
lifestyle.  Furthermore,  in  a  longitudinal  study  of  the 
problem-solving  performance  of  medical  students 
using  PBL  and  traditional  methods,  Hmelo  (1998) 
observed  that  students’  problem-solving  skills  and  pro¬ 
cesses  changed  qualitatively  over  time.  This  change 
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was  certainly  influenced  by  the  type  of  curriculum. 
The  students  in  the  PBL  curriculum,  she  noted,  gen¬ 
erated  more  accurate  hypotheses  and  coherent  expla¬ 
nations  for  their  hypotheses,  used  hypothesis-driven 
reasoning,  and  also  were  more  likely  to  explain  their 
hypotheses  and  findings  with  science  concepts  as  com¬ 
pared  to  traditional  students. 

Self-Directed  Learning/Life-Long  Learning 

The  ultimate  goal  of  PBL  is  to  educate  students  to  be 
self-directed,  independent,  life-long  learners.  Through 
actively  executing  problem-solving  processes  and 
observing  tutors’  modeling  problem-solving,  reason¬ 
ing,  and  metacognitive  processes,  PBL  students  learn 
how  to  think  and  learn  independently.  Though  their  data 
did  not  support  the  superiority  of  PBL  on  knowledge 
or  general  problem-solving  skills  acquisition,  Norman 
and  Schmidt  (1992)  concluded  that  PBL  appeared  to 
enhance  self-directed  learning.  This  conclusion  was 
supported  by  Woods’  (1996)  assessment  of  chemical 
engineering  students’  comfort  level  toward  self- 
directed  learning.  Ryan  (1993)  also  reported  a  signifi¬ 
cant  increase  in  PBL  students’  perceptions  of  their  abil¬ 
ities  as  self-directed  learners  at  the  end  of  the  semester 
in  a  health-science-related  course.  Moreover,  Blumberg 
and  Michael  (1992)  used  students’  self-reports  and 
library  circulation  statistics  as  measures  of  students’ 
self-directed  learning  behaviors  between  a  PBL  class 
(partially  teacher-directed)  and  a  lecture-based  class. 
They  concurred  that  PBL  promoted  self-directed  learn¬ 
ing  behaviors  in  students.  Similar  evidence  was  also 
found  in  a  number  of  studies,  such  as  those  by  Coulson 
and  Osborne  (1984),  Dwyer  (1993),  Dolmans  and 
Schmidt  (1994),  and  van  den  Hurk  et  al.  (1999). 

The  long-term  effects  of  PBL  on  helping  students 
develop  self-directed/life-long  learning  skills  and  pro¬ 
fessional  preparation  was  even  more  evident  in  other 
research  results.  Two  studies  revealed  that  PBL  grad¬ 
uates  rated  themselves  better  prepared  professionally 
than  their  counterparts  in  terms  of  interpersonal  skills, 
cooperation  skills,  problem-solving  skills,  self- 
directed  learning,  information  gathering,  professional 
skills  (e.g.,  running  meetings),  and  the  ability  to  work 
and  plan  efficiently  and  independently  (Schmidt  and 
van  der  Molen,  2001;  Schmidt  et  al.,  2006).  Moreover, 
in  Woods’  (1996)  study  mentioned  before,  the  PBL 
alumni  and  the  employers  who  hired  the  PBL  gradu¬ 
ates  gave  highly  positive  comments  regarding  their 
self-directedness  and  independence  in  solving  work- 
related  problems  and  improving  professional  develop¬ 
ment.  These  studies  provided  strong  evidence  for  the 
positive  long-term  effects  of  PBL  on  students’  self- 
directed  and  life-long  learning  skills  and  attitudes. 


Reflection  is  another  essential  element  required  for 
self-directed  learning  in  PBL  (Barrows  and  Myers, 
1993).  The  reflective  inquiry  process  used  in  the  study 
by  Chrispeels  and  Martin  (1998)  provided  the  students 
in  an  administrative  credential  program  with  a  meta¬ 
cognitive  framework.  This  reflective  process  helped 
the  students  become  effective  problem  solvers  by  exer¬ 
cising  higher  order  thinking  skills  to  identify  personal 
and  organizational  factors  that  constituted  the  admin¬ 
istrative  problems  they  faced  in  work  settings. 

Self-Perception  and  Confidence 

From  students’  perspectives,  the  effects  of  PBL  have 
been  positively  perceived.  Numerous  studies  have 
shown  that  students  consider  PBL  to  be  effective  in 
promoting  their  learning  in  dealing  with  complex  prob¬ 
lems  (Martin  et  al.,  1998),  enhancing  their  confidence 
in  judging  alternatives  for  solving  problems  (Dean, 
1999),  acquiring  social  studies  content  (Shepherd, 
1998),  enriching  their  learning  of  basic  science  infor¬ 
mation  (Caplow  et  al.,  1997),  developing  thinking  and 
problem-solving  skills  (Lieux,  2001),  improving  inter¬ 
personal  and  professional  skills  (Schmidt  and  van  der 
Molen,  2001;  Schmidt  et  al.,  2006),  and  advancing 
self-directed  learning,  higher  level  thinking,  and 
enhancement  of  information  management  skills  (Kauf¬ 
man  and  Mann,  1996). 

In  summary,  PBL  research  results  overall  have 
clearly  demonstrated  advantages  of  PBL  for  preparing 
students  for  real-world  challenges.  The  emphasis  of 
PBL  curricula  on  application  of  domain  knowledge, 
problem  solving,  higher  order  thinking,  and  self- 
directed  learning  skills  equips  students  with  profes¬ 
sional  and  life-long  learning  habits  of  mind,  which  are 
indispensable  qualities  of  successful  professionals. 
Although  PBL  students’  performance  in  basic  domain 
knowledge  acquisition  has  been  slightly  inferior  to 
traditional  students,  the  format  of  the  tests  and  the 
time-delay  effects  (PBL  students  have  been  found  to 
retain  information  much  longer  and  better  than  tradi¬ 
tional  students)  may  justify  this  result.  This  specula¬ 
tion  may  suggest  further  research  issues  and  merit 
empirical  evidence  to  shed  deeper  insight  on  these 
aspects  of  PBL. 

Implementation  of  PBL 

Problem-based  learning  is  considered  by  many 
researchers  to  be  the  most  innovative  instructional 
method  to  date.  As  indicted  before,  these  beliefs  are 
anchored  in  PBL's  atypical  instructional  process  and 
components.  They  include  learning  initiated  by  prob¬ 
lems,  self-directed  learning,  and  collaborative  learning 
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in  small  groups.  These  components,  which  are  radi¬ 
cally  different  from  traditional  instructional  methods, 
inevitably  produce  a  considerable  impact  on  the 
dynamics  between  instructors  and  students,  among 
students,  and  on  instructors  and  students’  roles  and 
responsibilities  during  the  course  of  PBL. 

Student  Roles,  Tutor  Roles,  and  Tutoring  Issues 

The  students  as  well  as  instructors  have  encountered 
great  challenges  when  transitioning  from  traditional 
instructional  methods  to  PBL.  These  challenges  might 
have  evolved  from  students’  as  well  as  tutors’  inter¬ 
pretations  of  self-directed  learning.  According  to  Mif- 
lin  and  associates  (Miflin,  2004;  Miflin  et  al.,  1999, 
2000),  self-directed  learning  in  PBL  could  range  from 
preorganized  teaching,  student-initiated  and  -selected 
but  instructor-guided  learning,  to  completely  self- 
taught  learning.  This  wide  spectrum  of  interpreting 
self-directed  learning  could  have  contributed  to  the 
confusion  or  unsettled  feeling  for  the  students  while 
defining  their  roles  in  PBL  courses .  Similar  uncertainty 
also  occurred  with  the  tutors  when  assuming  their  roles 
in  the  students’  learning  process.  In  the  following  sec¬ 
tions,  we  discuss  the  perceptions  of  students  as  well 
as  tutors  in  terms  of  their  roles  in  the  PBL  processes, 
as  well  as  the  tutoring  factors  that  influence  student 
learning. 

Students’  Transition  from  Traditional  Methods  to  PBL 

In  PBL,  the  students  become  the  initiators  of  their  own 
learning,  the  inquirers  and  problem  solvers  during  the 
learning  process,  and  they  are  no  longer  passive  infor¬ 
mation  receivers.  The  students  not  only  are  required 
to  redefine  their  roles  in  the  learning  process  but  must 
also  retune  their  learning  habits.  Woods  (1994,  1996) 
speculated  that  uncertainty  about  their  grades  was  one 
possibility  accounting  for  students’  uneasiness  about 
a  new  instructional  method,  resulting  in  some  resis¬ 
tance  to  change  and  making  the  initial  transition  from 
traditional  curriculum  to  PBL  curriculum  more  diffi¬ 
cult.  Schmidt  et  al.  (1992)  reported  that  students  need 
at  least  6  months  to  adapt  to  this  new  instructional 
method.  The  concern  about  the  sufficiency  of  content 
coverage  also  partially  contributed  to  students’  anxiety 
during  PBL  (Lieux,  2001;  Schultz-Ross  and  Kline, 
1999).  Jost  et  al.  (1997)  examined  students’  discomfort 
levels  with  PBL  in  the  initial  stage  of  instruction  by 
analyzing  the  students’  journals,  self-evaluations,  and 
a  survey.  They  found  that  the  students’  anxiety  mainly 
resulted  from  their  uncertainty  about  their  roles  and 
responsibilities  in  the  course  and  how  they  would  be 
evaluated.  The  difficulty  of  assuming  a  more  active 
role  with  more  responsibility  in  the  learning  process 


also  results  from  the  students’  “learned”  definition  of 
roles  in  traditional  methods  (Dean,  1999;  Jost  et  al., 
1997,  p.  90).  Similar  observations  were  also  reported 
in  studies  by  Fiddler  and  Knoll  (1995),  Dabbagh  et  al. 
(2000),  and  Lieux  (2001).  Furthermore,  as  Miflin  and 
associates  (1999,  2000)  conjectured,  the  questionable 
presumption  that  adult  learners  are  capable  of  conduct¬ 
ing  highly  self-directed  learning  may  also  play  a  role 
in  students’  difficulties  in  transiting  to  PBL. 

Although  the  sense  of  discomfort  and  anxiety  is 
common  among  students  during  the  initial  stage  of 
PBL  implementation,  Schultz-Ross  and  Kline  (1999) 
found  that  the  students’  discomfort  and  dissatisfaction 
levels  decreased  significantly  by  the  end  of  a  PBL 
forensic  psychiatry  course.  They  reported  that  some 
students  expressed  uneasiness  during  the  initial  tran¬ 
sition  stage  of  PBL  curriculum.  Nonetheless,  once  the 
students  adjusted  to  PBL  environments  and  realized 
the  merits  of  PBL,  their  perceived  comfort  levels  about 
the  learning  issues  of  testimony,  liability,  and  compe¬ 
tence  improved  significantly,  as  did  their  perceptions 
regarding  the  subject  matter  (forensic  psychiatry) 
learned  in  the  course.  Dabbagh  et  al.  (2000)  confirmed 
Schultz-Ross  and  Kline’s  observation. 

Tutors’  Roles  in  PBL 

Barrows  (1992)  asserted  that  the  two  major  responsi¬ 
bilities  of  tutors  in  PBL  are  facilitating  the  students’ 
development  of  thinking  or  reasoning  skills  that  pro¬ 
mote  problem  solving,  metacognition,  and  critical 
thinking,  as  well  as  helping  them  to  become  indepen¬ 
dent  and  self-directed  learners.  As  Maudsley  (1999) 
stated,  the  effectiveness  of  tutors  is  essential  to  the 
success  of  PBL.  Maudsley  suggested  that  PBL  pro¬ 
vides  an  opportunity  for  educators  to  redefine  the 
nature  of  learning  and,  in  turn,  reposition  their  roles 
in  teaching  from  a  knowledge/information  transmitter 
to  a  leaming/thinking  process  facilitator.  This  shift 
requires  PBL  tutors  to  undergo  a  fundamental  recon¬ 
ceptualization  of  their  educational  roles.  Research 
showed  that,  after  having  gone  through  this  reconcep¬ 
tualization  process,  a  conceptual  shift  similar  to  that 
of  the  students  also  occurred  among  tutors. 

Based  on  their  data,  Donaldson  and  Caplow  (1996) 
described  the  PBL  tutor’s  precarious  position  as  a 
dilemma.  Their  research  on  the  role  expectations  of 
PBL  tutors  revealed  two  major  dilemmas  perceived  by 
PBL  tutors:  the  conceptualization  of  facilitator  and  the 
tensions  that  arise  as  tutors  tried  to  redefine  their  role 
in  PBL  as  compared  to  their  previous  role  as  medical 
teacher.  Naturally,  PBL  tutors’  adjustments  and  per¬ 
haps  some  discomfort  about  their  new  roles  were  inev¬ 
itable  and  anticipated.  Margetson  (1998)  argued  that 
this  paradigm  shift  in  instructional  strategy  could  be 
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threatening  to  teachers  who  need  to  maintain  control 
of  the  learning  environment  and  prefer  passive  stu¬ 
dents.  In  addition,  teachers  who  conceive  knowledge 
as  a  body  of  information  that  should  be  transmitted 
from  the  knowledgeable  teacher  to  the  unknowing  stu¬ 
dent  could  also  feel  threatened  by  the  PBL  process; 
thus,  Maudsley  (1999)  cautioned  that  the  PBL  tutor 
must  balance  a  degree  of  participation  in  students’ 
learning  processes  and  refrain  from  the  temptation  to 
lecture.  Aguiar  (2000)  conducted  an  exploratory  qual¬ 
itative  case  study  that  examined  teachers’  perceptions 
and  experiences  in  their  roles  as  PBL  tutors.  Five  main 
themes  emerged  describing  how  tutors  perceived  their 
roles  within  PBL:  (1)  facilitating  group  work,  (2)  role 
modeling,  (3)  providing  feedback,  (4)  imparting  infor¬ 
mation,  and  (5)  supporting  students’  professional 
development.  Furthermore,  Wilkerson  and  Hundert 
(1998)  described  the  challenge  of  multiple  roles  expe¬ 
rienced  by  PBL  tutors  and  assigned  the  following 
names  to  the  roles  they  identified  in  PBL  tutors:  infor¬ 
mation  disseminator,  evaluator,  parent,  professional 
consultant,  confidant,  learner,  and  mediator. 

Cognitive  Congruence  and  Active  Involvement 

Schmidt  and  Moust  (1995)  introduced  the  concept  of 
cognitive  congruence  as  a  necessary  characteristic  of 
an  effective  PBL  tutor.  Cognitive  congruence  is  com¬ 
munication  skills  defined  as  "the  ability  to  express 
oneself  in  the  language  of  the  students,  using  the  con¬ 
cepts  they  use  and  explaining  things  in  ways  easily 
grasped  by  students”  (Schmidt  and  Moust,  1995,  p. 
709).  The  effective  communication  skills  are  a  premise 
for  the  other  components  of  effective  tutoring.  More¬ 
over,  the  authenticity  of  tutors’  interactions  is  exhibited 
in  their  ability  to  communicate  with  students  infor¬ 
mally  while  maintaining  an  empathetic  attitude.  In 
addition,  effective  tutors  must  be  willing  to  be  actively 
involved  with  students.  In  the  study  by  Martin  et  al. 
(1998),  over  75%  of  the  students  felt  that  the  faculty 
involved  in  the  PBL  course  were  passive  and  believed 
that  their  learning  experiences  would  have  been  better 
if  the  faculty  had  more  actively  supported  the  students. 
The  students’  perceptions  of  tutors’  passive  involve¬ 
ment  may  have  resulted  from  the  tutors’  misinterpre¬ 
tation  of  self-directed  learning  as  self-taught  learning 
discussed  earlier. 

Modeling  Metacognition  Skills 
and  Self-Directed  Learning 

Mayo  et  al.  (1993)  examined  students’  perceptions  of 
tutor  effectiveness  in  a  PBL  surgery  clerkship,  and 
their  data  indicated  the  importance  of  the  tutor  as  a 
“metacognitive  guide.”  As  metacognitive  guides,  PBL 
tutors  help  promote  students’  development  of  clinical 


reasoning  skills  through  actively  modeling  this  process 
for  the  students.  While  not  giving  the  answers,  the 
tutors  model  what  questions  an  expert  physician  would 
ask  in  a  clinical  setting  and  guide  students  to  formulate 
questions  as  expert  physicians  would.  Similar  results 
were  also  obtained  in  Wilkerson’s  (1995)  examination 
of  students’  perceptions  of  effective  tutors.  The  results 
of  a  similar  study  conducted  by  Donaldson  and  Caplow 
(1996)  echoed  previous  findings  that  effective  tutors 
fell  into  three  categories  of  role  content:  ( 1 )  facilitation 
expertise,  (2)  knowledge  or  cognitive  expertise,  and 
(3)  clinical  reasoning  expertise.  Students  deemed 
tutors  as  effective  and  helpful  when  they  encouraged 
students  to  critically  evaluate  the  information  gathered, 
questioned  and  probed  the  students’  clinical  reasoning 
processes,  and,  most  importantly,  allowed  students  to 
control  the  learning  process.  Questioning  the  students’ 
clinical  reasoning  processes  serves  two  functions:  ver¬ 
ifying  the  appropriateness  of  the  students’  reasoning 
and  modeling  expert  physician’s  reasoning  processes. 
Allowing  self-control  in  the  learning  process  is  essen¬ 
tial  for  students  to  develop  self-directedness  in  their 
own  learning. 

Group  Processing 

Collaborative  learning  is  another  essential  element  of 
PBL.  A  study  by  Martin  et  al.  (1998)  indicated  that 
collaborative  group  processing  in  PBL  was  identified 
as  an  enhancer  for  the  students’  metacognitive  skills. 
Utilization  of  collaborative  learning  in  instruction  is 
theoretically  sound;  however,  it  may  not  be  as  straight¬ 
forward  as  it  sounds  in  practice.  Achilles  and  Hoover 
(1996)  pointed  out  a  major  concern  in  their  study  of 
implementing  PBL  at  grades  6  to  12  that  students  had 
difficulty  working  in  groups.  The  need  for  effective 
guidance  of  group  processing  was  perceived  not  only 
by  K-12  students  but  also  by  the  medical  students. 
When  Mayo  et  al.  (1993)  examined  tutor  effectiveness 
in  facilitating  group  processing,  they  found  that  tutor 
skills  differed  significantly.  When  44  students  evalu¬ 
ated  16  tutors  using  12  characteristics  determined  to 
be  essential  to  tutor  effectiveness,  the  results  revealed 
four  consequential  facilitation  skills:  (1)  helping  the 
group  be  aware  of  how  group  processing  works,  (2) 
encouraging  feedback  within  the  group,  (3)  guiding 
the  group  to  set  appropriate  learning  issues,  and  (4) 
assisting  the  group  to  integrate  learning  issues. 

Similarly,  De  Grave  et  al.  (1999)  used  the  Tutor 
Intervention  Profile  (TIP)  to  assess  the  effectiveness 
of  PBL  tutors  and  found  that  mastering  the  enhance¬ 
ment  of  the  learning  process  in  the  tutorial  group  was 
one  of  the  characteristics  that  the  students  valued. 
Thus,  researchers  have  suggested  that  the  skills  and 
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knowledge  for  creating  productive  collaboration  rela¬ 
tionships  (Wilkerson,  1995)  and  an  unthreatening 
working  atmosphere  (Schmidt  and  Moust,  1995)  are 
critical.  Furthermore,  group  size  has  also  been  found 
to  be  a  factor  that  has  potential  effects  on  students’ 
learning  processes  and  outcomes.  In  studying  the 
effects  of  group  size  on  students’  self-directedness, 
Lohman  and  Finkelstein  (2000)  found  that  the 
medium-sized  group  (six  students)  performed  signifi¬ 
cantly  better  than  the  large  group  (nine  students). 
Group  processing  is  especially  difficult  when  PBL  is 
implemented  in  a  large  class.  To  address  this  issue, 
Shipman  and  Duch  (2001)  suggested  that  more  struc¬ 
ture  of  group  processes  is  needed  for  facilitating  large 
PBL  classes.  An  interesting  finding  obtained  by 
Elshafei  (1998)  was  that,  when  students’  higher  level 
thinking  in  solving  algebra  problems  was  assessed, 
PBL  did  not  show  positive  effects  on  students’  perfor¬ 
mance  when  they  were  tested  individually;  yet,  PBL 
appeared  to  be  more  effective  when  the  students  were 
tested  in  groups.  This  finding,  whether  PBL  students’ 
learning  outcomes  are  collective  or  individual,  seems 
worth  pondering  and  pursuing  further. 

Expert  Knowledge 

The  importance  of  expert  knowledge  is  a  relatively 
uncertain  characteristic  in  facilitating  group  process¬ 
ing.  When  assessing  the  effectiveness  of  PBL  tutors 
with  TIP,  De  Grave  et  al.  (1999)  suggested  that  the 
occurrence  of  effective  tutoring  depended  heavily  on 
the  use  of  expert  knowledge.  Also,  Schmidt  and  Moust 
(1995)  asserted  that  a  suitable  knowledge  base  with 
regard  to  the  topic  under  study  was  imperative;  yet, 
others  (Davis  et  al.,  1992;  Silver  and  Wilkerson,  1991) 
have  raised  concerns  that  content  experts  tend  to  lec¬ 
ture  and  give  explanations,  which  may  undermine  the 
intent  of  promoting  students’  self-directed  learning. 
The  majority  of  the  research  pertaining  to  this  debate 
showed  no  significant  differences  in  tutors’  perfor¬ 
mance  and  students’  perception  about  tutorial  pro¬ 
cesses  between  content-experts  and  non-content- 
experts  (Gilkison,  2003;  Kaufman  and  Holmes,  1998; 
Regehr  et  al.,  1995).  Students  generally  rated  expert 
tutors  more  effective  than  non-expert  tutors,  and  in 
some  of  the  studies  (such  as  that  by  Eagle  et  al.,  1992), 
students  performed  slightly  better  with  expert  tutors 
than  with  non-expert  tutors.  Yet,  using  students’  per¬ 
ceptions  and  immediate  learning  outcomes  as  mea¬ 
sures  of  effective  tutoring  could  have  masked  what 
really  happened;  for  example,  Kaufman  and  Holmes 
(1998)  observed  that  expert  tutors  have  a  more  difficult 
time  with  the  role  of  facilitator  and  tend  to  provide 
more  explanations  of  case  content.  Similarly,  Gilkison 
(2003)  noted  that  the  expert  tutor  initiated  more  topics 


for  discussion  than  the  non-expert  tutor  (52%  vs. 
12.5%),  and  the  non-expert  tutor  engaged  more  in 
facilitating  group  processes  (55.9%  vs.  38.5%)  and 
less  in  directing  learning  (5.9%  vs.  11.4%).  These 
observations  offered  a  plausible  explanation  for  the 
expert  tutors’  better  performance.  Further  examination 
of  the  interaction  of  expert  and  non-expert  tutoring  and 
students’  development  of  self-directed  learning  skills 
would  provide  better  insight  on  this  issue.  Also,  the 
implications  of  these  observations  should  be  taken  into 
account  by  PBL  curriculum  developers  or  designers 
when  considering  employing  tutors  who  possess 
expert  knowledge. 

Assessment  Issues 

The  assessment  used  in  the  early  implementation  of 
PBL  largely  relied  on  traditional  U.S.  board  exams, 
which  were  standardized  tests  designed  to  assess  stu¬ 
dents’  factual  knowledge  (NBME  step  1)  and  clinical 
reasoning  (NBME  step  2).  Nendaz  and  Tekian  (1999) 
criticized  traditional  assessment  as  not  being  in  line 
with  the  principles  of  PBL;  therefore,  the  PBL  stu¬ 
dents’  performance  might  have  been  at  a  disadvantage 
under  traditional  assessment.  Fortunately,  Blake  et  al. 
(2000)  noted  a  shift  in  USMLE  in  more  recent  years 
such  that  the  emphasis  of  assessment  has  moved  from 
testing  factual  knowledge  to  assessing  application  of 
the  knowledge.  This  change  not  only  benefits  the  stu¬ 
dents  who  study  under  PBL  curriculum  but  also  signals 
an  increasing  attention  to  students’  abilities  to  apply 
and  transfer  basic  knowledge  instead  of  focusing  on 
factual  knowledge  acquisition.  A  number  of  different 
methodologies  have  been  developed  to  assess  students’ 
problem-solving  skills,  reasoning  skills,  and  personal 
progress;  for  example,  according  to  the  classification 
by  Swanson  et  al.  (1998),  there  are  outcome-oriented 
instruments,  such  as  the  progress  test  (Van  der  Vleuten 
et  al.,  1996),  essay  exams,  oral  and  structured  oral 
examinations,  patient-management  problems,  clinical 
reasoning  exercises  (Wood  et  al.,  2000),  problem-anal¬ 
ysis  questions  (Des  Marchais  et  al.,  1993),  and  stan¬ 
dardized  patient-based  tests,  as  well  as  process-ori¬ 
ented  instruments,  such  as  the  triple-jump-based 
exercises  (Smith,  1993),  Medical  Independent  Learn¬ 
ing  Exercise  (MILE)  (Feletti  et  al.,  1984),  the  four- 
step  assessment  test  (4SAT)  (Zimitat  and  Miflin, 
2003),  formative  assessment  (Neufeld  et  al.,  1989), 
and  tutor,  peer,  and  self-assessment. 

As  Savin-Baden  (2004)  contended,  assessment  is 
probably  one  of  the  most  controversial  issues  in  PBL 
because  it  is  probably  the  most  important  indicator  for 
validating  its  effectiveness .  The  mixed  results  of  PBL 
students’  learning  outcomes  discussed  earlier  might 
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have  been  largely  due  to  incomparable  assessment 
being  used.  In  their  meta- analysis  of  PBL  research, 
Gijbels  et  al.  (2005)  found  that  the  effects  of  PBL 
varied  mostly  depending  on  the  focus  of  assessment 
instrument  used.  PBL  had  the  most  positive  effects 
when  the  instrument  focused  on  assessing  the  under¬ 
standing  of  principles  that  link  concepts.  This  may 
explain  the  pattern  seen  in  PBL  research  that  tradi¬ 
tional  students  performed  better  in  basic  knowledge 
acquisition  while  PBL  students  did  better  in  applica¬ 
tion  of  knowledge  and  clinical  reasoning.  Reviewing 
the  assessment  in  the  medical  schools  implementing 
PBL  from  1966  to  1998,  Nendaz  and  Tekian  (1999) 
concluded  that  a  lack  of  uniformity  existed  with  regard 
to  the  assessment  methodologies  used  in  measuring 
PBL  students’  performance. 

Curriculum  Design  in  PBL 

A  distinct  characteristic  of  PBL  is  that  learning  is 
initiated  by  presenting  a  problem  rather  than  teaching 
the  content.  If  so,  what  is  instruction  in  PBL?  To  this 
question,  Barrows  (1996,  p.  8)  stated  that:  “The  cur¬ 
ricular  linchpin  in  PBL  ...  is  the  collection  of  problems 
in  any  given  course  or  curriculum  with  each  problem 
designed  to  stimulate  student  learning  in  areas  relevant 
to  the  curriculum.”  This  collection  of  problems  is 
designed  to  fulfill  four  educational  objectives  in  PBL: 
(1)  structuring  of  knowledge  for  use  in  clinical  con¬ 
texts,  (2)  developing  an  effective  clinical  reasoning 
process,  (3)  developing  effective  self-directed  learning 
skills,  and  (4)  increasing  motivation  for  learning  (Bar- 
rows,  1986,  pp.  481-482).  Based  on  these  educational 
objectives,  Barrows  developed  a  taxonomy  for  classi¬ 
fying  PBL  curricula  into  six  categories  using  two  vari¬ 
ables  with  three  levels  each.  The  two  variables  include 
the  degrees  of  self-directedness  and  problem  struc¬ 
turedness.  He  further  defined  the  three  levels  of  the 
variable  of  self-directedness  as  teacher-directed,  stu¬ 
dent-directed,  and  partially  student  and  teacher 
directed.  The  three  levels  of  the  variable  of  problem 
structuredness  were  defined  as  complete  case,  partial- 
problem  simulation,  and  full-problem  simulation  (free 
inquiry).  The  combination  of  the  two  variables  and 
three  levels  creates  a  categorization  of  PBL  curriculum 
design,  which  includes  lecture-based  cases,  case-based 
lectures,  case  method,  modified  case-based,  problem- 
based,  and  closed-loop  problem-based.  The  decision 
regarding  which  category  of  PBL  design  a  given  PBL 
curriculum  should  take  should  be  based  on  the  degree 
of  the  educational  objectives  that  must  be  reached  and 
the  characteristics  of  learners. 

In  more  recent  developments  of  PBL  curriculum 
design,  students  were  gradually  being  included  in  the 


curriculum  design  process  to  provide  insights  from 
students’  perspectives.  Chung  and  Chow  (2004) 
reported  that  the  students’  workload  and  assessment 
methods  designed  in  the  curriculum  were  improved  to 
better  address  students’  capabilities  and  promoted 
learning  when  student  representation  was  included  in 
the  curriculum  design  process.  In  medical  schools, 
PBL  curricula  are  usually  designed  by  a  team  of  fac¬ 
ulty  members  and  instructional  designers:  however, 
PBL  in  K-12  education  and  higher  education,  as  Max¬ 
well  et  al .  (200 1 )  indicated ,  is  often  adopted  by  a  single 
teacher  or  implemented  in  a  single  course  rather  than 
as  a  departmental  curriculum.  It  is  much  more  chal¬ 
lenging,  therefore,  for  individual  teachers  to  indepen¬ 
dently  design  PBL  problems  for  their  classes  without 
resources  and  support  from  administration  (Angeli, 
2002).  This  may  explain  the  considerably  fewer  imple¬ 
mentations  of  PBL  in  K-12  and  higher  education  set¬ 
tings  than  in  medical-related  fields. 

Problem  Design 

Given  that  a  PBL  curriculum  consists  of  a  collection 
of  problems,  there  is  no  doubt  that  the  problems  them¬ 
selves  are  crucial  to  the  success  of  PBL  (Duch,  2001; 
Trafton  and  Midge tt,  2001).  Perrenet  et  al.  (2000)  con¬ 
tended  that  students’  learning  could  be  enhanced  by 
manipulating  the  quality  of  PBL  problems  because 
they  in  fact  could  influence  students’  activation  of  prior 
knowledge,  their  group  processing,  self-directed  learn¬ 
ing  (Gijselaers  and  Schmidt,  1990),  and  generation  of 
useful  learning  issues  (Dolmans  et  al.,  1993).  Selecting 
and  writing  appropriate  and  effective  PBL  problems 
are  very  challenging  and  difficult  tasks  (Angeli,  2002); 
however,  the  issues  of  the  effectiveness  of  problems 
and  designing  PBL  problems  have  not  been  researched 
adequately. 

Effectiveness  of  Problems 

The  effectiveness  of  problems  determines  the  effec¬ 
tiveness  of  PBL  curriculum.  The  quality  of  PBL  prob¬ 
lems  affects  not  only  various  aspects  of  student  learn¬ 
ing  but  also  academic  achievement.  Ineffective 
problems  could,  as  Dolmans  et  al.  (1993)  argued,  cause 
students  difficulty  in  generating  learning  issues  that 
the  problem  is  designed  to  cover  and  hence  lead  to 
insufficient  content  knowledge  acquisition.  To  eluci¬ 
date  the  effectiveness  of  PBL  problems  used  in  med¬ 
ical  education,  four  studies  set  out  to  investigate  this 
issue,  and  they  yielded  very  similar  results.  According 
to  Dolmans  et  al.  (1993),  the  effectiveness  of  problems 
is  defined  as  the  degree  of  correspondence  between 
student-generated  learning  issues  and  faculty  objec¬ 
tives.  When  assessing  how  accurately  the  students 
identified  learning  issues  that  were  specified  by  the 
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faculty  to  a  given  problem,  Coulson  and  Osborne 
(1984)  found  that  on  average  students  identified  24.0 
learning  issues  out  of  39.3  objectives  (about  62%). 
Dolmans  et  al.  (1993)  analyzed  the  correspondence 
between  the  instructors’  intended  objectives  and  stu¬ 
dent-generated  learning  issues  based  on  their  interpre¬ 
tations  of  the  PBL  problems.  They  found  that  only 
64%  of  intended  content  was  identified  in  the  student¬ 
generated  learning  issues.  The  degrees  of  correspon¬ 
dence  between  objectives  specified  by  the  faculty  and 
the  student-generated  learning  issues  for  the  12  prob¬ 
lems  ranged  from  27.7  to  100%.  Similarly,  O’Neill 
(2000)  reported  a  62%  correspondence  rate  between 
faculty  objectives  and  student-generated  learning 
issues.  In  the  study  by  van  Gessel  et  al.  (2003),  a 
62%  match  between  faculty  and  student  objectives 
and  learning  issues  was  obtained.  In  these  four  stud¬ 
ies,  in  addition  to  the  student-generated  learning 
issues  that  matched  faculty  objectives,  irrelevant 
learning  issues  were  generated  by  the  students.  These 
results  showed  that  insufficient  content  coverage  in 
PBL  could  in  fact  occur;  hence,  without  assurance  of 
the  quality  of  problem  or  intended  aims  being  met, 
the  effects  of  PBL  would  be  unpredictable  and  there¬ 
fore  questionable. 

Problem  Design  Models  and  Principles 

A  number  of  researchers  have  discussed  and  provided 
suggestions  and  guidelines  for  designing  PBL  prob¬ 
lems;  for  example,  Duch  (2001)  suggested  a  process 
of  five  stages  of  writing  PBL  problems  (choose  a  cen¬ 
tral  idea,  think  of  a  real-world  context  for  the  concept, 
stage  the  problem  to  lead  students’  research,  write  a 
teacher’s  guide,  and  identify  resources  for  students). 
Lee  (1999)  proposed  a  decision  model  for  problem 
selection  in  which  selection  of  the  PBL  problem  is  a 
function  of  learning  objectives,  prior  knowledge, 
domain  knowledge,  problem  structuredness  and  com¬ 
plexity,  and  time  availability.  Aiming  at  promoting 
higher  order  thinking,  Weiss  (2003)  suggested  several 
principles  for  designing  PBL  problems,  including  con¬ 
sidering  students’  prior  knowledge,  using  ill-structured 
and  authentic  problems,  and  promoting  collaborative, 
life-long,  and  self-directed  learning.  Stinson  and  Mil¬ 
ter  (1996)  also  offered  design  guidelines  that  PBL 
problems  should  be  holistic,  ill-structured,  and  con¬ 
temporary  and  should  mirror  professional  practice.  A 
step-by-step  PBL  problem  development  cycle  was  pro¬ 
posed  by  Drummond- Young  and  Mohide  (2001).  This 
eight-step  design  process  was  designed  specifically  for 
nursing  education  and  includes  the  following  steps:  (1) 
review  expected  learning  outcomes,  (2)  determine  con¬ 
tent,  (3)  select  a  priority  health  issue  and  develop  the 
problem,  (4)  develop  supplementary  material,  (5)  seek 
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Figure  38.1  The  3C3R  problem-based  learning  (PBL)  problem 
design  model. 

evaluative  feedback,  (6)  pilot  the  problem,  (7)  revise 
and  refine  the  problem,  and  (8)  integrate  the  problem 
into  the  curriculum.  These  problem  design  guidelines, 
principles,  and  processes  are  very  helpful  yet  overly 
general  or  excessively  profession  specific;  therefore, 
they  are  inadequate  for  providing  educators  and  prac¬ 
titioners  with  a  complete  conceptual  framework  and 
the  systematic  design  process  required  for  designing 
effective  PBL  problems  for  learners  across  disciplines 
and  ages. 

Compared  to  PBL  research  on  student  learning 
outcomes,  tutor  techniques,  student  perceptions,  or 
group  processing,  research  on  PBL  problem  design  is 
rather  scarce  and  unsystematic.  To  provide  PBL  edu¬ 
cators  and  practitioners  with  a  systematic  conceptual 
framework  for  designing  effective  and  reliable  PBL 
problems.  Hung  (2006a)  introduced  the  3C3R  model 
as  a  conceptual  framework  for  systematically  design¬ 
ing  optimal  PBL  problems.  The  3C3R  PBL  problem 
design  model  is  a  systematic  method  specifically 
designed  to  guide  instructional  designers  and  educa¬ 
tors  to  design  effective  PBL  problems  for  all  levels 
and  across  disciplines  of  learners  by  strengthening  the 
characteristics  of  PBL  and  alleviating  implementation 
issues  revealed  in  previous  research  on  PBL,  such  as 
dilemmas  of  depth  vs.  breadth  of  content  and  factual 
knowledge  acquisition  vs.  problem-solving  skills 
acquirement  (Albanese  and  Mitchell,  1993;  Gallagher 
and  Stepien,  1996;  Hung  et  al.,  2003).  The  3C3R 
model  (see  Figure  38.1)  has  two  classes  of  compo¬ 
nents:  core  components  and  processing  components. 
The  core  components— content,  context,  and  connec¬ 
tion— are  primarily  concerned  with  the  issues  of 
appropriateness  and  sufficiency  of  content  knowledge, 
knowledge  contextualization,  and  knowledge  integra¬ 
tion.  The  processing  components— researching,  rea¬ 
soning,  and  reflecting— deal  with  students’  acquisition 
of  content  knowledge  and  the  development  of  prob¬ 
lem-solving  skills  and  self-directed  learning  skills. 
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Following  the  establishment  of  the  3C3R  model. 
Hung  (2006b)  further  developed  a  nine-step  problem 
design  process  to  operationalize  the  conceptual  frame¬ 
work  into  a  step-by-step  process: 

Step  1.  Set  goals  and  objectives. 

Step  2.  Conduct  content/task  analysis. 

Step  3.  Analyze  context  specification. 

Step  4.  Select/generate  PBL  problem. 

Step  5.  Conduct  PBL  problem  affordance  analysis. 

Step  6.  Conduct  correspondence  analysis. 

Step  7.  Conduct  calibration  processes. 

Step  8.  Construct  reflection  component. 

Step  9.  Examine  inter-supporting  relationships  of 
3C3R  components. 

Use  of  Technology  in  PBL 

The  use  of  technology  in  PBL  follows  two  major  tra¬ 
jectories:  distance  learning  and  use  of  multimedia. 

Distance  Learning  and  PBL 

Most  commonly,  PBL  takes  place  in  a  small  group 
with  intensive  face-to-face  discussions  among  students 
with  guidance  from  tutors.  With  the  development  of 
technology  and  increasing  popularity  of  the  Internet, 
more  and  more  online  or  distributed  PBL  (dPBL, 
defined  as  the  use  of  PBL  in  an  online  environment; 
Cameron  et  al.,  1999)  curricula  have  been  experi¬ 
mented  with  or  implemented  in  the  subject  areas  of, 
for  example,  social  economy  (Bjorck,  2002),  education 
(Orrill,  2002),  and  science  (Kim  et  al.,  2001).  To  sup¬ 
port  PBL  implementation  at  the  University  of  Dela¬ 
ware,  a  Web-based  technology  system  was  utilized  to 
help  instructors  organize  courses  (syllabi,  groups, 
projects,  and  student  reports)  and  to  facilitate  elec¬ 
tronic  communication  (discussion  sessions  and 
between  instructors  and  students),  as  well  as  provide 
online  resources  in  support  of  PBL  course  develop¬ 
ment,  such  as  ingredients  for  writing  problems,  inspi¬ 
ration  for  problem  design,  and  information  for  solving 
problems  (Watson,  2002). 

In  studying  the  effects  of  Internet  technology  on 
students’  learning  in  PBL,  Reznich  and  Werner 
(2001)  observed  a  general  positive  effect,  especially 
on  the  discussion  process,  in  which  the  tutors  played 
an  important  role  in  ensuring  the  success  of  the  group 
sessions  and  guiding  students  to  use  electronic 
resources.  In  reviewing  the  literature  of  online  PBL, 
we  found  that  better  access  and  retrieval  of  informa¬ 
tion  are  the  main  advantages  of  online  PBL 
(Helokunnas  and  Herrala,  2001 ;  Reznich  and  Werner, 
2001;  Watson,  2002);  however,  online  environments 
seemed  to  fail  to  deliver  the  promise  of  fostering 


collaborative  learning,  which  many  online  PBL  advo¬ 
cates  have  claimed,  due  to  unsophisticated  and  cum¬ 
bersome  technology  (Barrows,  2002;  Orrill,  2002). 

Use  of  Multimedia  in  PBL 

Utilizing  multimedia  in  constructing  PBL  environ¬ 
ments  is  also  gaining  more  attention  as  technology 
advances.  The  promise  of  using  multimedia  to  enhance 
PBL  is  based  on  the  assumption  that  PBL  should  take 
place  in  an  authentic  context  (Albion  and  Gibson, 
1998)  to  help  students  encode  specificity  of  informa¬ 
tion,  which  is  one  of  the  necessary  conditions  for  learn¬ 
ing  to  occur  (Schmidt,  1983).  Some  researchers  (such 
as  Hoffman  and  Ritchie,  1997)  have  argued  that  paper 
or  oral  presentation  of  PBL  problems  does  not  provide 
sufficient  contextual  or  environmental  information  to 
prepare  students  to  be  able  to  recognize  salient  visual, 
auditory,  or  nonverbal  cues  that  are  crucial  in  some 
professions  (Bridges,  1992).  This  implicit  contextual 
information,  such  as  social  conventions  or  phenome¬ 
non  and  cultural/cross-cultural  issues  (Conway  et  al., 
2002;  Yamada  and  Maskarinec,2004)  or  locality  (Hays 
and  Gupta,  2003),  is  lost  in  most  conventional  paper- 
or  oral -based  problem  cases.  This  argument  was  con¬ 
firmed  by  a  study  by  Kamin  et  al.  (2001)  of  the  effects 
of  different  modalities  on  students’  critical  thinking 
abilities  in  a  PBL  course.  They  presented  two  groups 
of  students  with  a  problem  in  text  format  or  video 
format.  The  results  revealed  that  the  video  group  did 
not  identify  as  much  of  the  information  given  in  the 
problem  as  the  text  group.  This  performance  of  the 
video  group  in  fact  better  resembled  real-life  situa¬ 
tions;  however,  the  video  group  examined  the  infor¬ 
mation  more  critically  than  the  text  group,  who  tended 
to  accept  the  face  value  of  the  information  given  in  the 
problem.  Also,  the  video  group  had  more  active  group 
processing  than  their  counterparts.  Bowdish  et  al. 
(2003)  reported  an  experiment  with  a  prototype  of 
VPBL  (virtual  PBL).  The  VPBL  incorporated  multiple 
modalities,  including  digital  video,  images,  text,  ques¬ 
tions,  and  text  boxes,  to  present  problem  scenarios  and 
facilitate  the  PBL  process.  This  VPBL  environment 
allowed  the  learners  to  observe  the  patient-doctor  con¬ 
versations  and  the  doctor’s  bedside  manner,  to  examine 
a  chief  complaint  (for  example,  listening  to  heart  and 
lung  sounds) ,  and  to  order  and  view  diagnostic  studies . 
To  their  surprise,  no  significant  difference  was  found 
in  the  students’  Teaching  and  Learning  Environment 
Questionnaire  (TLEQ)  and  achievement  scores 
between  the  text-based  group  and  the  VPBL  group. 
Similarly,  William  et  al.  (1998)  reported  that  no  dif¬ 
ference  existed  in  achievement  scores  when  comparing 
computer-based  and  paper  PBL  with  seventh  graders 
in  learning  science  concepts.  Another  case  of  using 
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multimedia  in  PBL  to  promote  situated  learning  was 
reported  by  Zumbach  et  al.  (2004).  Their  results  also 
showed  no  significant  differences  in  elementary  stu¬ 
dents’  factual  knowledge  acquisition  and  problem¬ 
solving  skills  under  the  multimedia-enhanced  PBL  or 
traditional  class;  however,  the  multimedia-enhanced 
PBL  class  showed  a  significantly  higher  level  of  moti¬ 
vation  to  learn  as  well  as  retention  of  knowledge  than 
did  the  traditional  class. 


DIRECTIONS  FOR  FUTURE  RESEARCH 

As  indicated  earlier  in  this  chapter,  PBL  is  an  instruc¬ 
tional  methodology.  The  PBL  model  calls  for  the  con¬ 
struction  of  problem  sets  of  authentic  problems  and 
the  engagement  of  learning  groups  in  negotiating 
learning  issues  to  solve  those  problems.  Although  PBL 
has  been  shown  to  be  successful  in  supporting  deep 
levels  of  understanding,  problem-solving  skills,  and 
lifelong  learning,  PBL  research  should  pay  more  atten¬ 
tion  to  the  nature  of  the  problems  being  solved.  The 
PBL  methodology  assumes  that  all  problems  are 
solved  in  the  same  way  and  can  be  learned  in  the  same 
way.  We  believe  that  this  is  a  questionable  assumption. 

Problem  Types  and  PBL 

Probably  the  most  important  research  question  is  that 
of  addressing  the  nature  of  problems  that  are  amenable 
to  PBL.  PBL  emerged  in  medical  schools,  where  stu¬ 
dents  learn  to  solve  diagnosis-solution  problems, 
which  are  moderately  ill  structured.  The  goal  of  diag¬ 
nosis  is  to  find  the  source  of  the  physiological  anom¬ 
aly;  however,  numerous  paths  can  lead  to  a  diagnosis. 
In  the  treatment  or  management  part  of  the  process, 
the  problem  often  becomes  more  ill  structured  because 
of  multiple  treatment  options,  patient  beliefs  and 
desires,  insurance  companies,  and  so  on. 

Problem-based  learning  has  migrated  in  academic 
institutions  to  law  schools,  where  students  learn  to 
construct  arguments  based  on  evidentiary  reasoning,  a 
complex  form  of  rule -using  problem.  PBL  is  becoming 
increasingly  popular  in  graduate  business  programs, 
where  students  primarily  solve  case  analysis  problems 
that  are  fairly  ill  structured.  As  PBL  migrates  to  other 
academic  programs,  such  as  engineering,  research 
must  be  focused  on  the  nature  of  the  problems  being 
solved  and  how  efficacious  PBL  methodologies  are  for 
those  kinds  of  problems.  Along  the  continuum  from 
well-structured  to  ill-structured  problems  (Jonassen, 
2000),  which  kinds  of  problems  can  be  effectively 
supported  using  PBL?  For  example,  can  PBL  be 
adapted  to  word  problems  in  physics,  despite  the  inau¬ 


thentic  nature  of  those  problems?  The  kind  of  problem 
that  engineers  most  commonly  solve  is  the  design 
problem,  which  typically  tends  to  be  the  most  complex 
and  ill-structured  kind  of  problem  that  can  be  solved. 
Given  an  initial  statement  of  need,  an  infinite  number 
of  potential  solutions  exists.  Can  learners  self-direct 
their  ability  to  solve  this  kind  of  problem  or  is  some 
form  of  studio  course  required  to  accommodate  its 
complexity?  What  is  the  range  of  complexity  and 
structuredness  that  can  be  effectively  learned  using 
PBL?  When  Jacobs  et  al.  (2003)  surveyed  medical 
students  with  a  questionnaire  that  was  designed  based 
on  Jonassen’s  continuum  of  structuredness  and  com¬ 
plexity  of  problems,  they  found  that  students  weighted 
problem  structuredness  more  heavily  than  problem 
complexity,  which  indicated  that  students  preferred 
some  degree  of  structuredness  to  identify  a  solution 
more  easily.  Taking  students’  perceptions  into  account 
in  addition  to  the  nature  of  the  subject  matter,  then, 
how  well-structured  or  ill- structured  can  and  should 
PBL  problems  be?  This  will  require  comparing  suc¬ 
cesses  and  failures  across  domains. 

Assuming  that  PBL  is  effective  for  a  range  of  prob¬ 
lems,  a  related  question  is  whether  the  established  PBL 
methodology  is  equally  appropriate  for  all  kinds  of 
problems,  or  should  the  method  be  adapted  to  accom¬ 
modate  different  kinds  of  problems?  Jonassen  (2004) 
has  prescribed  different  models  for  designing  learning 
environments  for  story  problems,  troubleshooting 
problems,  and  case-analysis  problems.  Models  for 
additional  kinds  of  problems  (e.g.,  design,  decision 
making)  are  under  development,  but  we  do  not  know 
how  unique  each  model  for  each  kind  of  problem  will 
be.  A  number  of  instructional  supports,  such  as  case 
libraries,  question  ontologies,  simulations,  argumenta¬ 
tion  systems,  and  problem  representation  tools,  may 
be  effective  across  several  kinds  of  problems.  We  sim¬ 
ply  do  not  know. 

Internal  Factors  and  PBL 

Problem-based  learning  was  originally  developed  for 
training  medical  students.  In  those  contexts,  educators 
assume  that  students  are  cognitively  ready  for  solving 
ill- structured  problems  and  engaging  in  self-directed 
learning.  As  more  PBL  efforts  are  being  implemented 
in  K-12  schools,  because  of  human  development 
issues,  younger  students  may  not  be  ready  to  solve 
complex  and  ill- structured  problems  and  self-direct 
their  own  learning.  The  question  of  learner  character¬ 
istics  (e.g.,  developmental  level,  epistemological 
beliefs,  cognitive  controls,  maturity,  reading  ability) 
related  to  PBL  has  not  been  significantly  addressed. 
Moreover,  developing  problem  solving  and  self- 
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directed  learning  skills  is  both  a  goal  of  the  course  of 
learning  and  at  the  same  time  a  required  ability  to 
succeed  in  PBL  classes.  So,  frustration  or  detrimental 
effects  may  be  inevitable  if  the  learners  (younger  stu¬ 
dents  or  even  adult  students)  possess  few  problem-solv¬ 
ing  and  self-directed  learning  skills  when  they  begin  a 
PBL  course.  How  can  we  reconcile  this  circulative  pre¬ 
requisite  and  goal  requirement  in  the  PBL  process? 

Designing  Distributed  PBL 

With  the  emergence  of  online  learning  initiatives, 
researchers  are  working  to  implement  PBL  in  online 
environments  (Tan  and  Hung,  2007).  This  trend  raises 
numerous  implementation  issues.  How  faithfully  can 
PBL  methodologies  be  applied  online?  An  important 
element  in  PBL  group  processing  and  collaborative 
learning  is  building  a  sense  of  learning  community. 
Barrows  (2002)  and  Orrill  (2002)  showed  that  collab¬ 
oration  suffered  in  online  PBL  environments.  Clearly, 
there  is  a  distinct  difference  between  conventional 
face-to-face  and  online  PBL  in  the  degree  of  social 
presence,  which  is  defined  by  two  factors:  intimacy 
and  immediacy  (Wiener  and  Mehrabian,  1968).  Given 
current  technology,  a  low  degree  of  intimacy  and 
immediacy  is  inherent  in  online  environments;  thus, 
how  do  learning  groups  collaborate  effectively  to  nego¬ 
tiate  meaning?  How  can  tutors  effectively  nurture  and 
guide  learning  online?  How  can  we  support  self- 
directed  learning  online?  What  compromises,  if  any, 
are  required  to  engage  learners  in  PBL  online? 
Although  the  e-leaming  movement  is  not  conceptually 
driven  and  the  technology  has  not  been  as  sophisticated 
as  promised,  it  appears  to  be  inevitable  enough  that 
these  become  important  questions. 

Multimedia  can  have  a  strong  impact  on  the  effec¬ 
tiveness  of  PBL  when  studying  subjects  for  whom  the 
ability  to  detect  signs,  symptoms,  or  behaviors  through 
visual,  audio,  or  tactile  senses  is  crucial  for  solving 
problems.  Appropriate  modality  of  presentation  of  the 
problem  could  play  a  significant  role  in  enhancing 
students’  problem-solving  skills  in  their  fields.  When 
training  students,  such  abilities  are  one  of  the  focuses 
of  PBL  curricula.  Text-based  problems  would  either 
give  away  the  cues  or  be  unable  to  afford  the  learning 
objectives  of  developing  self-directed  problem-solving 
skills;  however,  based  on  research  results  to  date,  the 
use  of  multimedia  has  seemed  to  fail  to  produce  such 
effects.  Thus,  we  might  ask:  Is  it  that  the  tutors  did 
not  take  advantage  of  the  technology  and  practice  the 
same  tutoring  techniques  as  they  did  with  text-based 
problems?  What  kind  of  facilitation  should  be  given 
to  guide  students  to  the  critical  contextual  information 
presented  in  the  problem? 


These  issues  that  have  emerged  from  experiences 
in  implementing  PBL  in  the  past,  as  well  as  in  response 
to  new  technology  developments  in  the  present,  chal¬ 
lenge  PBL  researchers  and  practitioners  yet  provide 
opportunities  for  new  insights  to  be  discovered  in  the 
future.  Only  continuing  research  will  provide  intellec¬ 
tual  and  scientific  support  to  inform  and  improve  the 
practice  of  PBL  as  well  as  education  in  general. 
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ABSTRACT 

Often,  training  fails  to  improve  organizational  out¬ 
comes.  Increasingly,  practitioners  are  assuming  a  per¬ 
formance  improvement  perspective  that  takes  a  holistic 
and  systemic  perspective  in  analyzing  causes  of  and 
solutions  for  gaps  in  organizational  results.  We  review 
the  models  and  evidence  for  three  perspectives  on  per¬ 
formance  improvement:  behavioral  engineering,  cog¬ 
nitive-motivational,  and  technological.  The  behavioral 
engineering  models  prevalent  among  U.S.  practitioner- 
oriented  professional  societies  apply  a  pragmatic  sys¬ 
temic  approach  that  evaluates  performance  factors  at 
the  organizational,  process,  and  individual  worker  or 
team  levels.  The  cognitive-motivational  approaches 
prevalent  among  industrial  and  organizational  psychol¬ 
ogists  include  guidelines  for  goal  setting,  feedback,  and 
incentives.  The  cognitive  models  are  based  on  empir¬ 
ical  evidence  and  incorporate  motivational  and  cogni¬ 
tive  moderators  of  their  interventions.  Recent  advances 
in  computer  technology  offer  opportunities  to  deliver 
performance  support  in  new  ways.  In  our  discussion 
of  technological  approaches,  we  focus  primarily  on  the 
evolution  of  and  evidence  for  electronic  performance 
support  systems  (EPSSs)  in  the  workplace. 


KEYWORDS 

Electronic  performance  support  system  (EPSS): 
Enabler  of  work  tasks  that  are  delivered  by  elec¬ 
tronic  technology  provided  to  individuals  or  teams 
at  the  time  of  need  on  the  job.  Typical  support 
includes  procedural  guidance  or  references  to  fac¬ 
tual  information  needed  to  complete  tasks. 

Feedback:  Information  on  goal  attainment  designed  to 
help  workers,  teams,  or  functional  units  monitor 
and  evaluate  their  progress  in  achievement  of 
desired  accomplishments.  Feedback  may  be  quan¬ 
titative  or  qualitative;  it  may  or  may  not  include 
explanations  to  guide  performance;  it  may  be 
directed  to  individual,  team,  unit,  or  organizational 
levels;  and  it  may  be  provided  through  personal 
communications  or  through  impersonal  channels 
such  as  charts  and  graphs  posted  in  work  areas  or 
by  computer. 


Goal-setting  theory:  Guidelines  for  optimizing  worker 
or  team  performance  by  setting  specific  and  diffi¬ 
cult  goals;  goal-setting  theory  considers  the  effects 
of  self-efficacy,  goal  commitment,  feedback,  and 
incentives  on  goal  effectiveness. 

Incentives:  Tangible  and  social  rewards  intended  to 
optimize  performance  of  individuals  or  teams;  may 
include  money,  feedback,  and  social  recognition. 
Performance  improvement:  An  approach  to  optimizing 
organizational  outcomes  that  uses  a  systemic  com¬ 
prehensive  methodology  to  define  and  resolve  gaps 
at  the  organizational,  process,  and  individual 
worker  levels. 

INTRODUCTION 

Imagine  the  following  situations:  (1)  A  large  utility  is 
the  target  of  so  many  customer  complaints  of  rude  tele¬ 
phone  service  that  the  Public  Utilities  Commission 
directs  the  utility  to  remedy  the  problem  or  face  fines. 
(2)  A  team  of  high-performing  data  entry  clerks  moves 
to  a  new  facility,  bringing  along  all  of  their  existing 
furniture  and  computers;  after  a  couple  of  weeks,  man¬ 
agement  is  distressed  to  see  that  online  error  rates  have 
jumped.  (3)  A  piece  of  equipment  inexplicably  shuts 
off,  and  the  breakdown  halts  production  in  the  factory 
and  costs  the  company  millions  of  dollars  in  lost  pro¬ 
ductivity;  the  technician  knows  that  this  problem  has 
happened  before  during  another  shift  but  does  not  know 
the  procedure  to  repair  the  machine  because  it  happens 
so  infrequently.  (4)  Washington,  D.C.,  gets  federal  fund¬ 
ing  to  hire  an  extra  900  police  officers  to  fight  rising 
crime,  but  3  years  later  10  police  officers  hired  during 
that  time  are  arrested  for  abetting  drug  dealers.  All  of 
these  real-life  situations  resulted  in  bottom-line  negative 
consequences  for  the  organizations  involved,  and  in  all 
cases  the  responsible  managers  requested  a  training  pro¬ 
gram  to  solve  the  problem.  Even  after  a  well-designed 
training  program  was  implemented,  however,  the  prob¬ 
lems  remained.  Why?  The  reasons  for  the  performance 
gaps  had  nothing  to  do  with  lack  of  worker  knowledge 
and  skills.  Frequent  observations  that  well-designed 
training  programs  do  not  always  translate  into  improved 
organizational  outcomes  have  given  rise  to  the  princi¬ 
ples  and  practice  of  performance  improvement. 
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TABLE  39.1 

Definitions  of  Human  Performance  Technology 


Term 


Definition 


Source 


Human  performance 
enhancement 
Human  performance 
technology 
Human  performance 
technology 

Human  performance 
technology 

Performance 

improvement 


The  field  focused  on  systematically  and  holistically  improving  present  and  future 
work  results  achieved  by  people  in  organizational  settings 

The  systematic  and  systemic  identification  and  removal  of  barriers  to  individual 
and  organizational  performance 

The  process  of  selection,  analysis,  design,  development,  implementation,  and 
evaluation  of  programs  to  most  cost-effectively  influence  human  behavior  and 
accomplishment 

The  study  and  ethical  practice  of  improving  productivity  in  organizations  by 
designing  and  developing  effective  interventions  that  are  results  oriented, 
comprehensive,  and  systemic 

The  process  of  identifying  and  analyzing  important  organizational  and  individual 
performance  gaps,  planning  for  future  performance  improvement,  designing  and 
developing  cost-effective  and  ethically  justifiable  interventions  to  close 
performance  gaps,  implementing  the  interventions,  and  evaluating  the  financial 
and  non-financial  results 


Roth  well  (1996) 

International  Society  of  Performance 
Improvement  (www.ispi.org) 
Harless,  cited  in  Geis  (1986) 


Pershing  (2006) 


American  Society  for  Training  and 
Development  (http :  //w w w.astd  .org) 


Unlike  many  other  chapters  in  this  Handbook  that 
discuss  instructional  interventions  and  issues,  our 
chapter  focuses  on  factors  other  than  knowledge  or 
skills  that  influence  workforce  performance  in  organi¬ 
zational  settings. 

WHAT  IS  PERFORMANCE 
IMPROVEMENT? 

In  the  past  20  years,  scenarios  like  those  summarized 
above  have  led  to  a  professional  practice  called  human 
performance  technology.  Table  39.1  lists  some  of  the 
recent  definitions  for  human  performance  technology. 
These  definitions  share  the  following  two  features: 

•  A  focus  on  desired  organizational  accom- 
plishments — Performance  improvement  ini¬ 
tiatives  are  pragmatic:  to  improve  outcomes 
that  are  linked  to  bottom-line  goals  of  the 
organization.  In  the  commercial  sector,  out¬ 
comes  are  typically  related  to  metrics  for 
sales,  product  quality,  customer  satisfaction, 
and  work  efficiency.  In  the  government  sec¬ 
tor,  outcomes  are  typically  related  to  indica¬ 
tors  of  mission  accomplishment,  product  or 
service  quality,  and  efficiency. 

•  Comprehensive  and  systemic  perspec¬ 
tives—  Human  performance  analysis  and 
interventions  address  the  interrelated  parts 
of  an  organization,  including  the  individual 
worker,  teams,  departments,  business  pro¬ 
cesses,  and  organizational  levels. 


In  2006,  in  the  United  States  about  $56  billion  was 
invested  in  workforce  learning  ( Training  Magazine, 
2006).  The  actual  investment  was  much  greater  because 
this  figure  does  not  take  into  account  the  most  expensive 
element  of  any  training  program— the  salary  time  of  the 
training  participants  as  well  as  the  lost  opportunity 
costs.  While  sales  professionals  are  in  training,  they  are 
earning  a  salary  and  they  are  not  selling.  Because  train¬ 
ing  is  an  expensive  intervention,  a  performance 
improvement  perspective  first  defines  the  factors  needed 
to  attain  desired  outcomes  and  only  recommends  train¬ 
ing  when  there  is  a  gap  in  worker  knowledge  and  skills. 
Managers  often  assume  that  training  is  the  appropriate 
(and  only)  route  to  achieve  performance  goals.  The 
challenge  of  the  performance  improvement  professional 
is  to  partner  with  line  clients  to  jointly  assess  the  per¬ 
formance  environment,  define  barriers  to  and  enablers 
of  those  behaviors  that  lead  to  desired  organizational 
accomplishments,  and  recommend  interventions  (solu¬ 
tions)  that  reduce  or  eliminate  those  barriers. 

How  Is  Performance  Measured? 

In  this  chapter,  we  examine  three  perspectives  on  per¬ 
formance  improvement:  behavioral,  cognitive-motiva¬ 
tional,  and  technological.  All  three  perspectives  have 
a  similar  goal— namely,  the  improvement  of  desired 
organizational  end  results.  Unlike  instructional-depen- 
dent  variables,  which  usually  rely  on  learning  mea¬ 
sures  in  the  form  of  tests,  the  measures  of  performance 
improvement  vary  widely.  In  the  commercial  sector, 
bottom-line  metrics  are  linked  to  profitability  (e.g., 
sales,  expenses,  errors  or  rework,  profit).  In  many 
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cases,  measures  that  are  already  in  place  in  organiza¬ 
tions,  such  as  customer  satisfaction,  error  rates,  pro¬ 
ductivity  measures,  employee  turnover,  and  sales  vol¬ 
ume,  are  used  as  dependent  measures;  for  example,  in 
a  study  on  the  effects  of  different  incentives  on  per¬ 
formance  in  fast-food  stores,  the  dependent  measures 
included  store  profit,  drive-through  times,  and 
employee  turnover  (Peterson  and  Luthans,  2006).  In 
other  organizations,  such  as  nonprofits  or  government, 
the  measures  link  to  the  organizational  mission  state¬ 
ment  or  goals.  When  organizations  do  not  have  clearly 
defined  measurable  goals,  the  first  performance 
improvement  intervention  must  engage  responsible 
management  to  establish  relevant  goals  and  metrics. 

THREE  APPROACHES  TO 
PERFORMANCE  IMPROVEMENT 

There  is  nothing  new  about  a  focus  on  workplace  per¬ 
formance.  At  the  end  of  the  19th  century,  the  Industrial 
Revolution  spawned  some  of  the  first  attempts  to  make 
workplace  performance  more  efficient  and  effective. 
Frederick  Taylor  (1911)  promoted  scientific  manage¬ 
ment  based  on  time  and  motion  studies  to  maximize 
work  efficiency.  His  recommendation  that  rewards  be 
linked  to  the  level  of  performance  is  in  line  with  con¬ 
temporary  evidence  on  the  effect  of  compensation  on 
performance.  In  the  mid-1900s,  Maslow’s  theory  of 
needs  and  Vroom’s  expectancy  theory  both  focused  on 
motivational  explanations  of  performance.  At  around 
the  same  time,  behaviorism  based  on  Thorndike’s  law 
of  effect  and  extended  by  B.F.  Skinner  gave  rise  to  a 
human  engineering  approach  to  performance.  From 
these  paradigms,  two  branches  of  performance  improve¬ 
ment  have  evolved:  one  reflecting  behaviorist  roots  tem¬ 
pered  by  systems  theory  and  a  second  evolving  from 
industrial/organizational  (I/O)  research  and  theory  on 
motivation  in  the  workplace.  We  refer  to  these  two 
branches  as  behavioral  engineering  and  cognitive— moti¬ 
vational.  Recently,  the  evolution  of  computer-based 
technologies  has  prompted  a  third  approach  that  we  call 
technological.  Our  discussion  of  technological  models 
focuses  primarily  on  the  use  of  computer  technology  to 
offer  electronic  performance  support.  Electronic  perfor¬ 
mance  support  encompasses  an  eclectic  array  of  digi¬ 
tally  delivered  interventions  that  may  embody  either 
behavioral  or  cognitive-motivational  assumptions. 

The  Evolution  of  the  Behavioral 
Engineering  Models 

Around  the  mid-20th  century,  behaviorists  applied  the 
concept  of  contingencies  to  training  through  pro¬ 


grammed  instruction.  A  number  of  these  behavioral 
scientists  in  the  United  States  founded  the  National 
Society  for  Programmed  Instruction  in  1962.  In  the 
process  of  implementing  and  evaluating  programmed 
instruction  and  instructional  design  models,  several  of 
these  professionals  discovered  that  even  well-designed 
training  did  not  result  in  the  accomplishment  of  orga¬ 
nizational  goals.  Thomas  Gilbert,  for  example,  stated: 
“I  found  that  businesses  had  all  kinds  of  performance 
problems  and  most  had  nothing  to  do  with  training. 
That  really  struck  me.  I  realized  that  the  world  of  work 
was  really  screwed  up  and  that  before  I  could  make 
the  world  safe  for  instructional  design  I  had  to  get  all 
the  other  problems  cleaned  up.  That’s  where  the  first 
ideas  for  performance  technology  came  from”  (Dean, 
1994,  p.  37). 

The  frequent  failure  of  training  to  produce  organi¬ 
zational  results  led  to  a  focus  on  factors  in  addition  to 
knowledge  and  skills  that  influence  work  place  perfor¬ 
mance.  Gilbert’s  book.  Human  Competency:  Engi¬ 
neering  Worthy  Performance,  published  in  1978,  and 
Joe  Harless’  An  Ounce  of  Analysis  (Is  Worth  a  Pound 
of  Objectives),  published  in  1970,  represent  two  mile¬ 
stones  in  the  behavioral  engineering  approach  to  per¬ 
formance  improvement.  Over  the  past  40  years,  as  the 
behavioral  engineering  approach  evolved,  the  National 
Society  for  Programmed  Instruction  refocused  its  mis¬ 
sion  and  scope  and  today  is  called  the  International 
Society  for  Performance  Improvement  (www.ispi.org). 

The  Evolution  of  the 
Cognitive-Motivational  Models 

At  the  same  time,  the  cognitive-motivational  branch 
of  performance  improvement  scientists  focused  on 
managing  work  behaviors  through  motivational 
means,  most  notably  through  goal  theory, recently  inte¬ 
grated  with  social  cognitive  and  self-efficacy  theory. 
Based  on  35  years  of  empirical  research,  Locke  and 
Latham  (2002)  summarized  their  goal-setting  model, 
which  specifies  the  mechanisms  and  processes  that  link 
goal  setting  to  performance  improvement. 

The  Evolution  of  the  Technological  Models 

In  her  milestone  book.  Electronic  Performance  Sup¬ 
port  Systems,  Gery  (1991,  p.  34)  stated  that: 

The  goal  of  an  EPSS  is  to  provide  whatever  is  necessary 
to  generate  performance  and  learning  at  the  moment  of 
need.  We  now  have  the  means  to  model,  represent,  struc¬ 
ture,  and  implement  that  support  electronically — and  to 
make  it  universally  and  consistently  available  on  demand 
any  time,  any  place  and  regardless  of  situation. 
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Figure  39.1  A  basic  systems  model  of  performance. 


Performance  support  systems  have  evolved  alongside 
advances  in  technology.  Knowledge  management  sys¬ 
tems  allow  employees  to  tap  into  the  knowledge  of 
experts.  Learning  content  management  systems  now 
make  it  possible  to  link  to  learning  objects  originally 
developed  for  training  as  on-the-job  support.  Search 
engines  and  visual  mapping  systems  make  it  increas¬ 
ingly  easy  for  employees  to  locate  support  content. 
Mobile  devices  will  allow  employees  to  access  content 
anytime,  anywhere  from  their  cell  phones  and  iPods. 
Electronic  performance  support  system  (EPSS) 
approaches  to  performance  improvement  focus  on  the 
use  of  technology  in  various  ways  to  help  workers, 
teams,  and  functional  units  achieve  organizational  out¬ 
comes.  EPSS  interventions  are  varied  and  may  reflect 
behavioral  or  cognitive-motivational  perspectives. 

In  the  next  sections,  we  summarize  the  behavioral, 
cognitive,  and  technological  approaches  to  perfor¬ 
mance  improvement.  Although  we  could  have  summa¬ 
rized  many  behaviorist  performance  models,  we  briefly 
describe  three  here  that  have  played  a  significant  role 
in  the  evolution  of  the  International  Society  for  Perfor¬ 
mance  Improvement.  These  are  the  models  of  Gilbert, 
Rummler  and  Brache,  and  Pershing.  We  summarize  the 
Gilbert  model  to  describe  one  of  the  acknowledged 
foundational  approaches  to  human  performance  tech¬ 
nology.  The  Rummler-Brache  model  is  included  as  one 
of  the  most  influential  recent  performance  improve¬ 
ment  models  that  builds  upon  and  extends  Gilbert’s 
pioneering  work.  Finally,  we  summarize  the  Pershing 
model  as  a  very  recent  synthesis  of  behavioral 
approaches  summarized  in  the  2006  Handbook  of 
Human  Performance  Technology.  Regarding  cognitive- 
motivational  approaches,  we  summarize  goal  setting 
and  the  high  performance  cycle  from  Locke  and 
Latham  (2002)  as  well  as  other  recent  research  on 
goals,  feedback,  and  incentives  reflecting  a  cognitive 
perspective.  To  address  the  role  of  technology  in  per¬ 


formance  support,  we  will  also  look  at  research  and 
practice  in  the  use  of  electronic  performance  support 
systems  (EPSSs)  to  improve  workplace  performance. 

BEHAVIORAL  ENGINEERING  MODELS 
OF  PERFORMANCE  IMPROVEMENT 

Thomas  Gilbert,  who  worked  with  B.F.  Skinner,  is 
considered  one  of  the  founding  fathers  of  the  behav¬ 
ioral  performance  improvement  approach.  Much  of 
Gilbert’s  performance  model,  originally  published  in 
1978,  has  been  incorporated  into  more  recent  human 
performance  models. 

Gilbert’s  Human  Competence  Model 

As  shown  in  Figure  39.1,  Gilbert  articulated  a  behav¬ 
ioral/systems  model  in  which  individual  workers  act  on 
inputs  (stimuli)  in  ways  that  produce  results  (responses) 
which  in  turn  have  consequences.  Based  on  feedback, 
workers  modify  their  performance  to  increase  positive 
consequences.  This  systems  perspective  is  a  major  fea¬ 
ture  of  all  behavioral  engineering  approaches  to  per¬ 
formance  improvement.  Gilbert  defined  specific  per¬ 
formance  factors  in  terms  of  information  (inputs), 
instrumentation  (inputs),  and  motivation  (conse¬ 
quences)  in  a  two-tier  model  with  tier  1  focusing  on 
the  environmental  factors  and  tier  2  focusing  on  the 
individual  performer  (Figure  39.2).  Summarizing  envi¬ 
ronmental  performance  factors,  cell  1  includes  the 
informational  resources  required  to  perform  the  job, 
including  work  standards,  work  documentation,  and 
performance  feedback.  Cell  2  specifies  environmental 
resources,  including  money,  time,  equipment,  ergo¬ 
nomics,  leadership  support,  and  work  processes.  Cell 
3  focuses  on  incentives  including  compensation,  rec¬ 
ognition,  and  developmental  opportunities.  The  lower 
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•  Work  aids 
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•  Individual 
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Figure  39.2  Gilbert's  six-cell  performance  factors.  (Adapted  from  Gilbert,  T.,  Human  Competence:  Engineering  Worthy  Performance, 
McGraw-Hill,  New  York,  1978.) 


tier  includes  similar  categories  as  they  apply  to  the 
individual  worker:  Cell  4  includes  knowledge  and 
skills  based  on  adequate  training.  Capability  to  do 
the  job  is  the  theme  of  cell  5,  which  includes  the 
physical,  mental,  and  emotional  job  prerequisites. 
Cell  6  addresses  motives  of  the  individual  worker  to 
invest  effort  and  persist  in  response  to  the  available 
incentives . 

Many  of  the  basic  elements  of  human  performance 
improvement  are  reflected  in  the  Gilbert  model,  which 
has  a  focus  on  accomplishment— outcomes  rather  than 
events.  He  distinguished  between  environmental  influ¬ 
ences,  such  as  tools  and  standards,  and  factors  related 
to  individual  performers,  such  as  working  aids  and 
selection.  The  Gilbert  model  is  a  behavioral  systems 
approach  that  depicts  workers  as  respondents  to  inputs 
adjusting  outputs  based  on  feedback  and  consequences 
of  their  actions.  Aspects  of  the  Gilbert  model  have  been 
carried  forward  in  all  of  the  behavioral  models  that 
followed  it. 


Rummler-Brache  Three-Level  Model 

The  Rummler-Brache  (1995)  model  incorporates  Gil¬ 
bert’s  behavioral  engineering  concepts,  adding  a  sys¬ 
tems  view  of  organizations  and  the  opportunity  for 
improvement  through  process  alignment.  In  their  1995 
book  Improving  Performance:  How  to  Manage  the 
White  Space  on  the  Organization  Chart ,  they  stated 
that:  “In  our  experience,  the  greatest  opportunities  for 
performance  improvement  often  lie  in  the  functional 
interfaces  — those  points  at  which  the  baton  is  being 
passed  from  one  department  to  another”  (p.  9).  As 
shown  in  Figure  39.3,  the  Rummler-Brache  model 
focuses  on  planning  and  management  at  three  levels 
of  performance— the  organizational  level,  the  process 
level,  and  the  individual  job  performer  level.  The  job 
level  performance  factors  are  similar  to  those  specified 
in  the  Gilbert  model:  however,  the  Rummler-Brache 
model  emphasizes  a  holistic  view  of  organizations  and 
the  need  for  alignment  among  the  three  main  interfaces 


The  Organization 


Figure  39.3  Rummler-Brache  three  performance  levels  model.  (Adapted  from  Rummler,  G.A.  and  Brache,  A.R,  Improving  Perfor¬ 
mance:  How  to  Manage  the  White  Space  on  the  Organization  Chart,  2nd  ed.,  Jossey-Bass,  San  Francisco,  CA,  1995.) 
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Figure  39.4  Pershing  performance  improvement  process  model.  (Adapted  from  Pershing,  J.A.,  in  Handbook  of  Human  Performance 
Technology,  3rd  ed.,  Pershing,  J.A.,  Ed.,  Pfeiffer,  San  Francisco,  CA,  2006,  pp.  5-34.) 


of  organization,  process,  and  individual  performer.  A 
Rummler-Brache  analysis  evaluates  performance  fac¬ 
tors  at  all  three  levels:  the  strategies,  goals,  measures, 
structures,  and  resources  at  the  organizational  level; 
the  flow  of  inputs  and  outputs  through  functional  units 
at  the  process  level;  and  the  various  individual  worker 
factors  described  in  the  Gilbert  model.  One  major 
underlying  premise  is  that  the  three  levels  are  interde¬ 
pendent  and  that  planning  and  management  must  occur 
at  all  three  levels.  To  address  individual  worker  factors 
outside  of  the  larger  context  will  often  be  counterpro¬ 
ductive.  The  Rummler-Brache  model  is  widely  used 
today  as  a  guide  for  a  comprehensive  behavioral  engi¬ 
neering  approach  to  performance  improvement. 

Pershing’s  Performance 
Improvement  Process  Model 

We  include  the  Pershing  model  (2006),  summarized  in 
Figure  39.4,  because  it  focuses  on  the  performance 
improvement  process  and  thus  adds  a  different  quite 
recent  perspective  on  human  performance  improve¬ 
ment.  Typical  initiators  of  a  performance  improvement 
project  might  be  performance  problems  such  as  the 
ones  we  summarized  in  our  opening  paragraph,  quality 
improvement  initiatives  or  business  opportunities  such 
as  acquisitions  and  mergers.  The  Pershing  perfor¬ 
mance  assessment  process  recommends  initiating  a 
performance  improvement  effort  by  identifying  per¬ 


ceptions  or  perceived  needs.  This  process  includes 
identifying  the  sponsors,  champions  and  stakeholders 
of  a  performance  improvement  initiative,  determining 
how  the  need  originated  and  the  importance  of  the 
initiative.  An  early  step  at  this  stage  is  to  assess  the 
alignment  of  any  performance  initiative  with  the  oper¬ 
ational  objectives  of  the  organization.  If  the  problem 
is  resolved  or  the  goal  is  achieved,  will  the  outcomes 
align  with  the  values,  culture,  structure,  and  goals  of 
the  larger  organizational  environment?  As  Pershing 
(2006,  p.  18)  stated:  “Organizations  are  often  caught 
up  in  expending  resources  and  time  in  solving  prob¬ 
lems  or  seizing  opportunities  with  little  or  no  payback 
or  contribution  to  organizational  goals.” 

Once  an  alignment  is  realized,  the  performance 
assessment  examines  factors  in  organizational,  man¬ 
agement,  physical,  and  human  systems.  At  the  organi¬ 
zational  level  evaluating  the  structure,  communication 
lines,  divisions  of  labor,  and  methods  of  operation  and 
decision  making  may  offer  avenues  for  improving 
organizational  viability  and  effectiveness.  Manage¬ 
ment  system  analysis  assesses  the  quality  of  the  key 
management  functions  to  delegate,  develop  others, 
conduct  performance  appraisals,  and  set  priorities.  An 
assessment  of  physical  and  technical  systems  considers 
the  adequacy  of  the  technical  processes  as  well  as  the 
equipment  and  tools  that  workers  are  using  to  accom¬ 
plish  their  goals.  This  includes  ergonomic  issues,  func¬ 
tionality  of  technical  systems,  process  engineering,  and 
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job  aids  both  traditional  and  electronic.  Last,  a  review 
of  the  human  systems  focuses  on  selection  processes, 
knowledge,  and  skills,  along  with  rewards  and  incen¬ 
tives.  As  a  result  of  the  performance  scan,  gaps  are 
identified  related  to  one  or  more  of  these  systems  to 
be  resolved  by  an  appropriate  intervention. 


Interventions  in  Behavioral 
Engineering  Models 

Interventions  in  the  behavioral  engineering  approaches 
involve  either  a  removal  of  barriers  or  the  addition  of 
enablers  of  performance  within  the  organizational, 
management,  physical,  or  human  systems.  Typical 
interventions  can  include  process  reengineering,  man¬ 
agement  standards  and  training,  ergonomic  modifica¬ 
tions,  job  aids  (traditional  and  electronic),  job  stan¬ 
dards,  performance  feedback,  and  incentives,  as  well 
as  training,  to  name  a  few.  Recall  the  performance 
problems  we  mentioned  in  our  introduction.  In  all  of 
these  situations,  management  called  for  a  training  solu¬ 
tion;  for  example,  the  utility  management  called  for  a 
telephone  courtesy  course  to  reduce  customer  com¬ 
plaints  of  rude  service  ,  the  data  entry  clerks  were  given 
a  refresher  training  course  to  reduce  input  errors,  and 
so  on.  The  performance  problems  did  not  wane  after 
training,  though.  A  closer  look  at  the  organizational, 
management,  technical,  and  human  systems  found  that 
the  gaps  were  unrelated  to  knowledge  and  skills. 

Structured  observations  of  customer  call  simula¬ 
tions,  for  example,  showed  that  most  of  the  utility 
customer  service  representatives  were  capable  of  mak¬ 
ing  courteous  responses;  however,  their  job  standards, 
feedback,  and  incentives  were  set  at  120  calls  per  day. 
There  was  a  misalignment  of  job  standards,  feedback, 
and  incentives  with  desired  work  accomplishments 
(e.g.,  courteous  responses).  As  long  as  management 
demanded  and  rewarded  quantitative  outcomes  only, 
quality  would  suffer. 

With  regard  to  the  data  entry  clerks,  analysis  of 
error  patterns  showed  that  most  were  made  in  the  later 
afternoon  after  2:00  p.m.  Interviews  and  observations 
of  work  conditions  showed  that  sunlight  coming 
through  the  large  windows  in  the  new  facility  created 
a  glare  on  the  computer  screens  that  reduced  the  level 
of  visibility  on  the  screens  in  the  afternoon  hours.  An 
inexpensive  ergonomic  solution  was  in  order— good 
window  shades  (Addison  and  Haig,  2006). 

In  Washington,  D.C.,  the  $500  million  in  federal 
funds  available  to  hire  new  police  officers  had  to  be 
spent  within  a  2-year  period.  As  a  result,  there  was  no 
time  for  the  usual  psychological  screening  and  back¬ 
ground  checks  (Flaherty  and  Harrison,  1993).  This 


Figure  39.5  The  iceberg  model  of  performance  gaps.  (From 
Addison.  R.M.  and  Haig,  C.,  in  Handbook  of  Human  Perfor¬ 
mance  Technology,  3rd  ed.,  Pershing,  J.A.,  Ed.,  Pfeiffer,  San 
Francisco,  CA,  2006,  pp.  35-54.  With  permission.) 


performance  problem  had  its  roots  in  a  failure  to  apply 
an  effective  selection  system. 

As  shown  in  Figure  39.5,  only  a  small  proportion 
of  performance  gaps  stem  from  lack  of  knowledge  and 
skills  and  warrant  a  training  solution.  The  vast  majority 
require  solutions  that  address  the  organizational,  man¬ 
agement,  physical,  or  human  systems.  Rothwell  and 
Kazanas  (2004)  identified  five  of  the  most  frequent 
alternatives  to  instructional  solutions:  feedback,  job 
performance  aids,  reward  systems,  employee  selection 
practices,  and  organizational  redesign. 

Features  of  the  Behavioral 
Engineering  Models 

The  strength  of  the  behavioral  perspective  is  its  holistic 
and  pragmatic  approach  that,  when  effectively  applied, 
improves  organizational  results  by  aligning  organiza¬ 
tional  efforts  vertically,  horizontally,  and  at  the  individ¬ 
ual  worker  level.  The  knowledge  base  of  the  behavioral 
engineering  community  rests  primarily  on  extensive 
case-study  work  and  the  articulation  of  models  such  as 
those  summarized  above.  The  behavioral  engineering 
perspective  lacks  experimental  evidence;  for  example, 
a  review  of  articles  published  in  the  main  ISPI  research 
journal.  Performance  Improvement  Quarterly,  found 
that  36%  of  the  articles  in  the  period  from  1997  to  2000 
were  based  on  data  (Klein,  2002),  a  number  that 
increased  to  54%  for  the  time  period  2001  to  2005 
(Marker  et  al.,  2006).  Of  the  reports  based  on  data  in 
the  Marker  et  al.  analysis,  only  5%  reflected  experi¬ 
mental  or  quasi-experimental  research  studies. 
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As  reflected  in  the  models  reviewed  here,  a  diverse 
set  of  practitioner  perspectives  and  skills  is  often  needed 
to  define  and  to  implement  a  comprehensive  perfor¬ 
mance  solution.  A  second  potential  limitation  of  the 
behavioral  models  is  the  one-size-fits-all  perspective.  A 
basic  assumption  of  the  behavioral  engineering  orienta¬ 
tion  is  that  performance  can  be  engineered  by  modifi¬ 
cations  to  the  environment  with  little  consideration  for 
how  individual  worker  or  task  differences  might  interact 
with  the  modifications.  This  assumption  may  lead  to 
recommendations  that  do  not  factor  in  critical  differ¬ 
ences  in  the  nature  of  the  work  tasks  or  differences 
among  individual  worker  motivations  and  skills. 

COGNITIVE-MOTIVATIONAL 
APPROACHES  TO  PERFORMANCE 
IMPROVEMENT 

In  the  previous  section,  we  reviewed  the  main  tenets 
and  assumptions  of  the  behavioral  engineering  per¬ 
spective  on  performance  improvement.  In  contrast, 
industrial  and  organizational  psychologists  have  relied 
to  a  large  degree  on  experimental  evidence  as  well  as 
a  cognitive  perspective  on  workplace  performance.  In 
this  section,  we  summarize  the  main  findings  from 
goal-setting  theory  as  well  as  the  effects  of  feedback 
and  incentives  on  performance  outcomes. 

Similar  to  the  behavioral  engineering  models  sum¬ 
marized  in  the  previous  paragraphs,  the  cognitive-moti¬ 
vational  performance  models  are  designed  to  predict, 
explain,  and  positively  influence  organizational  results. 
In  addition,  cognitive  approaches  address  some  of  the 
same  performance  factors  as  the  behavioral  models  — 
most  notably,  performance  standards  (goals),  feedback, 
and  incentives.  Unlike  the  behavioral  engineering  mod¬ 
els,  however,  cognitive-motivational  guidelines  are 
based  on  numerous  experiments— both  controlled  lab¬ 
oratory  experiments  as  well  as  field  studies.  In  addition. 


these  models  incorporate  interactions  between  the 
workplace  environment  and  internal  psychological  pro¬ 
cesses.  Theories  based  on  interactions  between  internal 
mental  processes  and  external  environmental  factors  are 
one  of  the  features  of  cognitive  models  that  distinguish 
them  from  behavioral  approaches;  for  example,  a 
behavioral  perspective  assumes  that  worker  feedback 
will  generally  exert  a  positive  effect  on  work  outputs. 
In  contrast,  cognitive  approaches  show  that  sometimes 
feedback  has  a  detrimental  effect  on  performance 
because  it  directs  attention  to  the  self  rather  than  the 
task  (Kluger  and  DeNisi,  1996).  For  success,  feedback 
must  be  designed  and  delivered  in  ways  that  direct 
attention  to  the  task— not  the  self.  When  addressing 
performance  factors  such  as  goals,  feedback,  and  incen¬ 
tives,  cognitive  models  ask  about  how  they  exert  their 
effects  and  how  they  should  be  implemented  adaptively 
based  on  their  mechanisms  of  action. 

Goal-Setting  Theory 

The  core  finding  of  goal  theory  is  that  there  is  a  direct 
and  positive  relationship  between  goal  difficulty  and 
subsequent  performance  with  effect  sizes  ranging  from 
.52  to  .82— in  the  moderate  to  high  range.  Locke  and 
Latham  (2002)  have  found  that  specific  difficult  goals 
consistently  lead  to  higher  performance  than  more  gen¬ 
eral  goals  such  as  “do  your  best.”  When  goal  levels 
are  held  constant,  higher  levels  of  performance  are 
realized  from  individuals  with  high  self-efficacy  who 
believe  that  with  effort  they  can  attain  the  goals. 

The  High  Performance  Cycle  Model 

Goal-setting  theory  has  studied  both  the  mechanisms  and 
modifiers  of  goals  summarized  in  the  high  performance 
cycle  model  shown  in  Figure  39.6.  Regarding  mecha¬ 
nisms,  goals  are  effective  because  they  direct  attention 
followed  by  effort  toward  goal-relevant  activities  and 
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Figure  39.6  The  high  performance  cycle  of  goal  setting  theory.  (From  Locke,  E.A.  and  Latham,  G.P.,  Am.  Psychol .,  57(9),  705-717, 
2002.  With  permission.) 
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away  from  activities  that  do  not  achieve  goals.  In  addi¬ 
tion,  goals  energize  performance,  prolong  effort  and 
stimulate  more  effective  approaches  to  job  tasks. 

Challenging  specific  goals  may  not  always  result 
in  better  performance,  however,  because  there  are 
some  proven  moderators  of  goal  setting  on  perfor¬ 
mance.  First,  workers  must  make  a  personal  commit¬ 
ment  to  achieving  the  goal.  To  ensure  goal  commit¬ 
ment,  goal  attainment  should  be  positioned  as 
important,  and  workers  should  be  assured  that  they  can 
attain  the  goal.  Although  there  has  been  a  popular 
belief  that  participative  setting  of  goals  results  in 
greater  goal  commitment,  research  has  not  born  this 
out;  rather,  an  assigned  goal  is  as  effective  as  one  set 
collaboratively  provided  the  purpose  or  rationale  for 
the  goal  is  given.  Meta-analysis  of  the  effects  of  par¬ 
ticipation  in  decision  making  on  performance  has 
yielded  a  low  effect  size  of  .11  (Locke  and  Latham, 
2002).  Apparently,  the  greatest  benefits  derived  from 
participative  goal  setting  are  cognitive  in  nature  as  a 
result  of  information  exchange  among  the  participants. 

Goal  commitment  can  be  enhanced  by  monetary 
incentives,  with  more  money  leading  to  greater  com¬ 
mitment;  however,  if  the  goal  is  very  difficult  and 
incentives  are  contingent  on  reaching  the  goal,  workers 
will  tend  to  abandon  the  goal  once  they  sense  they  will 
not  reach  it.  Instead,  goals  should  be  moderately  diffi¬ 
cult,  or  employers  should  pay  for  performance  levels 
rather  than  goal  attainment.  To  boost  self-efficacy  (e.g., 
to  promote  self-confidence  in  attaining  the  goal),  ade¬ 
quate  training  must  be  provided  to  increase  mastery, 
combined  with  role  modeling  or  persuasive  communi¬ 
cations  that  express  confidence  in  the  workers’  abilities. 

A  second  moderator  of  goal-setting  effects  is  feed¬ 
back.  For  goals  to  be  effective,  workers  benefit  from 
feedback  that  shows  their  progress  in  relationship  to 


the  goals.  Combining  goals  with  feedback  is  more 
effective  than  goal  setting  alone;  however,  a  recent 
meta-analysis  on  the  effects  of  feedback  on  perfor¬ 
mance  (Kluger  and  DeNisi,  1996),  summarized  in  the 
next  section,  offers  additional  insights  about  how  to  use 
feedback  most  effectively  to  maximize  performance. 

Task  complexity  is  a  third  moderator  of  goal-set- 
ting  effectiveness.  Most  of  the  research  has  been  con¬ 
ducted  on  tasks  of  relatively  low  complexity.  As  the 
complexity  of  the  task  increases,  goal  effects  will 
increasingly  depend  on  the  ability  of  the  worker  to 
discover  appropriate  task  strategies.  Because  some 
workers  will  be  more  successful  than  others,  the  effect 
of  goals  on  complex  tasks  is  smaller  than  on  simple 
tasks.  Meta-analysis  shows  an  effect  size  of  .48  for 
complex  tasks  compared  to  .67  for  more  simple  tasks . 
On  complex  tasks,  a  learning  goal  might  lead  to  better 
performance  than  a  performance  goal. 

In  summary,  Locke  and  Latham  (2002,  p.  714) 
concluded  that: 

...the  effects  of  goal  setting  are  very  reliable.  ...With 
goal-setting  theory,  specific  difficult  goals  have  been 
shown  to  increase  performance  on  well  over  100  different 
tasks  involving  more  than  40,000  participants  in  at  least 
eight  countries  working  in  laboratory,  simulation,  and 
field  settings. 

The  Effects  of  Feedback  on  Performance 

Providing  feedback  on  goal  attainment  results  in  better 
performance  than  goal  setting  alone.  Kluger  and 
DeNisi  (1996)  summarize  guidelines  and  a  theory  of 
feedback  based  on  a  meta-analysis  of  607  effect  sizes 
contained  in  131  papers  spanning  23,663  observations. 
A  histogram  of  these  effect  sizes  is  shown  in  Figure 
39.7.  Although  the  average  effect  size  was  moderate 


Figure  39.7  Distribution  of  607  effects  of  feedback  intervention  on  performance.  (From  Kluger,  A.N.  and  DeNisi,  A.,  Psychol.  Bull., 
119(2),  254-284,  1996.  With  permission.) 
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and  positive  (.4),  over  one  third  of  the  feedback  inter¬ 
ventions  either  had  no  effects  or  sometimes  even 
depressed  performance.  Kluger  and  DeNisi  (1996,  p. 
254)  emphasized  that  “a  considerable  body  of  evidence 
suggesting  that  feedback  intervention  effects  on  per¬ 
formance  are  quite  variable  has  been  historically  dis¬ 
regarded  by  most  researchers.  This  disregard  has  led 
to  a  widely  shared  assumption  that  feedback  interven¬ 
tions  consistently  improve  performance.” 

To  explain  the  variable  effects  of  feedback  on  per¬ 
formance,  Kluger  and  DeNisi  proposed  that  feedback 
primarily  refocuses  the  workers’  attention  to  one  of 
three  levels:  self,  task,  or  task  detail.  A  shift  of  attention 
to  the  task  level  or  the  task  detail  level  usually  will 
result  in  a  positive  effect;  however,  if  attention  is  refo¬ 
cused  to  the  self,  performance  suffers.  The  reason  is 
that,  for  many,  feedback  that  draws  attention  to  the  self 
is  threatening  and  diverts  limited  cognitive  resources 
from  the  productive  task  and  task  details  level. 

When  Klugar  and  DeNisi  evaluated  the  moderators 
of  feedback,  they  found  that  cues  that  direct  the  work¬ 
ers’  attention  to  self  such  as  normative  feedback  (how 
you  do  compare  to  others?),  feedback  that  is  either 
discouraging  or  includes  praise,  and  feedback  from  a 
person  compared  to  feedback  from  a  computer  all 
attenuate  the  positive  effects  of  feedback.  That  is 
because  all  of  these  cues  direct  mental  resources  to 
self  goals  rather  than  to  task  goals.  In  contrast,  feed¬ 
back  that  shows  progress  from  previous  attainments, 
emphasizes  correct  solutions,  and  comes  from  a  less 
personal  source  such  as  a  computer  augments  feedback 
effects  by  focusing  attention  on  the  task.  These  effects 
are  moderated  by  self-esteem.  Individuals  low  in  self¬ 
esteem  will  direct  negative  feedback  to  the  self  more 
than  individuals  high  in  self-esteem;  therefore,  nega¬ 


tive  feedback  will  have  a  more  negative  effect  on  the 
performance  of  those  with  low  self-esteem.  In  addi¬ 
tion,  as  summarized  by  Locke  and  Latham  (2002), 
Kluger  and  DeNisi  (1996)  also  reported  that  feedback 
improves  performance  more  in  the  presence  of  goals. 

The  Effects  of  Incentives  on  Performance 

The  role  of  consequences  (rewards)  is  a  key  element 
of  all  performance  improvement  paradigms,  but  what 
do  we  know  from  research  about  the  types  of  incen¬ 
tives  that  improve  performance?  Two  meta-analyses 
published  in  2003  offer  some  insights  on  the  effects 
of  incentives  on  performance.  Stajkovic  and  Luthans 
(2003)  reviewed  the  average  individual  and  combined 
effects  of  three  common  incentives— money,  feedback, 
and  social  recognition— on  work  performance.  They 
hypothesized  that  the  three  reinforcers  together  would 
have  a  synergistic  effect  that  was  greater  than  the  sum 
of  the  three  used  individually  as  a  result  of  different 
and  complimentary  mechanisms.  Money  has  high 
instrumental  value  and  will  likely  lead  to  extra  effort 
to  increase  performance;  however,  monetary  rewards 
do  not  provide  much  information  about  the  perfor¬ 
mance.  Workers  may  not  be  sure  what  must  be  accom¬ 
plished,  where  to  get  resources,  or  how  to  correct 
unproductive  behaviors;  therefore,  a  second  rein- 
forcer— feedback— derives  its  power  from  the  infor¬ 
mation  it  provides.  Finally,  social  recognition  has  pos¬ 
itive  effects  on  performance  as  a  predictor  of  future 
positive  consequences  such  as  promotions  and  raises. 

The  data  from  the  meta-analysis  are  summarized 
in  Figure  39.8,  which  shows  the  percentage  effects 
of  various  reinforcement  interventions  on  perfor¬ 
mance  both  individually  and  in  combination.  The 


C4  =  Money  +  Feedback  +  Social  Recognition  (SR) 

Cl  =  Money  +  Feedback 

C3  =  Feedback  +  Social  Recognition 

C2  =  Money  +  Social  Recognition 

Figure  39.8  Percentage  effects  of  reinforcement  interventions  on  employee  performance.  (From  Stajkovic,  A.D.  and  Luthans,  F., 
Pers.  Psychol.,  56(1),  155-194,  2003.  With  permission.) 
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three  incentives  combined  (C4)  clearly  have  a  greater 
positive  effect  than  any  single  incentive  or  the  math¬ 
ematical  sum  of  the  three  effects  measured  individ¬ 
ually.  Of  the  individual  incentives,  money  has  the 
largest  effect  followed  by  social  recognition.  The 
studies  that  evaluated  the  effects  of  all  three  reinforc¬ 
ers  showed  a  45%  impact  on  performance.  The  sum 
of  the  individual  effect  sizes  equaled  1.48;  however, 
the  actual  effect  size  from  studies  that  combined  all 
three  equaled  1 .88 .  This  difference  in  actual  and  com¬ 
bined  effect  sizes  suggests  a  synergistic  effect  of  all 
three  in  concert.  The  authors  reported  an  average 
effect  size  of  16%  improvement  in  performance. 
Money  improved  performance  23%,  social  recogni¬ 
tion  17%,  and  feedback  10%.  Stajkovic  and  Luthans 
(2003)  suggest  that  the  impact  of  feedback  will  be 
even  greater  in  more  complex  tasks  where  the  infor¬ 
mation  value  of  feedback  will  exert  greater  leverage 
than  with  less  complex  tasks. 

A  second  meta-analysis  on  incentives  by  Condly 
et  al.  (2003)  reported  that  all  incentive  programs  in  all 
work  settings  and  on  all  work  tasks  resulted  in  a  22% 
gain  in  performance,  a  figure  quite  close  to  the  16% 
reported  by  Stajkovic  and  Luthans  (2003).  Condly  et 
al.  (2003)  found  that  team-directed  incentives  had  a 
markedly  superior  effect  compared  to  individual- 
directed  incentives  and  that  monetary  incentives 
resulted  in  higher  performance  gains  than  tangible 
incentives  such  as  gifts  and  travel.  They  also  found 
that  long-term  incentive  programs  led  to  greater  per¬ 
formance  gains  than  short-term  programs. 

A  recent  quasi-experimental  control  group  study 
reported  by  Peterson  and  Luthans  (2006)  compared  the 
effects  of  financial  and  nonfinancial  incentives  applied 
to  fast-food  franchise  store  teams  on  three  outcome 
measures  of  performance  over  a  9-month  time  period. 
In  their  research,  21  fast-food  stores  were  randomly 
assigned  to  two  treatments  or  a  control  group.  One 
treatment  offered  a  monetary  incentive  to  the  entire 
team,  the  amount  of  which  was  contingent  on  the  num¬ 
ber  of  points  earned  by  structured  management  obser¬ 
vations  of  desired  behaviors.  The  nonfinancial  treat¬ 
ments  involved  feedback  provided  on  charts  and  social 
recognition  offered  to  the  entire  team  by  the  managers. 
Peterson  and  Luthans  (2006)  compared  the  perfor¬ 
mance  outcomes  of  store  profit,  drive-through  times, 
and  employee  turnover  at  3,  6,  and  9  months. 

With  regard  to  profits,  they  found  significant  main 
effects  for  both  interventions.  At  3  months,  both  the 
financial  and  feedback-recognition  groups  outper¬ 
formed  the  control  stores ,  with  the  financial  interven¬ 
tions  outperforming  the  nonfinancial.  By  6  and  9 
months,  however,  both  the  financial  and  nonfinancial 
interventions  had  a  similar  positive  impact  on  profits. 


With  regard  to  drive-through  times,  at  all  three  time 
periods  both  the  financial  and  nonfinancial  groups 
improved  performance  equally  compared  to  the  con¬ 
trol  stores.  Although  both  treatment  groups  experi¬ 
enced  less  turnover  than  the  control  groups,  the  finan¬ 
cial  incentive  had  significantly  greater  impact  on 
turnover  than  the  feedback-recognition  incentives. 
Specifically,  the  study  showed  that  average  profits 
rose  from  30%  from  pre-intervention  to  post-inter¬ 
vention  over  the  9  months  for  the  financial  condition 
and  36%  for  the  nonfinancials;  drive-through  times 
decreased  19%  for  the  financials  and  25%  for  the 
nonfinancials;  and  turnover  was  reduced  by  13%  for 
the  financials  and  by  10%  for  the  nonfinancials.  An 
interesting  outcome  of  this  study  is  the  differential 
effects  of  the  incentives  on  different  performance 
outcomes;  for  example,  turnover  responded  differ¬ 
ently  than  profitability  to  monetary  and  feedback- 
recognition  incentives.  Consequently,  we  recommend 
that  research  studies  use  more  than  one  performance 
metric  as  a  dependent  measure  and  evaluate  results 
over  a  period  of  time. 

In  summary,  the  recent  meta-analyses  and  experi¬ 
ments  on  incentives  suggest  that:  (1)  incentive  pro¬ 
grams  in  general  will  lead  to  about  a  20%  improvement 
in  performance  outcomes;  (2)  a  coordinated  program 
that  uses  a  combination  of  incentives  such  as  monetary 
rewards,  feedback,  and  social  recognition  contingent 
on  desired  behaviors  will  exert  the  largest  effects  on 
performance;  (3)  team-administered  incentives  may 
yield  greater  performance  outcomes  than  individual 
incentives;  (4)  there  are  likely  to  be  different  effects 
of  various  combinations  of  incentives  for  low-  and 
high-complexity  tasks  and  on  different  performance 
metrics;  and  (5)  feedback  on  performance  of  complex 
tasks  should  incorporate  explanatory  or  task-specific 
advice  for  maximum  value. 

Features  of  the  Cognitive-Motivational  Models 

In  contrast  to  the  behavioral  engineering  models 
reviewed  previously,  the  cognitive  motivational  mod¬ 
els  rely  to  a  greater  extent  on  experimental  evidence. 
These  models  are  also  more  cognitive  in  that  they 
incorporate  internal  personality  factors,  such  as  self- 
confidence,  and  cognitive  factors,  such  as  attention,  to 
explain  the  diverse  effects  of  various  motivators  on 
different  performance  outcomes.  The  cognitive-moti¬ 
vational  approach  considers  interactions  among 
diverse  factors  more  than  the  behavioral  perspectives, 
which  assume  a  consistent  effect  of  a  given  interven¬ 
tion.  The  behavioral  engineering  approach,  for  exam¬ 
ple,  views  feedback  linked  to  desired  behaviors  as  a 
consistently  effective  performance  improvement 
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mechanism.  In  contrast,  the  motivational  perspective 
proposes  that  feedback  may  be  more  or  less  effective 
depending  on  how  effectively  feedback  cues  direct 
attention  to  tasks. 


TECHNOLOGICAL  APPROACHES 
TO  PERLORMANCE  IMPROVEMENT 

In  the  behavioral  engineering  models  of  performance 
improvement,  we  identified  memory  support  in  the 
form  of  job  aids  as  one  common  intervention  recom¬ 
mended  in  situations  where  it  is  not  efficient  or  possi¬ 
ble  to  train  knowledge  and  skills  required  for  job  per¬ 
formance.  Technical  support  personnel  often 
encounter  new  issues  and  must  quickly  find  or  devise 
solutions.  Sales  representatives  are  often  challenged  to 
keep  up  with  ever-evolving  product  lines  and  features. 
Finance  controllers  may  be  asked  to  conduct  financial 
analyses  that  they  learned  months  ago  in  training. 
Although  training  could  be  developed  and  delivered  to 
address  these  situations,  instructional  interventions 
may  not  be  timely  or  cost  effective. 

To  address  these  issues,  electronic  performance  sup¬ 
port  systems  provide  users  with  "individualized  on-line 
access  to  the  full  range  of  ...  systems  to  permit  job 
performance”  (Gery,  1995,  p.  21).  Unlike  training, 
which  requires  participants  to  spend  time  away  from 
their  jobs,  EPSS  provides  the  information  and  tools  that 
they  need  to  do  their  job,  on  the  job.  Although  much  of 
the  EPSS  research  summarized  below  assumed  desktop- 
computer-delivered  support,  the  evolution  of  mobile 


computing  devices  opens  greater  opportunities  for  EPSS 
in  settings  where  computers  have  not  been  readily  avail¬ 
able;  aircraft  mechanics,  factory  workers,  and  sales 
agents  will  be  able  to  access  performance  support  con¬ 
tent  any  time  they  have  access  to  their  cell  phones, 
iPods,  or  other  small  form  factor  computing  devices. 

Types  of  Performance  Support 

Among  the  models  proposed  to  differentiate  the  types 
of  performance  support  systems,  Raybould  (2000) 
argued  that  performance  technologists  should  first 
employ  embedded  EPSS  and  then  move  to  less  pow¬ 
erful  linked  or  external  systems  if  embedded  systems 
are  not  possible  for  a  particular  performance  problem. 
Rossett  and  Schafer  (2006,  p.  67)  pointed  out  that 
planner  systems  are  used  when  a  performer  “gets 
ready  to  act  and  afterward,  when  [they]  reflect  on 
[their]  efforts,”  while  sidekick  systems  support  per¬ 
formers  while  they  are  “in  the  work.” 

The  most  widely  cited  and  discussed  categorization 
of  performance  support  was  developed  by  Gery  ( 1995), 
who  proposed  three  categories  of  performance  support 
systems:  external,  extrinsic,  and  intrinsic.  External  sys¬ 
tems  store  content  used  to  support  task  performance  in 
an  external  database.  This  content  is  not  integrated 
within  a  user’s  work  interface;  as  a  result,  users  are 
forced  to  manually  locate  relevant  information  in  the 
external  EPSS.  Figure  39.9  illustrates  a  common  form 
of  external  EPSS:  a  search  engine.  Other  examples 
include  frequently  asked  question  pages  and  help 
indexes.  External  performance  support  also  “may  or 
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may  not  be  computer  mediated”  (Gery,  1995,  p.  53). 
Job  aids  or  documentation  are  examples  of  non-com- 
puter-based  performance  support  interventions. 

Extrinsic  “[performance  support  ...  is  integrated 
with  the  system  but  is  not  in  the  primary  workspace” 
(Gery,  1995,  p.  51).  In  other  words,  extrinsic  systems 
integrate  with  the  user’s  work  interface  in  such  a  way 
that  the  EPSS  can  identify  the  user’s  location  in  a  sys¬ 
tem  or  even  the  exact  task  they  may  be  working  on. 
With  this  contextual  information,  the  extrinsic  system 
can  intelligently  locate  content  that  may  be  relevant  to 
the  task  at  hand.  Figure  39.10  shows  an  extrinsic  system 
that  embeds  help  links  in  a  software  interface.  These 
links  enable  the  EPSS  to  intelligently  locate  the  appro¬ 
priate  support  content  for  a  particular  work  context. 
Like  external  performance  support  systems,  the  content 
used  to  support  the  task  is  external  to  the  work  interface. 

Intrinsic  systems  provide  users  with  task  support 
that  is  incorporated  directly  within  their  work  inter¬ 
face.  Due  to  this  direct  integration  with  the  interface, 
Gery  asserted  that  intrinsic  EPSS  provides  “[perfor¬ 
mance  support  that  is  inherent  to  the  system  itself.  It’s 
so  well  integrated  that,  to  workers,  it’s  part  of  the 
system”  (Gery,  1995,  p.  51).  Under  this  rather  broad 
definition,  examples  of  intrinsic  performance  support 
systems  can  include  tools  that  automate  tasks  and  pro¬ 
cesses,  user-centered  design  of  work  interfaces  to 
reduce  complexity  and  improve  usability,  or  embedded 
knowledge  that  is  displayed  directly  in  the  work  inter¬ 
face,  as  illustrated  in  Figure  39.11. 


Applications  of  Performance  Support 

Case  studies  report  the  application  of  performance  sup¬ 
port  systems  to  a  wide  range  of  settings  and  perfor¬ 
mance  problems;  for  example,  performance  support 
systems  have  been  used  in  educational  settings.  Brush 
et  al.  (1993)  developed  a  performance  support  system 
to  improve  collaboration  among  teachers  in  rural  com¬ 
munities.  McCabe  and  Leighton  (2002)  created  an 
EPSS  to  help  master’s  students  with  analysis  and 
instructional  design.  Darabi  (2004)  explained  how  a 
similar  system  was  used  to  help  graduate  students  with 
performance  analysis. 

Performance  support  systems  have  also  been 
widely  used  in  industry.  Dorsey  et  al.  (1993)  and  Cole 
et  al.  (1997)  applied  performance  support  systems  to 
support  local  and  remote  sales  employees.  Huber  et  al. 
(1999)  provided  three  examples  of  how  intrinsic, 
extrinsic,  and  external  EPSS  were  applied  to  automo¬ 
bile  manufacturing,  insurance,  and  civil  engineering. 
Kasvi  and  Vartiainen  (2000)  demonstrated  four  differ¬ 
ent  ways  in  which  EPSS  has  been  applied  in  factories. 
Gery  (2003)  cited  examples  of  how  performance  sup¬ 
port  systems  have  been  used  in  investment  and  finan¬ 
cial  planning,  real  estate,  travel,  and  government  appli¬ 
cations  .  A  survey  conducted  by  McManus  and  Rossett 
(2006)  showed  that  performance  technologists  had 
applied  EPSS  to  problems  ranging  from  vessel  track¬ 
ing  by  the  U.S.  Coast  Guard  to  coaching  restaurant 
managers. 
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Figure  39.11  An  example  of  an  intrinsic  performance  support  system  using  tooltips.  (From  Nguyen,  F.  et  al..  Perform.  Improv.  Q., 
18(4),  71-86,  2005.  With  permission.) 


The  broad  proliferation  of  performance  support 
can  be  partially  attributed  to  cost  savings.  Several 
authors  have  proposed  methods  to  calculate  returns  on 
investment  for  performance  support  systems  (Altalib, 
2002;  Chase,  1998;  Desmarais  et  al.,  1997;  Hawkins 
et  al.,  1998).  Desmarais  et  al.  (1997)  implemented  an 
EPSS  for  support  customer  service  representatives  at 
a  utility  company  and  estimated  a  modest  monetary 
savings  realized  through  benefits  such  as  reduced  call 
time,  improved  productivity,  and  reduced  training. 
Hawkins  et  al.  (1998)  reported  that  an  EPSS  developed 
for  a  government  agency  saved  over  $17.6  million  over 
the  lifetime  of  the  system. 

The  benefits  of  performance  support  extend 
beyond  dollars  and  cents.  Hunt  et  al.  (1998)  conducted 
a  meta-analysis  on  performance  support  systems  in  the 
medical  field.  The  authors  reviewed  a  total  of  68  stud¬ 
ies  with  systems  that  supported  drug  dosing,  patient 
diagnoses,  and  preventive  care.  User  performance 
improved  in  42  of  the  studies  reviewed,  was  not  sig¬ 
nificantly  changed  in  19  cases,  and  decreased  in  only 
7  instances.  Although  studies  on  EPSSs  designed  for 
diagnoses  and  certain  types  of  drug  dosing  were  incon¬ 
sistent,  the  researchers  noted  that  systems  for  other 
areas  such  as  preventive  care  demonstrate  positive 
results  for  these  systems  in  ambulances,  clinics,  and 
hospitals.  Villachica  et  al.  (2006)  reported  the  benefits 
of  performer-centered  design  (PCD)  winners  from 
1997  to  2004.  The  impact  of  these  projects  ranged  from 
reductions  in  training  time,  increased  user  satisfaction, 
and  improved  productivity,  among  other  factors. 


Research  on  Performance  Support  Systems 

Research  has  also  examined  the  most  effective  types 
of  performance  support  systems.  Bailey’s  (2003)  meta¬ 
analysis  found  that  linking,  as  one  would  do  in  an 
intrinsic  or  extrinsic  performance  support  system, 
“tends  to  be  more  effective  than  searching  to  find  con¬ 
tent”  as  in  an  external  system  (Bailey,  2003,  p.  1). 
Similarly,  Spool  (2001,  p.  1)  found  that:  “The  more 
times  users  searched,  the  less  likely  they  were  to  find 
what  they  wanted”;  in  fact,  Spool  observed  that  users 
were  successful  in  finding  the  correct  support  infor¬ 
mation  55%  of  the  time  on  the  first  try.  On  the  second 
search  attempt,  only  38%  were  successful,  and  none 
was  successful  on  the  third  attempt.  Nielsen  (2001) 
found  similar  results  in  another  study  where  5 1  %  were 
successful  on  the  first  search  attempt,  32%  on  the 
second,  and  18%  on  the  third  attempt. 

A  study  by  Nguyen  et  al.  (2005)  validated  some 
of  these  results.  The  researchers  tested  three  different 
types  of  performance  support  systems  that  aligned  with 
Gery’s  intrinsic,  extrinsic,  and  external  EPSS  catego¬ 
ries.  Results  from  the  study  indicated  that  users  pro¬ 
vided  with  intrinsic  and  extrinsic  performance  support 
systems  performed  significantly  better  on  a  software 
procedure  compared  to  a  control  group  with  no  EPSS. 
Also,  all  users  provided  with  an  EPSS  had  significantly 
more  positive  attitudes  than  the  control  group. 

Research  addressing  other  assumptions  about  per¬ 
formance  support  is  currently  lacking  or  inconclusive. 
One  of  the  most  widely  held  notions  about  EPSS,  for 
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example,  is  that  implementing  on-the-job  support 
interventions  can  reduce  or  even  eliminate  the  amount 
of  training  necessary  to  address  a  performance  prob¬ 
lem  (Chase,  1998;  Desmerais  et  al.,  1997;  Foster, 
1997;  Sleight,  1993).  This  notion  of  reducing  training 
through  EPSS  and  enabling  “day-one  performance” 
has  been  a  major  attraction  for  performance  technol¬ 
ogists.  Bastiaens  et  al.  (1997)  examined  this  assump¬ 
tion  by  comparing  the  effectiveness  of  different  com¬ 
binations  of  computer-based  and  paper-based 
performance  support  with  computer-based  and  instruc¬ 
tor-led  training.  They  found  that  users  preferred  paper- 
based  forms  over  the  electronic  software  tool  as  well 
as  instructor-led  training  over  computer-based  train¬ 
ing.  They  found  no  significant  difference  on  test 
achievement  scores,  performance,  or  sales  results  for 
the  other  treatments  over  a  1-year  period.  Mao  and 
Brown  (2005)  examined  the  effect  of  using  an  EPSS 
to  support  pre-task  exercises  and  on-the-job  perfor¬ 
mance  as  compared  to  an  instructor-led  training 
course.  Users  provided  with  the  EPSS  performed  sig¬ 
nificantly  better  on  an  achievement  test  than  those 
provided  with  training.  They  found  no  significant  dif¬ 
ference  between  the  two  groups  on  a  procedural  task. 
It  should  be  noted  that  both  of  these  studies  suffered 
from  potential  validity  problems,  ranging  from  small 
sample  sizes  to  procedural  issues  (such  as  using  the 
EPSS  for  practice  exercises— not  an  authentic  use  of 
on-the-job  support). 

RESEARCH  RECOMMENDATIONS 
FOR  PERFORMANCE 
IMPROVEMENT  MODELS 

The  advent  of  meta-analyses  and  effect  sizes  has  in  a 
short  time  period  harvested  useful  performance 
improvement  guidelines  gleaned  from  many  research 
studies.  In  this  chapter,  we  have  summarized  findings 
from  meta-analyses  of  research  on  goal  setting,  feed¬ 
back,  incentives,  and  EPSS.  After  gathering  relevant 
studies,  the  research  team  reviews  and  selects  only 
those  studies  that  meet  criteria  to  be  included  in  the 
meta-analysis.  Most  meta- analyses  are  able  to  use  only 
a  small  fraction  of  the  available  studies  because  many 
do  not  meet  criteria  for  validity.  We  forward  their  call 
for  research  of  higher  quality  that  is  either  experimen¬ 
tal  or  quasi-experimental  with  control  groups  to  aug¬ 
ment  the  many  survey,  case  study,  and  anecdotal 
reports  on  performance  improvement  initiatives. 

We  cited  the  study  by  Peterson  and  Luthans  (2006) 
of  incentives  in  fast-food  stores  as  an  example  of 
research  that  is  useful  because:  (1)  it  was  quasi-exper¬ 
imental  in  that  it  included  a  control  group;  (2)  it  com¬ 


pared  two  types  of  incentives— monetary  vs.  feedback 
with  recognition;  (3)  it  examined  the  effects  of  the 
incentives  on  three  outcomes  — profit,  drive-through 
times,  and  turnover;  (4)  it  evaluated  the  effects  of 
incentives  awarded  to  a  team  rather  than  to  individuals; 
and  (5)  it  evaluated  outcomes  over  a  9-month  period 
of  time. 

We  recommend  that  the  following  issues  or  ques¬ 
tions  be  addressed  in  future  research: 

•  How  do  task  complexity  and  worker  expertise 
affect  performance  interventions ?  Many  of 
the  studies  on  goal  setting,  feedback,  and 
incentives  have  focused  on  low-complexity 
tasks;  however,  the  core  strategic  advantage 
for  many  organizations  relies  more  on  high- 
complexity  work  such  as  design,  analysis, 
communication,  and  human  interactions.  The 
outputs  of  these  tasks  are  more  challenging 
to  quantify,  which  may  explain  their  scarcity 
in  the  literature  to  date.  Task  complexity  is 
relative  to  the  experience  of  the  worker  or 
team.  We  know  from  cognitive  load  theory 
(Clark  et  al.,  2006)  that  instructional  interven¬ 
tions  that  work  well  for  novice  learners  not 
only  do  not  help  those  with  more  experience 
but  in  many  cases  depress  their  performance. 

It  is  likely  that  the  types  of  goals,  feedback, 
incentives,  and  EPSSs  that  are  appropriate  for 
higher  complexity  work  and  less  experienced 
workers  will  be  different  than  those  that  best 
serve  low-complexity  tasks  and  new  workers; 
for  example,  the  research  reports  on  the 
effects  of  feedback  suggest  that  information 
cues  in  the  feedback  may  have  greater  posi¬ 
tive  results  on  high  than  on  low  task  complex¬ 
ity  (Kluger  and  DeNisi,  1996). 

•  How  should  performance  support  be 
deployed  through  new  technology?  The 
research  described  previously  on  elec¬ 
tronic  performance  improvement  initia¬ 
tives  assumed  delivery  using  a  desktop 
computer.  With  the  convergence  of  com¬ 
puting  and  telecommunications,  small 
hand-held  mobile  devices  offer  the  poten¬ 
tial  to  deliver  performance  support  to  a 
new  generation  of  technology.  Research  on 
feedback  showed  better  results  from  com¬ 
puter-delivered  feedback  than  from  feed¬ 
back  provided  by  a  person  (Kluger  and 
DeNisi,  1996).  We  need  research  to  define 
what  kinds  of  feedback,  training,  and 
EPSSs  are  best  provided  on  devices  such 
as  cell  phones,  PDAs,  and  iPods. 
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•  Build  explanatory  models  and  taxonomies. 
Performance  improvement  models  that 
incorporate  moderators  of  interventions  offer 
opportunities  for  building  performance  theo¬ 
ries  and  for  adapting  interventions  to  unique 
situations.  The  Kluger  and  DeNisi  (1996) 
feedback  model,  for  example,  suggests  that 
feedback  will  have  its  effects  based  on  the 
types  of  cues  embedded  in  the  feedback  with 
cues  that  direct  attention  to  the  task  being 
most  useful.  This  model  can  be  tested  and,  if 
valid,  offers  practitioners  guidelines  for 
delivering  feedback  in  ways  that  obtain  the 
most  benefit.  Regarding  EPSS,  more  robust 
taxonomies  of  the  type  of  performance  sup¬ 
port  needed  would  help  practitioners  define 
the  best  types  of  support  systems  to  use  for 
given  tasks,  workers,  and  devices. 

•  Define  the  effects  of  combined  interventions . 
The  majority  of  research  reports  focused  pri¬ 
marily  on  a  single  intervention  or  perhaps 
on  a  combination  of  two  such  as  goals  and 
feedback.  We  could  benefit  from  more  stud¬ 
ies  that  look  at  combinations  of  incentives, 
such  as  the  Stajkovic  and  Luthans  (2003) 
meta-analysis  suggesting  a  synergistic  effect 
among  goals,  feedback,  and  incentives.  We 
need  more  data  to  understand  the  interac¬ 
tions  among  interventions  such  as  EPSSs 
and  training  or  incentives  and  goal  setting. 

•  Support  and  disseminate  research.  In  this 
chapter,  we  have  briefly  summarized  a 
behavioral  engineering,  a  cognitive-motiva¬ 
tional,  and  a  technological  perspective  on 
performance  improvement.  We  suspect  that 
these  communities  of  practice  often  work 
independently  and  fail  to  leverage  their 
potential  combined  strengths.  In  addition, 
practitioners  rarely  read  the  same  publica¬ 
tions  and  websites  or  attend  the  same  con¬ 
ferences  as  the  research  community.  We 
recommend  a  greater  exchange  of  informa¬ 
tion  among  these  communities  initiated  by 
professional  associations  involved  in  perfor¬ 
mance  improvement. 


THE  FUTURE  OF 
PERFORMANCE  IMPROVEMENT 

We  began  our  chapter  by  providing  four  vignettes  of 
organizational  failures  that  were  not  caused  by  a  lack 
of  knowledge  and  skills.  We  believe  that  a  performance 


improvement  perspective  is  an  essential  tool  set  to 
develop  solution  systems  that  do  not  rely  on  training 
alone  to  reach  organizational  objectives.  The  recent 
adoption  of  performance  improvement  certification 
programs  by  practitioner  professional  associations  that 
formerly  emphasized  training  only,  such  as  the  Amer¬ 
ican  Society  for  Training  and  Development  and  the 
International  Society  for  Performance  Improvement, 
along  with  increased  mobility  of  electronic  technolo¬ 
gies,  is  advancing  the  opportunities  for  performance 
improvement  initiatives.  At  the  same  time,  the  lack  of 
experimental  studies  combined  with  failure  to  commu¬ 
nicate  the  research  results  outside  of  the  academic 
communities  limit  the  full  potential  of  performance 
improvement  interventions.  This  is  an  opportune  time 
to  focus,  improve,  and  disseminate  research  efforts  to 
build  a  more  evidence-based  foundation  for  perfor¬ 
mance  improvement  initiatives.  We  hope  this  chapter 
and  the  Handbook  contribute  to  that  effort. 
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ABSTRACT 

The  nature  of  information  has  changed  dramatically  dur¬ 
ing  the  past  25  years.  The  digital  age  has  redefined  the 
nature  of  a  resource  and  dramatically  changed  how 


resources  are  used  to  support  learning.  Media  of  often 
unknown  origin  and  quality  are  now  used  across  a  wide 
range  of  learning  systems  to  address  diverse  epistemo¬ 
logical  beliefs  and  associated  learning  goals.  The  number 
and  types  of  resources  have  also  grown  exponentially. 
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Several  factors  increase  the  viability  of  resources  for 
learning,  including  access  to  resources  in  contexts  not 
previously  available,  increased  flexibility  in  their  use, 
and  ready  availability,  manipulability,  and  sharability. 
The  purposes  of  this  chapter  are  to  present  a  brief 
historical  perspective  on  resource-based  learning; 
describe  components  of  resource-based  learning; 
introduce  the  influence  of  underlying  epistemology, 
foundations,  and  assumptions  in  grounding  resource 
use;  critically  analyze  related  research;  and  identify 
implications  for  RBL. 


KEYWORDS 

Resource:  “Media,  people,  places  or  ideas  that  have 
the  potential  to  support  learning”  (Hill  and  Hanna¬ 
fin,  2001,  p.  38). 

Resource-based  learning:  The  use  and  application  of 
available  assets  to  support  varied  learning  needs 
across  contexts. 

Scaffolding:  Process  through  which  individuals  are 
supported  in  identifying,  interpreting,  or  otherwise 
using  resources. 

Tools:  Devices  that  aid  individuals  to  engage  and 
manipulate  resources  and  ideas. 


INTRODUCTION 

The  digital  age  has  both  redefined  the  nature  of  infor¬ 
mation  and  transformed  educational  resources. 
Resources  need  no  longer  be  intact  and  self-contained 
but  can  be  readily  repurposed  as  well  as  accessed, 
created,  modified,  and  assembled  from  virtually  any¬ 
where  to  address  individual  goals  and  needs.  The  edu¬ 
cational  implications  have  only  begun  to  emerge  as 
homes,  schools,  classrooms,  workplaces,  and  commu¬ 
nity  centers  become  increasingly  resource  centered. 

The  potential  and  challenges  of  resource-based 
learning  (RBL)  are  considerable.  Increasingly,  design¬ 
ers  and  learners  must  evaluate  growing  numbers  of 
digital  resources  that  are  developed  for  purposes  other 
than  those  being  sought  and  used,  that  exist  in  ever- 
expanding  and  geographically  dispersed  repositories, 
and  that  are  often  of  unknown  quality,  accuracy,  and 
integrity.  So,  while  technology  has  been  lauded  for 
potentially  democratizing  access  to  information  (Kell¬ 
ner,  2003),  educational  use  remains  fraught  with  issues 
of  literacy,  misinterpretation,  and  propagandizing 
(Brooks,  2003)  as  well  as  problems  with  copyright  and 
fair  use  (Kahle,  1997). 

No  singular,  universally  accepted  definition  exists 
as  to  what  constitutes  a  resource  or  how  resources 


should  be  used  during  learning.  One  distinction  has 
focused  on  the  extent  to  which  resources  embody  ped¬ 
agogical  attributes.  Wiley  (2001)  characterized  a  learn¬ 
ing  object  as  a  digital  resource  that  embodies  pedagog¬ 
ical  properties  that  can  be  reused  to  facilitate  learning. 
Downes  (2004)  characterized  a  resource  as  being  com¬ 
prised  of  and  supporting  varied  functions  distributed 
across  users  and  locations.  Increased  granularity  may 
enable  educators  to  utilize  resources  to  address  a  wide 
variety  of  learning  needs  and  to  support  a  broad  range 
of  learning  models,  as  well  as  to  realize  the  potential 
of  digital  resources  (Hodgins,  2001;  Schatz,  2005). 

Earlier,  we  characterized  a  resource  as  an  informa¬ 
tion  asset  of  broad  applicability  and  defined  resources 
as  “media,  people,  places,  or  ideas  that  have  the  poten¬ 
tial  to  support  learning”  (Hill  and  Hannafin,  2001,  p. 
38).  Furthermore,  we  argued  that  the  meaning  of  a 
given  resource  varies  according  to  the  epistemological, 
psychological,  and  pedagogical  context  of  its  use. 
Accordingly,  we  define  resource-based  learning  as  the 
use  and  application  of  available  assets  to  support  var¬ 
ied  learning  needs  across  contexts  (Beswick,  1977; 
Doiron  and  Davies,  1998;  Haycock,  1991). 

Several  factors  increase  the  viability  of  resources 
for  learning,  including  access  in  contexts  not  previ¬ 
ously  available,  increased  flexibility  in  their  use,  and 
ready  availability,  manipulability,  and  sharability 
across  multiple  contexts  and  purposes.  The  purposes 
of  this  chapter  are  to  present  a  brief  perspective  on  and 
describe  components  of  resource-based  learning, 
describe  how  resource  definitions  and  use  are  mediated 
by  the  underlying  beliefs  and  practices  associated  with 
different  learning  models,  and  identify  implications  for 
creating  and  using  digital  resources  for  varied  learning 
purposes. 

THE  EMERGENCE  OF 
RESOURCE-BASED  LEARNING 

Although  the  use  of  resources  for  learning  is  not  new 
(Beswick,  1977;  Haycock,  1991),  their  creation  and  use 
were  limited  in  predigital  environments.  Resources 
were  typically  static,  created  and  published  by  compa¬ 
nies,  and  used  largely  intact  to  address  specific,  well- 
defined  needs  and  goals.  In  some  instances,  parts  of  an 
intact  resource  were  used  (e.g.,  chapter  in  a  textbook), 
but  it  remained  difficult  to  adapt  resources  to  address 
individual  needs.  The  digital  age  has  changed  the  very 
nature  of  a  resource.  The  resulting  opportunities  are 
considerable  and  range  from  increasing  the  number  and 
availability  through  granularization,  to  repurposing 
component  resources  to  address  myriad  individual 
needs,  to  providing  multiple  perspectives.  We  now  have 
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ready  access  to  more  traditional  historic  (e.g.,  books, 
articles)  and  contemporary  (e.g.,  daily  news)  informa¬ 
tion  sources  (Maddux  and  Johnson,  1997),  as  well  as 
emerging,  dynamic  information  sources  (e.g.,  blogs, 
wikis,  podcasts)  (Martindale  and  Wiley,  2005). 
Although  the  opportunities  afforded  through  digitiza¬ 
tion  are  considerable,  challenges  remain.  In  this  chap¬ 
ter,  we  describe  the  transformation  of  resources  for 
educational  purposes;  introduce  the  foundations, 
assumptions,  and  components  of  resource-based  learn¬ 
ing;  and  describe  a  range  of  studies  utilizing  resource- 
based  learning  in  varied  teaching-learning  models. 

Metamorphosis  of  Media 

Media  have  become  vastly  easier  to  produce  and 
access;  for  example,  high-quality,  sharable  digital  doc¬ 
uments  have  become  commonplace  through  word  pro¬ 
cessing,  movie  making,  and  desktop  publishing 
(Counts,  2006).  It  is  no  longer  necessary  for  large 
design  and  production  facilities  to  assemble  or  develop 
information  resources,  as  individuals  and  small  teams 
now  have  the  ability  to  create— even  customize  — 
resources  rapidly.  Digitization  has  also  reduced  the 
need  to  warehouse  resources  at  centralized  reposito¬ 
ries.  According  to  Lyman  and  Varian  (2003),  the  Web 
has  become  the  leading  technology  for  accessing  and 
sharing  information.  Traditional  publishing  and  pro¬ 
duction  houses,  although  important,  are  no  longer 
required  to  produce,  store,  and  distribute  information. 
Users  must  no  longer  wait  for  a  book  to  arrive  in  a 
library  or  bookstore;  many  titles  are  now  available 
directly  from  authors  (see,  for  example,  Stephen 
King’s  downloadable  novels  at  http://www.stephen- 
king.com/).  Likewise,  television  and  movie  titles  are 
now  downloaded  directly  or  accessed  on-demand— a 
trend  that  is  projected  to  expand  rapidly  (Avery,  2004) . 
Numerous  online  brokers  and  vendors  (e.g.,  iTunes, 
Microsoft,  Amazon.com)  allow  users  to  download  dig¬ 
ital  music,  computer  software,  books  (.pdf  documents 
or  audiobooks),  and  video  resources  directly.  Finally, 
public  information,  such  as  historical  documents, 
weather  information,  and  government  reports,  is 
increasingly  digital  in  nature.  Digital  media  are  now 
available  directly  from  a  myriad  of  primary  sources 
and  brokers,  democratizing  availability  and  access 
across  a  vast  array  of  hitherto  centralized  resources. 

Socially  Constructed  Resources 

Blogs  and  wikis  have  become  a  primary  source  of 
information  across  a  growing  range  of  users.  Blogs 
enable  a  wide  distribution  of  ideas,  and  wikis  enable 
broad  distribution  as  well  as  the  collaborative  building 


of  ideas  (Engstrom  and  Jewett,  2005).  Rainie  (2005), 
in  his  work  with  the  Pew  Internet  and  American  Life 
Project,  documented  a  58%  rise  in  blog  readership  in 
2004.  Political  blogs,  such  as  Instapundit  and  Daily 
Kos  (Harp  and  Tremayne,  2006),  were  among  the  top 
blogs  visited,  with  9%  of  respondents  to  the  Pew  Inter¬ 
net  survey  reporting  regular  or  occasional  readership 
(Rainie,  2005). 

The  increased  use  of  political  and  news-related 
blogs  provides  an  important  indication  of  how  socially 
constructed  resources  have  democratized  access  to 
ideas  and  information  (Bichard,  2006).  Educators  have 
used  blogs  and  wikis  both  within  their  classes  and  to 
share  ideas  across  a  variety  of  disciplines  and  grade 
levels  (Martindale  and  Wiley,  2005).  (For  a  description 
of  wikis  in  a  graduate-level  information  systems 
course,  see  Raman  et  al.,  2005;  for  a  discussion  of 
preservice  teachers  using  blogs  to  facilitate  learning  in 
an  instructional  technology  course,  see  West  et  al., 
2006;  for  a  discussion  of  the  use  of  wikis  with  middle- 
grades  students  to  engage  the  learners  in  the  critical 
examination  of  an  environmental  challenge,  see  Eng¬ 
strom  and  Jewett,  2005.) 

Whereas  interest  in  and  the  use  of  forums  such  as 
blogs  and  wikis  appear  to  be  increasing,  some  question 
their  contributors,  readers,  and  value  of  the  integrity 
of  their  contents.  Harp  and  Tremayne  (2006)  indicated 
that  the  majority  of  blogging  in  political  contexts  is 
male  dominated.  Likewise,  upon  examining  bloggers 
in  the  Pew  Internet  project,  Rainie  (2005,  p.  2)  con¬ 
cluded  that  “like  bloggers,  blog  readers  are  more  likely 
to  be  young,  male,  well-educated,  Internet  veterans.” 
This  suggests  that  the  presumably  open-forum  blog¬ 
ging  available  on  the  Internet  may  actually  reflect 
rather  narrow,  rather  than  democratic  and  open,  infor¬ 
mation  resources.  Other  challenges  discussed  in  the 
literature  include  variable  reading  level,  unique  com¬ 
munication  styles,  quality  of  information,  and  infor¬ 
mation  literacy  skills  needed  to  generate,  interpret,  and 
use  such  resources  (McPherson,  2006). 

Sophistication  of  Information  Systems 

During  the  past  two  decades,  the  amount  of  informa¬ 
tion  available  has  increased  exponentially;  for  exam¬ 
ple,  Lawrence  and  Giles  (1998)  estimated  that  the 
indexable  Web  was  comprised  of  approximately  320 
million  pages.  More  recently,  DeKunder  at  Tilburg 
University  estimated  the  total  Web  pages  to  be  14.3 
billion,  with  a  sustained  growth  rate  of  2%  per  month 
(Tilburg  University,  2006).  With  the  extraordinary  vol¬ 
ume  of  available  resources,  problems  have  emerged, 
particularly  with  locating  the  “right”  information. 
Mark-up  technologies  provide  descriptive  metatags 
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where  authors  can  specify  unique  search  terms  and 
categories  for  the  end  user  (Hill  et  al.,  2007),  which 
provides  fine-grained  indexing  previously  unavailable 
by  a  resource’s  creator;  however,  some  have  suggested 
that  the  use  of  a  given  resource  is  continually  redefined 
by  subsequent  users.  Metadata  may  provide  guidance 
in  identifying  media  attributes  but  little  insight  into 
appropriateness  for  use  in  given  contexts  or  for  learn¬ 
ing  models  (Mwanza  and  Engestrom,  2005). 

Affordability,  Power,  and  Availability 

Technologies  have  decreased  in  cost  while  growing 
exponentially  in  capability.  According  to  recent  esti¬ 
mates,  the  cycle  processing  rates  of  computer  operat¬ 
ing  systems  double  every  18  to  20  months  (Ekman  et 
al.,  2004).  At  the  onset  of  the  21st  century,  a  high 
percentage  of  households  in  Western  countries 
reported  at  least  one  computer  with  Internet  access 
(NTIA,  2000).  Low-cost  wireless  computer  technolo¬ 
gies  are  now  widely  available  and  in  use,  enabling  cost- 
effective  access  to  almost  anything,  anytime,  and  any¬ 
place  (for  a  recent  study  exploring  the  promises  and 
challenges  of  nomadic  computing,  see  Cousins,  2004). 
Internet  access  has  also  improved  exponentially. 
According  to  recent  Federal  Communication  Commis¬ 
sion  statistics  (FCC,  2006),  access  speeds  range  from 
telephone  dial-up  at  56  kilobits  per  second  to  broad¬ 
band  speeds  of  200  kilobits  per  second  to  the  home. 
T- 1  access  speeds  ( 1 .5  megabytes  per  second)  are  now 
commonplace  in  public  and  business  facilities.  For 
RBL,  both  traditional  digital  text  and  graphic 
resources,  as  well  as  streaming  video  and  audio 
resources,  can  now  be  provided  to  the  home  as  well 
as  to  education  settings  affording  ready  access  to  a 
wealth  of  basic  and  high-end  media  resources. 

Changing  Nature  of  Resources 

The  emergence  of  learning  objects  (Northrup,  2007; 
Wiley,  2001)  has  stimulated  alternative  conceptions  of 
resources.  Traditional  video  content  can  be  segmented 
into  a  series  of  scenes  or  clips  that  can  be  accessed 
individually  or  viewed  in  their  entirety.  Books  need 
not  be  defined  in  their  entirety  but  can  be  comprised 
of  individual  objects  (e.g.,  words,  tables,  figures,  and 
pictures)  that  can  be  used  independently  or  in  combi¬ 
nations.  The  meaning  is  continually  redefined  as  audio 
recordings,  video  scenes,  and  Web  resources  are  recon¬ 
textualized.  Individual  resource  components,  thus,  are 
malleable  rather  than  fixed,  affording  infinite  potential 
to  redefine  meanings,  allow  multiple  literacies  (Huot 
et  al.,  2004),  and  support  different  learning  goals,  con¬ 
texts,  and  models. 


Economic  and  Practical  Influences 

According  to  the  National  Education  Association 
(NEA,  2004),  the  past  two  decades  have  brought  rad¬ 
ical  budgetary  constraints  to  both  formal  and  informal 
educational  initiatives.  Availability  of  resources  is 
required  across  learning  contexts  and  models  to  enable 
use  for  varied  purposes.  Resource-based  learning,  par¬ 
ticularly  in  digital  environments,  offers  promise  for 
broad  applicability  across  a  variety  of  contexts  to  pro¬ 
vide  access  to  an  expanding  global  library  of  digital 
resources. 


COMPONENTS 

Resource-based  learning  involves  establishing  con¬ 
texts  for,  tools  for  acting  on  and  with,  and  scaffolds  to 
guide  the  differentiated  interpretation,  use,  and  under¬ 
standing  of  resources  in  ways  that  are  consistent  with 
the  epistemology,  foundations,  and  assumptions  of  a 
given  learning  model  (Hill  and  Hannafin,  2001).  RBL 
does  not  embody  a  particular  epistemology  but  rather 
provides  a  process  through  which  epistemologically 
different,  but  grounded,  learning  models  are  enacted. 
WebQuests,  for  example,  are  “inquiry-oriented 
activities]  in  which  some  or  all  of  the  information  that 
learners  interact  with  comes  from  resources  on  the 
Internet”  (Dodge,  1995,  p.  12).  In  open-ended  learning 
environments,  resources  are  presumed  to  be  epistemo¬ 
logically  neutral  until  contextualized  and  designed  to 
support  unique  learning  goals  (Hannafin  et  al.,  1999). 
In  the  following  sections,  we  describe  how  resource- 
based  learning  is  enacted  differentially  to  support  dif¬ 
ferent  teaching-learning  models. 

Context 

External  contexts— supplied  situations  or  problems  — 
are  provided  by  an  instructor  or  external  agent  to  orient 
learners  to  a  particular  learning  goal.  Specific  problem 
statements  or  questions,  consistent  with  objectivist 
epistemology,  direct  expectations  explicitly  and  guide 
the  strategies  and  resources  used  by  teachers  and  stu¬ 
dents  (Hannafin  et  al.,  1997).  In  individual  contexts, 
learners  establish  learning  contexts  based  on  their 
unique  circumstances  or  needs.  Consistent  with  con¬ 
structivist  epistemology,  individuals  establish  the 
learning  context,  define  their  knowledge  and  skill 
needs,  identify  resources  to  meet  the  needs,  and  situate 
the  resource’s  meaning  to  address  their  needs.  Nego¬ 
tiated  contexts  combine  aspects  of  external  and  indi¬ 
vidual  contexts.  Typically,  an  external  problem  or  issue 
is  provided  (e.g.,  social  justice,  civil  rights),  but  the 
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specific  problems  to  be  addressed  (e.g.,  what  led  to 
nonviolent  protests  in  the  mid-20th  century?)  are 
unique  to  individual  learners  (Hill  et  al.,  2007). 

In  problem-based  learning  models,  the  context  typ¬ 
ically  enables  multiple  issues  to  be  identified  and  stud¬ 
ied  using  varied  resources  to  support  individual 
approaches.  Iiyoshi  ( 1999),  for  example,  examined  the 
effects  of  different  orienting  contexts  on  nursing  stu¬ 
dents’  search  strategies,  interpretation,  and  under¬ 
standing  using  multimedia  resources  contained  in  a 
multimedia  database,  the  Human  Body.  Students  were 
provided  tasks  that  explicitly  directed  them  to  specific 
resources,  focused  on  relationships  between  two  or 
more  resources,  or  posed  a  problem  requiring  a  syn¬ 
thesis  of  information  across  multiple  resources.  Par¬ 
ticipants  tended  to  learn  specific  concepts  using 
directed  tasks  but  were  only  able  to  synthesize  mean¬ 
ing  across  resources  when  oriented  accordingly.  Sim¬ 
ilarly,  the  Jasper  Woodbury  series  establishes  problem 
contexts  using  authentic  video  vignettes  depicting 
dilemmas  wherein  different  problems  and  subprob¬ 
lems  can  be  identified  to  establish  and  implement  a 
plan  (Barron  et  al.,  1995;  Cognition  and  Technology 
Group  at  Vanderbilt,  1992). 

Tools 

Tools  enable  learners  to  engage  and  manipulate  both 
resources  and  ideas.  Tool  use  varies  with  context  and 
intention;  the  same  tool  can  support  different  activities 
based  on  its  alignment  with  given  learning  models. 
Numerous  tool  types  (see,  for  example.  Hill  and 
Hannafin,  1997)  and  functions  (Iiyoshi  et  al.,  2005) 
have  been  identified.  Processing  tools  help  to  manage 
cognitive  demands  associated  with  different  RBL 
models.  When  embodying  objectivist  epistemology, 
spreadsheets  enable  learners  to  manipulate  scenarios 
to  test  the  limits  within  a  simulation,  extend  their  cog¬ 
nitive  abilities,  and  reduce  the  extraneous  cognitive 
load  associated  with  tasks  and  mental  manipulation 
(Hannafin  et  al.,  2007).  The  same  tools,  in  contrast, 
may  embody  constructivist  epistemology  when  used 
to  negotiate  a  resource’s  meaning  in  technology- 
enhanced,  distance  learning  environments  (Hill  et  al., 
2007). 

Searching  tools  can  be  used  in  directed  as  well  as 
learner-centered  environments.  General-purpose  Web- 
based  search  engines  such  as  Google  (www.goo- 
gle.com)  identify  the  location  of  and  direct  access  to 
a  variety  of  electronic  resources.  Google  Scholar, 
which  seeks  and  generates  Web-indexed  links  among 
published  documents,  and  Apple  Learning  Interchange 
(http://ali.apple.com/ali_appleworks/templates.shtml), 
which  supports  searching  for  highly  indexed  pre-12 


lesson  planning  resources,  provide  sophisticated  func¬ 
tion-specific  searching  tools.  Domain-specific 
resources  can  be  searched  using  the  State  of  Georgia’s 
GALILEO  database  (www.galileo.usg.edu),  which 
contains  a  variety  of  subject-specific  as  well  as  general 
search  engines,  enabling  direct  access  to  associated 
electronic  resources  (e.g.,  journal  articles,  reports)  as 
well  as  resources  in  libraries. 

Manipulation  tools  are  used  to  test  or  explore.  Con¬ 
sistent  with  constructivist  epistemology,  applications 
such  as  SimCity  or  SimEarth  empower  learners  to 
hypothesize,  design,  build,  test,  and  rebuild  cities  and 
ecosystems  to  investigate  relationships  between  and 
among  objects  and  systems.  Manipulation  tools  allow 
the  testing  of  environmental  scenarios  and  manipula¬ 
tion  of  systemic  concepts  such  as  global  warming  in 
ways  otherwise  impossible  or  impractical. 

Finally,  communication  tools  support  the  ability  to 
exchange  information  and  ideas.  Asynchronous  tools 
such  as  e-mail,  podcasts,  blogs,  and  wikis  enable  and 
sustain  access  among  learners,  instructors,  and  experts. 
Synchronous  communication  tools  (e.g.,  instant  mes¬ 
saging,  videoconferencing)  provide  access  to  others  in 
real  time,  whether  they  are  located  a  few  rooms  away 
or  around  the  globe. 

Scaffolds 

Scaffolding  is  support  provided  initially  and  subse¬ 
quently  faded  in  a  continuous  cycle  as  knowledge  and 
understanding  develops.  The  amount  and  frequency  of 
scaffolding  vary  with  the  individual  learner,  problems 
encountered,  and  the  needs  or  demands  of  a  specific 
context  (Sharma  and  Hannafin,  2007).  Procedural, 
conceptual,  metacognitive,  and  strategic  scaffolds  are 
especially  relevant  for  RBL.  Procedural  scaffolds 
emphasize  how  to  use  the  features  and  functions  of  a 
given  resource.  Greene  and  Land  (2000),  for  example, 
made  extensive  use  of  procedural  scaffolds  to  support 
preservice  teachers  in  their  development  of  an  Internet- 
based  RBL  lesson.  Procedural  scaffolds  allow  learners 
to  focus  cognitive  resources  for  other  learning  activi¬ 
ties  (e.g.,  problem  solving).  Conceptual  scaffolds 
guide  learners  in  what  to  consider  by  assisting  with 
the  identification  of  knowledge  related  to  a  problem 
or  by  making  connections  between  concepts  more 
apparent.  Resource-intense  toolkits  were  created  by 
the  Open  University  to  assist  students  with  learning  at 
a  distance  in  a  directed  environment  (Jelfs  et  al.,  2004). 
Metacognitive  scaffolds,  common  in  inquiry-based 
environments,  prompt  reflection,  comparison,  and 
revision  based  on  self-assessments  of  understanding. 
Checklists,  for  example,  can  assist  the  learner  with 
reflecting  upon  decisions  made  or  actions  to  take. 
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Cases  also  support  reflection  and  comparison,  present¬ 
ing  scenarios  for  learners  to  consider  as  well  as  confir¬ 
mation  points  where  learners  reflect  on  their  understand¬ 
ing  to  reveal  what  they  do  or  do  not  understand  (Kim 
and  Hannafin,  in  press  a;  Kolodner  et  al.,  2004) .  Finally, 
strategic  scaffolds  provide  assistance  in  identifying 
ways  to  analyze,  plan,  and  respond  (e.g.,  identifying  and 
selecting  information,  evaluating  resources).  Many 
models  have  been  developed  to  guide  learners  in  devel¬ 
oping  and  applying  information  literacy  skills  (see,  for 
example,  Eisenberg  et  al.,  2004).  Several  libraries  have 
created  websites  to  guide  learners  with  evaluating  print 
and  electronic  resources  that  can  be  used  across  learning 
contexts.  The  University  of  California-Berkeley  (http:// 
www.lib.berkeley.edu/TeachingLib/Guides/Evalua- 
tion.html)  and  Purdue  University  (http://owl.english. 
purdue  .edu/handouts/research/r_evalsource  Jitml)  have 
established  extensive  websites  for  evaluating  resources. 
Similarly,  Trochim’s  (2004)  social  science  research 
website  provides  strategic  scaffolding  to  guide  the 
research  process. 

EPISTEMOLOGY,  FOUNDATIONS, 

AND  ASSUMPTIONS 

In  grounded  design  practice,  epistemology,  founda¬ 
tions,  and  assumptions  are  aligned  to  maximize  their 
coincidence.  Although  isolated  for  clarity  in  the  fol¬ 
lowing,  the  extent  to  which  activities  are  aligned  with 
the  associated  foundations  and  assumptions  deter¬ 
mines  how  (or  if)  a  given  learning  model  is  consistent 
with  its  espoused  epistemology. 

Epistemology 

We  have  advocated  that  learning  systems  be  aligned 
with  and  grounded  in  underlying  epistemological 
beliefs  and  associated  foundations  and  embody  the 
assumptions  and  practices  underlying  those  beliefs  and 
foundations  (Hannafin  and  Hill,  2006;  Hannafin  et  al., 
2004).  Objectivists’  values  and  beliefs  about  the  nature 
of  knowledge  and  learning,  for  example,  differ  funda¬ 
mentally  from  those  of  relativists  (Jonassen,  1991).  To 
ground  design  practices  that  reflect  and  manifest  the 
values  of  objectivist  epistemology,  corresponding 
design  foundations  and  practices  are  applied  to  support 
learning.  Conversely,  constructivists  emphasize  the 
uniquely  individual  construction  of  knowledge  and  the 
generation  of  meaning;  accordingly,  practices  are 
designed  to  support  unique  rather  than  particular  sense 
making  (Hannafin  et  al.,  1997).  In  effect,  grounded 
design  criteria  require  that  the  components,  strategies, 
and  activities  of  any  learning  environment  reflect  an 


alignment  of  associated  foundations  and  assumptions 
underlying  epistemological  beliefs.  The  unique  ways 
in  which  design  practices  are  enacted  vary  according 
to  the  epistemology,  foundations,  and  assumptions  con¬ 
sistent  with  and  extended  from  a  given  learning  model 
(for  a  discussion  of  grounded  design  practices  in  case- 
based  learning,  see  Kim  and  Hannafin,  in  press  a). 

Foundations  and  Assumptions 

Psychological 

Specific  instantiations  vary  based  on  epistemological 
differences,  but  the  importance  of  psychological  foun¬ 
dations  has  been  long  recognized  (Bednar  et  al.,  1995; 
Brown  and  Campione,  1996;  Gagne  and  Glaser,  1987). 
Beginning  in  the  early  20th  century,  associationism 
dominated  the  psychological  learning  landscape. 
Operant  behavioral  theory  and  research,  reflecting 
objectivist  epistemology  and  influenced  strongly  by 
Skinner’s  research  in  operant  conditioning,  posit  that 
learning  involves  an  enduring  change  in  behavior  reg¬ 
ulated  by  associations  between  a  stimulus,  a  response, 
and  a  reinforcer:  The  more  a  desired  behavior  is  rein¬ 
forced  in  the  presence  of  controlling  stimuli,  the 
greater  the  likelihood  of  repeating  the  response  and 
the  stronger  the  learning.  Characteristically,  behavior- 
ists  paid  little  attention  to  thinking  or  the  organization 
of  knowledge,  tending  instead  to  characterize  learning 
in  terms  of  complex  stimulus-response-reinforcement 
associations— -that  is,  observable  phenomena  deemed 
to  influence  learning  (Burton  et  al.,  2004). 

Like  behaviorism,  cognitive  psychology  is  largely 
rooted  in  objectivist  epistemology;  unlike  behaviorists, 
cognitive  psychologists  emphasize  the  individual’s 
processing  of  information  and  how  knowledge  is  stored 
and  retrieved  (Winn,  2004).  Cognitive  theorists,  for 
example,  posit  that  mental  processes  mediate  what  is 
selected,  remembered,  recalled,  and  generalized  based 
on  individual  background,  needs,  and  interests.  Infor¬ 
mation  processing  involves  an  exchange  between 
external  stimuli  and  internal  mental  processes  such  as 
sensory  registers,  selective  perception,  short-term 
memory  (STM),  and  long-term  memory  (LTM) 
(Shuell,  1986).  According  to  information-processing 
theory,  sensory  registers  filter  external  stimuli,  selec¬ 
tively  isolating  information  and  signals  to  be  processed 
more  deeply.  This  information  is  processed  temporarily 
in  working  memory,  where  it  is  further  filtered.  Rele¬ 
vant  information  is  encoded  in  permanent  memory,  and 
the  remaining  information  is  discarded.  Learning,  then, 
involves  the  transfer  of  information  from  working  to 
permanent  memory,  while  retrieval  involves  retrieving 
knowledge  from  permanent  to  working  memory.  The 
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richer  the  initial  and  updated  encoding,  the  more  likely 
knowledge  will  be  activated  and  retrieved  under  appro¬ 
priate  conditions,  retrieved  (decoded),  and  applied  or 
transferred  (Hannafin  et  al.,  2007). 

The  psychological  foundations  of  contemporary 
resource -based  environments,  such  as  inquiry-based 
learning  and  problem-based  learning,  are  often  rooted 
in  constructivist  epistemology.  Kim  and  Hannafin  (in 
press  b),  for  example,  described  numerous  examples 
of  situated,  constructivist-inspired  inquiry  learning. 
Situated  cognition  researchers  and  theorists  consider 
knowledge  and  context  to  be  inextricably  intertwined; 
that  is,  meaning  does  not  exist  independent  of  context, 
but  rather  context  shapes  and  defines  meaning  (Brown 
et  al.,  1989). 

Pedagogical 

Psychological  and  pedagogical  foundations  are  inter¬ 
dependent.  Direct  instruction,  for  example,  typically 
emphasizes  explicit  identification  of  objective  out¬ 
comes,  hierarchical  structures,  and  objective-based 
activities  and  assessment  consistent  with  objectivist 
epistemology  and  behavioral  psychology,  whereas  stu¬ 
dent-centered  pedagogical  approaches  tend  to  be  con¬ 
textually  based  and  epistemologically  aligned  with 
constructivism.  Taken  together,  they  reflect  underlying 
beliefs  about  the  nature  of  learning,  the  methods  and 
strategies  employed,  and  the  ways  in  which  domain 
information  is  organized  and  made  available. 

Technological 

Technological  capabilities  dictate  the  extent  to  which 
features  can  support  learning,  but  pedagogical  require¬ 
ments  dictate  how  and  which  capabilities  should  be 
integrated.  Technological  capabilities  vary  widely,  but 
it  is  the  manner  in  which  they  support  or  hamper 
efforts,  not  their  mere  availability,  that  influences 
learning.  Some  features  are  available  but  not  appropri¬ 
ate  given  the  pedagogical  requirements  of  a  given 
learning  environment,  while  others  may  be  desirable 
but  inherently  limited. 

Cultural 

Cultural  considerations  reflect  beliefs  about  education, 
the  role  of  individuals  in  society,  traditions  in  how 
different  disciplines  teach  and  learn,  and  the  prevailing 
practices  of  a  given  community.  They  influence  design 
by  defining  the  values  of  a  given  setting.  As  an  exam¬ 
ple,  “back  to  basics”  learning  cultures  tend  to  embrace 
objectivist  epistemology,  emphasize  behavioral  or  cog¬ 
nitive  psychological  foundations,  and  apply  direct 


instruction  pedagogy.  In  effect,  different  learning  mod¬ 
els  are  both  reflections  and  extensions  of  the  cultures 
for  which  they  are  designed. 

Pragmatic 

Each  setting  has  unique  situational  constraints  that 
affect  how  a  learning  model  is  implemented.  Collins 
(1996),  for  example,  described  competing  conse¬ 
quences  involved  in  making  practical  tradeoffs,  such 
as  determining  what  should  be  taught,  assessing  costs 
and  benefits,  and  evaluating  activity  alternatives. 
Issues  such  as  hardware  and  software  type  and  avail¬ 
ability  and  costs  routinely  influence  the  adoption  and 
diffusion  of  innovations.  They  establish,  from  a  prac¬ 
tical  perspective,  why  a  particular  approach  may  or 
may  not  be  feasible  in  a  given  learning  environment. 

RESOURCE-BASED 
LEARNING  RESEARCH 

As  noted  previously,  RBL-related  approaches  are  com¬ 
monly  reflected  in  research  on  different  learning  mod¬ 
els.  We  will  not  attempt  to  discuss  research  within 
different  learning  models;  instead,  we  examine 
research  involving  the  use  of  resources  where  no  for¬ 
mal  learning  model  was  specified  or  apparent.  Armatas 
et  al.  (2003)  explored  how  individual  characteristics 
(e.g.,  learning  goals,  study  strategies,  subject  interest, 
computer  attitudes)  influenced  learner  engagement 
with  course  resources  in  an  online  environment  involv¬ 
ing  731  on-  and  off-campus  students  enrolled  in  a  first- 
year  psychology  course  in  Australia.  A  learning  man¬ 
agement  system  (LMS)  linked  the  course  structures  to 
a  variety  of  prescribed  and  optional  resources,  ranging 
from  a  textbook  to  lectures  to  websites  to  discussion 
groups.  A  165-item  survey  that  assessed  study  habits, 
resources  accessed,  difficulties  accessing,  and  per¬ 
ceived  value  of  the  resources  was  distributed  toward 
the  end  of  the  term.  In  addition,  computer  attitudes, 
learning  goals,  study  strategies,  and  interest  in  psy¬ 
chology  were  assessed. 

Results  indicated  a  significant  influence  for  back¬ 
ground  factors,  engagement  with,  and  perceived  value 
of  RBL  resources.  Among  enrollees,  off-campus  stu¬ 
dents  reported  spending  more  time  studying  than  on- 
campus  students.  Off-campus  students  also  indicated 
a  preference  for  electronic  resources  and  reported 
fewer  problems  with  accessing  the  LMS  than  on-cam- 
pus  students  and  were  more  positive  toward  resource 
organization  and  usefulness.  Overall,  older  students 
(more  prevalent  among  off-campus  enrollees)  pre¬ 
ferred  working  with  the  computer  more  than  younger 
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students,  who  reported  less  confidence  in  study  strat¬ 
egies  than  older  students.  On-campus  students,  gener¬ 
ally  younger,  preferred  print-based  resources. 

Newnham  et  al.  (1998)  explored  the  use  of  Inter¬ 
net-based  resources  in  an  undergraduate  geography 
course.  The  learning  environment  was  largely  exter¬ 
nally  directed,  with  the  course  instructor  supplying 
extensive  structure  and  explicit  directions  (for  detailed 
discussions  of  directed  distance  learning  environ¬ 
ments,  or  DDLEs,  see  Sharma  et  al.,  2007).  Course 
resources  were  predetermined  by  the  instructor  and 
downloaded  to  a  local  network.  In  addition,  students 
were  encouraged  to  use  bulletin  boards  related  to  top¬ 
ics  under  study  (volcanic  and  earthquake  activity).  Stu¬ 
dent  performance  and  student  course  evaluations  were 
gathered  over  two  academic  years. 

Results  indicated  significant  improvements  in  stu¬ 
dent  performance  associated  with  externally  directed 
use  of  Internet-based  resources.  According  to  the 
researchers,  the  standard  of  academic  coursework  was 
higher  when  students  were  provided  directed  access  to 
Internet -based  resources— particularly  in  the  second 
year.  Student  perceptions  related  to  the  use  of  Internet 
resources  were  also  positive,  both  for  completing 
course  assignments  and  for  long-term  employability. 

Greene  and  Land’s  (2000)  qualitative  analysis  of 
the  use  of  resources  in  a  college-level  problem-focused 
RBL  environment  examined  how  learners  used 
resources  during  project-based  learning,  how  proce¬ 
dural  scaffolds  influenced  project  progress,  and  how 
interactions  (student-student,  student-instructor) 
shaped  project  quality.  Participants  were  1 8  undergrad¬ 
uate  preservice  teachers  (16  female,  2  male)  in  an  edu¬ 
cational  technology  course.  The  activity  involved  stu¬ 
dents  developing  a  project  that  integrated  Web  resources 
into  the  curriculum.  Participants  chose  their  own  topics 
(e.g.,  trip  planning  for  third  graders)  and  elected  to  work 
in  groups  of  two  to  four  members  or  independently. 
Data  collection  focused  on  surveys,  researcher  obser¬ 
vations,  videotaped  observations  of  Web  use,  and  tran¬ 
scriptions  of  student  verbalizations  during  Web  use. 

Results  indicated  that  the  ability  to  recognize  and 
integrate  specific  Web  resources  into  projects  was 
influenced  by  knowledge  of  the  Web,  the  topic  under 
investigation,  and  a  willingness  to  consider  alterna¬ 
tives.  Within-group  student-student  interactions  and 
instructor-student  interactions  were  reportedly  valued 
and  useful;  across-group  student-student  interactions 
were  successful  only  when  the  interaction  was  per¬ 
ceived  to  be  of  value.  The  dynamic  questions  and  dis¬ 
cussions  between  instructor  and  students  promoted 
reflection  on  why  students  used  Web  resources.  Greene 
and  Land  (2000)  concluded  that  the  contextualization 
and  scaffolding  supporting  resource  use  influenced  stu¬ 


dents’  perceived  and  actual  usability,  indicating  that 
learners  may  need  explicit  guidance  in  selecting 
resources  until  they  become  sufficiently  familiar  with 
their  topic  or  the  context.  They  also  noted  that  online 
information  lacks  adequate  indexing  to  support  educa¬ 
tional  purposes,  and  they  underscored  the  need  to  struc¬ 
ture  and  scaffold  both  search  and  use  of  Web  resources. 

Jelfs  et  al.  (2004)  explored  resource  “toolkits”  for 
students  in  a  directed,  online  learning  environment  at 
the  British  Open  University.  The  researchers  sought  to 
determine  which,  when,  and  how  to  provide  resources 
to  assist  distant  undergraduate  students  during  their 
online  learning.  Participants  included  60  part-time 
undergraduates,  and  data  included  questionnaires  and 
interviews.  The  questionnaires  focused  on  students’  per¬ 
ceptions  of  the  resource  scaffolding  toolkits.  Interviews 
were  conducted  by  phone  with  students  throughout  the 
United  Kingdom  (England,  Wales,  Scotland,  and  North¬ 
ern  Ireland)  to  supplement  questionnaire  responses. 

Overall,  although  the  resource  toolkits  were  effec¬ 
tive  in  scaffolding  resource  use,  students  requested 
alternative  formats  (e.g.,  print,  people)  and  the  ability 
to  access  resources  when  needed.  Some  students  pre¬ 
ferred  receiving  the  toolkits  in  print  format,  and  others 
preferred  retrieving  materials  from  the  Internet.  Gener¬ 
ally,  students  indicated  a  preference  for  fewer  activities 
and  for  aesthetically  attractive  resources  in  the  toolkits. 

Macdonald  et  al.  (2001)  examined  resource  use  in 
both  an  undergraduate  and  a  graduate  course  taught 
over  a  3-year  period  at  the  British  Open  University. 
Both  courses  included  a  variety  of  resources  (elec¬ 
tronic,  human,  Web-based)  in  multiple  formats  (print, 
video,  animations)  to  support  the  learning  process.  The 
researchers  examined  the  influence  of  information¬ 
handling  skills  and  factors  that  influence  the  acquisi¬ 
tion  of  those  skills.  Data  sources  included  computer 
conferences,  observations,  interviews,  and  e-mail 
reflective  follow-up;  questionnaires  were  used  to 
gather  quantitative  data.  Undergraduates  indicated  that 
they  enjoyed  the  resource-based  learning  approach; 
however,  both  time  constraints  for  using  the  resources 
and  information  overload  were  cited  as  problems.  Par¬ 
ticipants  indicated  that  access  to  resources  (primarily 
via  CD-ROM)  was  relatively  easy,  although  the  skills 
needed  to  analyze  and  evaluate  the  information  were 
not  sufficiently  developed. 

Graduate  participants  also  indicated  that  they 
favored  the  resource-based  approach,  but  the  research¬ 
ers  noted  that  they  may  have  had  greater  prior  experi¬ 
ence  and  confidence  in  using  multiple  resources.  Some 
participants  indicated  that  RBL  may  be  more  applica¬ 
ble  in  some  areas  than  in  others  (e.g.,  engineering)  and 
that  resource-based  approaches  may  be  more  appro¬ 
priate  for  upper-division  undergraduate  and  graduate 
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level  study.  The  authors  concluded  that:  (1)  a  variety 
of  skills  is  needed  to  effectively  use  resources  (e.g.,  IT 
skills,  cognitive  skills);  (2)  course  complexity  should 
inform  how  many  and  what  resources  are  introduced; 
and  (3)  opportunities  are  needed  for  students  to  under¬ 
stand  the  benefits  (as  well  as  challenges)  of  RBL. 

Although  published  research  typically  focuses  on 
postsecondary  environments,  a  few  researchers  have 
examined  the  use  of  resource-based  approaches  in  ele¬ 
mentary  and  secondary  settings.  In  a  seventh-grade 
project,  “Children,  Access,  and  Learning,”  McNichol 
et  al.  (2002)  examined  the  RBL  approaches  of  four 
U.K.  schools.  One  class  was  selected  from  each 
school,  and  students  were  required  to  use  information 
and  communication  technology  (ICT)  resources  (e.g., 
Internet,  CD-ROM)  at  school,  at  home,  or  in  the  com¬ 
munity  (e.g.,  public  library).  Participants  represented 
a  range  of  socioeconomic  backgrounds  (low  socioeco¬ 
nomic  to  prosperous)  and  environmental  settings 
(semi-rural,  semi-urban,  inner-city,  rural) .  Student  logs 
recorded  resource  use,  summary  logs  documented 
reflection  on  the  process,  and  individual  follow-up 
interviews  obtained  student  perceptions.  Parent  or 
guardian  questionnaires  and  interviews  provided  inde¬ 
pendent  data  related  to  home  resource  usage. 

Results  indicated  that  ICT  resource  use  in  the  home 
was  high,  even  in  poorer  regions  of  the  country  (93%). 
While  access  varied  (individual  machine  to  a  single 
shared  computer  in  the  house),  the  majority  of  the 
students  had  access  to  and  made  use  of  ICT  resources. 
The  perceived  value  of  ICT  resources  differed  between 
students  and  parents.  Students  expressed  concern  over 
the  limited  number  of  resources  available,  while  par¬ 
ents  expressed  concerns  related  to  laziness  (i.e .,  too 
easy  to  find  information),  inadequate  information  lit¬ 
eracy,  access  to  inappropriate  information,  the  cost  of 
Internet  access,  the  use  of  chatrooms,  and  inappropri¬ 
ate  use  of  the  ICT  resources. 

School-based  ICT  access  varied  widely  from  loca¬ 
tion  to  location.  At  one  school,  most  resources  were 
accessed  in  the  classroom,  library,  and  at  home. 
Another  school  provided  very  limited  and  highly  cen¬ 
tralized  access  to  ICT  resources,  housing  most  com¬ 
puters  in  the  library.  As  a  result  of  limited  access, 
resources  use  was  limited  and  the  project  teacher 
developed  few  skills. 


ISSUES  AND  IMPLICATIONS 

One  area  of  particular  interest  to  RBL  is  literacy.  As 
we  have  noted  throughout  the  chapter,  literacy  concerns 
related  to  the  multiple  types  and  forms  of  resources 
include  the  rapid  but  largely  unmanaged  growth  of 


resources ,  the  presence  of  erroneous  or  misinformation , 
and  the  use  of  resources  as  propaganda.  The  largely 
unregulated  generation,  distribution,  access,  and  use  of 
digital  resources  pose  both  significant  opportunities 
and  challenges  for  resource-based  learning. 

Given  the  challenges  associated  with  the  identifi¬ 
cation  of  reliable  and  valid  resources,  developing  lit¬ 
eracy  across  media  and  multiple  ways  of  knowing  have 
become  increasingly  important  (Mackey,  2002).  Infor¬ 
mation  literacy  has  a  long  history  with  both  traditional 
text  and  electronic  contexts.  Barnard  et  al.  (2005,  p. 
509)  noted:  “Individuals  who  are  information  literate 
can  determine  the  extent  of  information  needed,  access 
information  effectively,  and  critically  evaluate  infor¬ 
mation  and  its  sources.”  Researchers  in  the  United 
States  and  Western  Europe  have  underscored  the  grow¬ 
ing  significance  of  information  literacy;  for  example, 
in  the  study  by  McNichol  et  al.  (2002),  one  participant 
stated:  “They  use  [information  literacy]  in  schools  and 
businesses  and  this  is  the  way  things  are  going,  so  they 
need  to  have  a  good  knowledge  of  computers.  It’s  the 
future;  if  they  don’t  know,  they’re  stuck”  (p.  399). 
Other  researchers  have  advocated  an  expansion  of  our 
definitions  and  understandings  of  what  it  means  to  be 
literate  in  the  information  age.  Mackey  (2002) 
described  the  “ecologies  of  literacy”  in  a  1 .5-year-long 
study  exploring  young  students  (age  10  and  13)  using 
multiple  resources.  Huot  et  al.  (2004),  along  with  other 
scholars  writing  in  their  edited  book,  advocated  that 
“literacy  is  not  one  thing  and  that  time  changes  what 
we  mean  by  literacy”  (p.  1). 

Despite  widespread  agreement  as  to  the  impor¬ 
tance  of  information  literacy,  little  is  known  about  how 
recontextualized  resources  (as  the  raw  materials  of  the 
information  age)  influence  interpretation,  meaning, 
and  understanding  during  RBL.  In  externally  imposed 
contexts,  the  range  of  intended  interpretations  and 
meaning  is  largely  constrained  by  the  tasks,  activities, 
and  goals  to  which  their  use  is  directed.  The  objectiv- 
ist-inspired  learning  models  typically  convey  common 
rather  than  unique  meaning;  however,  in  individual  and 
negotiated  learning  models  where  individual  selection 
and  interpretation  of  a  resource’s  meaning  are 
expected,  considerable  variability  has  been  reported  in 
the  resources  used  and  their  meaning. 

Resource-based  learning  components  are  designed 
to  instantiate  individual  resource  selection  and  mean¬ 
ing  making  by  providing  the  contexts  and  tools  to 
explore  and  refine  understanding  and  the  scaffolds  to 
guide  and  support  reasoning.  In  some  cases,  unique 
sets  of  resources,  tools,  and  scaffolds  have  been  devel¬ 
oped  for  specific  learning  models;  see,  for  example, 
the  description  by  Linn  et  al.  (2003)  of  the  Web-Based 
Inquiry  Science  Environment  (WISE)  program,  as  well 
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as  the  summary  of  Jasper  s  affordances  by  the  Cogni¬ 
tion  and  Technology  Group  at  Vanderbilt  (1992).  In 
others,  however,  existing  resources  are  recontextual¬ 
ized  by  the  individual.  Little  evidence  exists  docu¬ 
menting  their  effectiveness  in  different,  often  distal, 
learning  contexts.  We  need  to  further  explore  how  lit¬ 
eracy  affects  the  use  and  interpretation  of  resources 
within  these  contexts. 

Likewise,  a  great  deal  of  literature  has  been  pub¬ 
lished  on  orienting  individuals  to  varied  learning  goals, 
activities  that  engage  individuals  with  learning  con¬ 
tent,  and  guiding  performance.  Although  this  research 
provides  some  guidance  for  resource  use,  the  episte¬ 
mology,  foundations,  and  assumptions  embodied  in  the 
research  often  differ  from  those  embodied  in  specific 
learning  models.  Findings  related  to  providing  pre- 
questions  may  prove  to  be  of  value  in  designing 
directed  learning  models  but  of  little  utility  in  enabling 
contexts  designed  to  induce  unique  learning  goals.  In 
addition,  comparatively  few  researchers  have  exam¬ 
ined  interactions  among  contexts,  tools,  and  scaffolds 
in  learner-centered  models.  So,  although  a  great  deal 
of  related  research  has  been  published,  their  relevance 
to  learning  models  reflecting  fundamentally  different 
epistemologies  has  not  been  documented  conclusively. 

Finally,  Wellington  (2001)  identified  a  host  of 
issues  related  to  resource  use,  including  teacher  con¬ 
trol,  worries  about  plagiarism,  and  perceived  threats  to 
teacher  authority.  Clearly,  increased  availability  and 
quality  have  not  tempered  educators’  concerns  regard¬ 
ing  the  risks  associated  with  everyday  resource  use. 
Significant  barriers  derive  more  from  teaching-learn¬ 
ing  traditions  than  concerns  over  the  inherent  vs.  mal¬ 
leable  properties  of  a  resource.  These  barriers  are 
deeply  engrained  in  education  culture  and  may  prove 
more  formidable  than  quality,  interpretive,  and  techni¬ 
cal  issues  related  to  recontextualization. 
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ABSTRACT 

This  chapter  seeks  to  investigate  grounds  for  the  appli¬ 
cation  and  adaptation  of  general  instructional  models 
in  specific  domains  and  professions  and  for  the  gen¬ 
eration  of  domain-specific  instructional  methods  using 
the  examples  of  reading,  mathematics,  science,  law, 
and  medicine.  It  does  so  by  first  investigating  the  spe¬ 
cific  teaching  and  learning  difficulties  of  those 
domains  by  showing  and  analyzing  domain-  and  pro¬ 
fession-specific  solutions  to  these  problems.  The  chap¬ 
ter  consists  of  a  general  analysis  of  the  problems  with 
general  instructional  design  models  that  have  to  be 
solved  before  they  can  be  used  in  specific  domains, 
the  requirements  for  the  use  of  these  models,  and  the 
effects  of  internal  and  external  pressures  for  change. 
The  chapter  is  completed  by  five  contributions  from 
these  domains. 

KEYWORDS 

Abstract  mathematical  knowledge:  Mathematics  as  a 
body  of  knowledge  that  may  be  experienced  as 
experientially  real. 

Apprenticeship:  Learning  a  skilled  practical  trade 
through  participation  in  practice. 

Clinical  education:  Supervised  acquisition  of  clinical 
skills. 

Conceptual  model:  A  theoretically  based  explanation 
of  a  natural  phenomenon. 

Constructivism:  A  point  of  view  related  to  cognitive 
development;  cognitive  development  depends 
upon  a  child’s  actions  on  his  own  surroundings 
to  change  and  learn  actively  from  those  surround¬ 
ings  (Piaget)  and  on  the  strong  relationship 
between  a  child’s  surroundings  and  his  cognitive 
development  (Vygotsky). 

Discovery  learning  environment:  Environment  that 
provides  students  with  a  set  of  events  or  data  that 
they  can  explore  independently  or  in  a  prestruc¬ 
tured  way  to  discover  the  regularities  in  these  data; 
a  rich  learning  environment  that  facilitates  chil¬ 
dren’s  own  constructive  endeavors  to  learning 
experiences. 

Domain-specific  theories:  Theories  on  instruction  that 
are  tailored  to  a  specific  subject-matter  domain. 
Embodiments:  Tactile  or  visual  models  that  are  meant 
to  represent  mathematical  relationships  and  con¬ 
cepts  in  a  ready  apprehensible  form. 

Emergent  modeling:  A  dynamic  approach  to  modeling 
in  mathematics  education,  in  which  the  models  that 
the  students  use  develop  from  models  of  informal 
mathematical  activity  into  models  for  more  formal 
mathematical  reasoning. 


Expertise  development:  Acquisition  of  knowledge  and 
skill  in  a  domain. 

Gaming:  Taking  part  in  open-ended  activities  that  give 
opportunity  to  elicit  conceptual  learning. 
Ontology:  A  shared  and  agreed  upon  explicit  formal 
representation  of  a  domain. 

Problem-based  learning:  Acquiring  knowledge  as  part 
of  a  learner  group  by  analyzing  a  problem,  studying 
privately,  using  various  learning  resources,  and  col¬ 
lectively  synthesizing  knowledge. 

Reasoning:  Formulating  and  substantiating  conclu¬ 
sions  based  on  given  information. 

Science  learning:  Understanding  concepts  related  to 
biology,  physics,  chemistry,  and  astronomy. 

INTRODUCTION 

Henny  P.  A.  Boshuizen 

The  individual  examples  discussed  in  this  chapter  give 
the  reader  a  view  of  how  instructional-design  models 
are  developed,  adapted,  and  used  in  specific  domains. 
These  examples  are  from  diverse  fields:  reading,  math¬ 
ematics,  science,  law,  and  medicine.  They  cover  both 
initial  and  advanced  learning;  they  include  knowledge 
and  skills  that  everyone  in  Western  societies  are  sup¬ 
posed  to  learn,  as  well  as  fields  that  are  only  taught  at 
the  university  level;  and  they  show  a  variety  of  meth¬ 
ods  and  models  that  are  being  used  in  these  domains. 
In  this  chapter,  we  move  away  from  research  into  the 
models  themselves  and  enter  the  world  where  instruc¬ 
tional  designers,  pedagogues,  and  teachers  work 
together  to  realize  adequate  instruction  in  a  particular 
domain.  The  worlds  of  research  and  professional  prac¬ 
tice  have  different  commitments,  confront  different 
problems,  and  work  with  different  assumptions,  prin¬ 
ciples,  and  expectations.  Still,  they  build  on  each  other. 
We  will  analyze  the  reasons  for  these  differences  in 
different  domains  and  highlight  problems  that  affect 
transfer  of  the  insights  and  methods  that  have  been 
described  in  previous  chapters. 

Different  Commitments 

There  are  differences  in  focus  and  in  research  strategy 
between  investigators  and  developers  of  instructional 
models  on  the  one  hand  and  domain-specific  instruc¬ 
tional  designers  on  the  other.  Instructional  designers 
and  researchers  whose  first  concern  is  to  investigate 
and  improve  instructional  models  choose  their  topics 
and  domains  in  such  a  way  that  the  effectiveness  of 
these  models  can  be  best  examined.  Discovery  learning, 
for  example,  is  primarily  investigated  using  specific 
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topics  or  laws  in  science  (such  as  optical  laws  or  elec¬ 
tronics).  Discovery  learning  environments  provide  stu¬ 
dents  with  a  set  of  events  or  data  that  they  can  explore 
independently  or  in  a  prestructured  way  to  discover 
the  regularities  in  these  data.  Discovering  these  regu¬ 
larities  is  easiest  when  the  relations  under  investigation 
are  rather  straightforward.  From  a  research  perspec¬ 
tive,  this  is  a  wise  approach  that  optimizes  the  chances 
for  success.  A  particular  domain  and  context  provide 
a  suitable  venue  for  testing  specific  teaching  methods 
while  allowing  variations  in  the  model  or  the  method 
that  are  required  to  test  hypotheses  about  effectiveness 
and  learning  outcomes.  Choosing  the  correct  domain 
is  a  crucial  part  of  a  successful  research  project;  how¬ 
ever,  the  link  between  model  or  theory  and  the  domain 
in  which  it  is  investigated  has  drawbacks.  Investigators 
may  not  build  up  systematic  knowledge  of  situations 
or  parts  of  the  domain  where  their  methods  and  models 
do  not  apply. 

The  situation  is  different  for  instructional  designers 
and  teachers  in  specific  domains  and  professions.  For 
them,  the  specific  domain  comes  first;  they  cannot 
select  specific  parts  and  leave  out  others.  Their  primary 
commitment  is  to  solve  specific  teaching  and  learning 
problems.  In  this  process,  they  may  use  and  often  adapt 
generic  models  and  methods  developed  by  researchers. 
Domain-specific  instructional  research  has  produced 
models  that  have  later  proven  more  generally  applica¬ 
ble.  An  example  is  problem-based  learning  (PBL),  first 
developed  in  medical  education  (Barrows,  1984)  but 
now  used  in  domains  as  diverse  as  engineering  and 
economics.  Emergent  modeling,  a  method  developed 
to  teach  mathematics  and  described  in  this  chapter,  may 
prove  to  be  a  significant  contribution  to  other  domains. 

Domain-Specific  Problems 

Using  and  adapting  models  and  methods  are  not  with¬ 
out  problems.  Many  of  these  problems  have  to  do  with 
learning,  but  there  are  more.  First,  learning  problems 
can  be  inherent  to  a  domain.  Much  domain-specific 
instructional  research  is  meant  to  uncover  domain- 
specific  learning  challenges  and  determine  the  impli¬ 
cations  for  instructional  design;  for  example,  research¬ 
ers  in  medicine  found  out  that  the  integration  of  dif¬ 
ferent  topic  areas  is  a  problem  for  students  following 
a  traditional  medical  curriculum.  Often  these  topics 
are  taught  at  different  times  by  teachers  who  have  little 
knowledge  about  what  other  teachers  do  and  who 
emphasize  the  principles  and  idiosyncrasies  of  their 
own  topic  area  rather  than  focus  on  an  integrated  appli¬ 
cation  of  knowledge  in  the  context  of  clinical  prob¬ 
lems.  Even  in  PBL  approaches,  integration  has  to  be 
explicitly  designed  into  the  curriculum. 


Problems  and  challenges  in  mathematics  are  dif¬ 
ferent.  Gravemeijer  (this  chapter)  shows  that  the 
abstract  view  of  mathematical  knowledge  that  perme¬ 
ates  mathematical  teaching  and  textbooks  makes  it 
very  difficult  for  high-school  students  to  understand 
and  apply  mathematics.  Gravemeijer  and  colleagues 
have  resolved  this  difficulty  by  beginning  with  practi¬ 
cal  mathematical  problems  and  gradually  working 
toward  higher  levels  of  abstraction,  building  deeper 
insights  based  on  naive,  procedural  applications  and 
directed  knowledge  and  skills. 

The  problem  with  learning  law  has  unique  aspects 
(Nievelstein  and  colleagues,  this  chapter).  The  legal 
profession  has  explicitly  defined  concepts.  These  def¬ 
initions  only  partially  overlap  with  everyday  use.  The 
legal  definitions  cannot  easily  be  changed.  This  strict 
ontology  is  something  students  must  learn.  Another 
problem  is  the  adversarial  argumentation  style. 
Whereas  medical  education  has  a  long  tradition  in 
instructional  design  research,  legal  education  has 
been  slow  to  embrace  instructional  design,  partly 
because  lawyers  are  used  to  considering  nearly  every¬ 
thing  as  contentious  or  arguable. 

We  present  several  examples  of  problems  and  how 
domain-specific  instructional  designers  try  to  deal  with 
them.  Research  in  mathematics  education  has  been 
most  successful  in  terms  of  instructional  procedures  and 
methods.  The  strict  ontology  in  law  is  at  odds  with  the 
everyday  use  of  the  same  words  that  students  are  accus¬ 
tomed  to  using.  Learning  difficulties  in  science  may  be 
linked  to  differences  in  parental  style  and  verbalizations 
used  at  home.  A  better  understanding  of  these  and  other 
difficulties  is  being  sought,  so  instructional  designers 
can  directly  address  domain-specific  difficulties. 

A  major  instructional  problem  is  the  effect  of 
socioeconomic  status  (SES).  Several  of  the  examples 
presented  in  this  chapter  report  on  this  factor  in  several 
domains;  however,  without  insight  into  how  children 
and  parents  from  high-  and  low-status  groups  might 
differ  in  terms  of  relevant  experience,  knowledge, 
skills,  values,  and  expectations,  SES  will  remain  a 
formidable  barrier. 

A  third  problem  area  concerns  the  timing  and 
sequencing  of  the  constituent  parts  of  the  educational 
process.  Instructional  measures  should  be  adapted  to 
the  level  of  mastery  of  the  learner.  Learning  new 
knowledge  and  skills  builds  on  previous  learning.  Fail¬ 
ure  to  help  learners  establish  meaningful  links  to  exist¬ 
ing  knowledge  typically  leads  to  problems. 

These  problems  relate  to  optimizing  instruction  in 
a  domain  to  support  the  acquisition  of  knowledge  and 
skill.  There  are  other  influences  and  problems  to  con¬ 
sider,  including  the  stakeholders,  the  political  climate, 
and  developments  within  a  profession  and  in  society 
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in  general.  The  contribution  on  reading  in  this  chapter 
shows  how  political  change  affects  research.  It  also 
shows  that  choices  made  by  external  bodies  can  shape 
research  agendas  and  strategies.  Medicine  provides 
another  example  of  external  influences  on  the  content 
of  the  curriculum  and  the  methods  and  models  applied 
or  under  development. 

Problems  with  Models 

Models  do  not  automatically  or  perfectly  fit  specific 
domains,  topics,  skills,  or  competencies.  Transfer  from 
one  domain  to  another  will  almost  always  require 
adaptations.  Instructional  models  must  be  reinvented 
or  significantly  modified  for  a  particular  domain.  None 
of  the  authors  of  the  coming  domain  contributions 
raises  this  point.  An  interesting  example  of  this  phe¬ 
nomenon  occurred  at  Maastricht  University  in  the 
1980s.  At  that  time,  the  university  was  still  under 
development  and  consisted  of  two  faculties,  medicine 
and  health  sciences,  with  two  new  ones  planned.  The 
university  constitution  prescribed  an  innovative,  PBL 
curriculum.  This  had  been  a  political  decision;  the 
university  would  never  have  come  into  existence  with¬ 
out  this  commitment.  In  hindsight,  designers  and 
teachers  could  have  foretold  the  problems  that  would 
emerge  when  they  tried  to  get  the  two  new  faculties, 
law  and  economics,  to  embrace  PBL.  Two  decades 
later,  Nievelstein  and  colleagues  (this  chapter)  can 
show  that  some  factors  that  impede  learning  law  and 
that  affect  adapting  PBL  to  law  education  are  still 
under  investigation. 

One  reason  why  adopting  PBL  is  so  slow  is  related 
to  the  fact  that  introducing  PBL  methods  and  models 
to  completely  change  a  curriculum  creates  a  significant 
departure  from  a  traditional,  teacher-centric,  highly 
structured  curriculum.  In  PBL  curricula,  the  successful 
teacher  not  only  communicates  knowledge  to  students 
but  also  diagnoses  individual  problems  and  adjusts 
methods  accordingly.  Much  of  the  required  instruc¬ 
tional  knowledge  in  such  cases  is  tacit.  Making  this 
tacit  working  knowledge  explicit  might  be  necessary 
to  overcome  this  difficulty  with  the  use  and  implemen¬ 
tation  of  PBL  instructional  models  in  the  domain  of 
law  education. 

Another  problem  with  instructional  models  is 
related  to  the  representations  and  artifacts  used.  Three 
of  the  subsequent  examples  reveal  that  the  use  of  unfa¬ 
miliar  or  dissimilar  representations  is  part  of  the  prob¬ 
lem  (children  in  science  education  seem  to  grasp  the 
conceptual  meaning  behind  artifacts  but  are  unable  to 
express  that  verbally),  but  such  variations  in  represen¬ 
tations  and  artifacts  can  also  be  part  of  the  solution. 
Phythian-Sence  and  Wagner  (this  chapter)  refer  to 


studies  that  show  that  multiple,  transient,  informal  rep¬ 
resentations  can  help  illiterate  students  bridge  the  gap 
between  phonological  decoding  of  words  and  the  for¬ 
mal  representation  of  English  spelling.  Similarly, 
Gravemeijer  (this  chapter)  deliberately  uses  several 
different  formal  and  informal  representations  familiar 
to  students  before  introducing  the  canonical  mathemat¬ 
ical  representation.  This  suggests  that  representations 
should  be  an  integral  part  of  an  educational  design. 

A  third  problem  revealed  in  the  examples  below  is 
a  consequence  of  dividing  a  curriculum  into  stages, 
such  as  junior  and  senior  high  school,  bachelor  and 
graduate  education,  and  preclinical  and  clinical  peri¬ 
ods.  These  different  stages  often  have  different 
focuses.  Roughly  speaking,  generic  goals  (such  as  in 
liberal  arts  and  sciences  or  general  skills  programs) 
precede  specific,  profession-oriented  goals.  It  seems 
obvious  that  methods  and  models  should  be  adapted 
to  these  different  goals. 

Requirements  for  Domain-Specific 
Use  of  Instructional  Models 

The  general  trend  in  society  toward  more  transparency 
and  accountability  has  also  permeated  the  educational 
field.  The  international  Programme  for  International 
Student  Assessment  (PISA;  http://www.pisa.oecd.org) 
studies— begun  in  2000  and  including  more  than  40 
countries  — are  an  example  of  this  trend.  The  PISA 
studies  investigate  how  well  15-year-old  students  have 
reached  educational  goals  in  reading,  mathematical, 
and  scientific  literacy.  The  tests  cover  not  only  mastery 
of  school  curricula  but  also  important  knowledge  and 
skills  needed  in  adult  life  (PISA,  2006).  Other  expres¬ 
sions  of  this  trend  are  the  call  for  evidence-based  edu¬ 
cation  in  the  United  States  or  the  treaties  in  the  Euro¬ 
pean  Union  on  accreditation  of  higher  education  and 
student  exchange  that  require  more  commonality  in 
standards  and  criteria. 

The  call  for  accountability  and  transparency 
involves  different  aspects  of  education.  In  terms  of 
goals,  the  questions  raised  are  whether  education 
reaches  the  desired  goals  and  whether  these  goals  are 
the  right  ones  in  the  view  of  diverse  stakeholders.  In 
terms  of  methods ,  the  question  is  whether  the  means 
used  to  reach  goals  are  effective  and  efficient.  The  call 
for  evidence-based  education  is  an  expression  of  the 
latter  concern.  Phythian-Sence  and  Wagner  (this  chap¬ 
ter)  show  that  reading  research  in  the  United  States  is 
dominated  by  this  concern.  They  also  suggest  that  the 
investigation  of  important  questions  has  come  to  a 
standstill  due  to  the  debate  on  allowable  and  acceptable 
methods.  Similar  problems  have  been  reported  from 
the  domain  of  mathematics  education. 
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Medicine  struggles  with  a  different  problem.  Dor- 
nan  and  colleagues  (this  chapter)  show  that  the  plea 
for  accountability  in  medical  education  mainly  con¬ 
cerns  the  content  of  the  curriculum  and  the  choice  of 
appropriate  methods  as  for  the  type  of  desired  out¬ 
come-knowledge,  skills,  competencies,  attitudes. 
The  methods  themselves  are  not  under  debate. 

Pressures  for  Change 

What  are  the  perils  of  exerting  so  much  pressure  on 
education  to  evaluate  and  change  its  methods  and 
content?  Societal  pressure  and  pressure  from  profes¬ 
sional  organizations  tend  to  focus  on  performance  and 
the  discrepancies  between  graduates’  knowledge  and 
skills  and  the  requirements  of  the  jobs  for  which  they 
have  been  trained.  Frustration  about  existing  practices 
further  complicates  the  pressure  for  changing  profes¬ 
sional  education.  The  question  is  whether  it  is  always 
wise  to  dramatically  change  practice  and  professional 
preparation,  even  assuming  that  job  and  task  analyses 
have  been  done  adequately.  At  least  three  aspects 
should  be  considered  before  deciding  that  a  change 
in  curriculum  content  and  methods  is  a  good  solution. 
First,  should  a  problem  with  the  performance  of  grad¬ 
uates  be  addressed  during  the  initial  phase  of  educa¬ 
tion  or  is  the  workplace  or  continuing  education  pre¬ 
ferred,  or  perhaps  a  combination?  Different  schools 
and  professions  provide  different  answers.  Second, 
new  entrants  in  a  professional  field  are  not  likely 
change  agents.  They  do  not  typically  have  sufficient 
influence  to  assume  that  role,  although  they  may  serve 
as  a  source  of  new  knowledge  and  skills.  If  standing 
practices  have  to  be  changed,  students  can  be  prepared 
for  it  by  a  timely  adaptation  of  training;  however, 
changes  in  practices  must  be  initiated  and  imple¬ 
mented  by  those  who  are  accountable  for  these  prac¬ 
tices,  such  as  the  heads  of  departments.  Finally,  pleas 
for  other  content  tend  to  overload  the  curriculum,  not 
only  with  more  content  but  also  with  more  diversity 
in  learning  methods  and  places  as  well  as  in  episte¬ 
mologies,  as  Dornan  and  colleagues  (this  chapter) 
make  evident.  The  question  becomes  one  of  how 
much  diversity  students  and  teachers  can  stand  with¬ 
out  ending  up  confused. 

The  use  of  instructional  methods  and  models  in 
domains  and  professions  is  affected  by  many  factors 
that  are  internally  and  externally  related  to  the  instruc¬ 
tional  design  process  itself.  Both  types  of  factors  must 
be  carefully  deliberated.  Especially  in  case  of  major 
innovations,  instructional  design  questions  are  affected 
by  curriculum  design  questions.  The  next  five  sections 
of  this  chapter  address  these  issues  within  the  context 
of  five  different  domains. 


LEARNING  TO  READ:  AN 
ORIENTATION  OF  WHAT  WORKS* 

Caroline  Phythian-Sence  and  Richard  K.  Wagner 

Introduction  to  Learning  to  Read 

Experimental  research  on  reading  appears  to  have 
begun  at  Wundt’s  laboratory  in  the  late  1800s.  This 
work  resulted  in  Huey’s  The  Psychology  and  Pedagogy 
of  Reading  in  1908  (Rayner  and  Pollatsek,  1989). 
From  the  beginning,  researchers  were  as  concerned 
about  pedagogy— instructional  practices  that  made  it 
possible  to  learn  to  read— as  they  were  about  the  per¬ 
ceptual  and  cognitive  processes  upon  which  reading 
depends.  Although  interest  in  reading  instruction  has 
waxed  and  waned  over  the  years,  recent  decades  have 
witnessed  a  huge  upswing  in  interest  and  activity  moti¬ 
vated  by  the  desire  to  improve  reading  performance. 

Despite  the  fact  that  a  number  of  children  learn  to 
read  with  seeming  ease,  for  many  children  learning  to 
read  is  a  difficult,  trying,  and  stressful  ordeal.  In  the 
United  States,  the  most  recently  published  Nation’s 
Report  Card  (Perie  et  al.,  2005)  finds  that  in  a  repre¬ 
sentative  sample  of  fourth  and  eighth  graders,  less  than 
a  third  of  students  were  reading  English  at  or  above 
the  proficient  level.  More  striking  still  are  the  results 
for  poor  and  minority  children.  For  fourth  and  eighth 
graders  eligible  to  receive  free  or  reduced  lunches ,  only 
16%  were  reading  at  or  above  proficient  level,  with 
only  46%  reading  at  or  above  basic  level.  Findings  for 
minority  children  are  equally  bleak.  Whereas  41%  of 
Caucasian  children  read  at  or  above  proficient  level, 
with  76%  at  or  above  basic  level,  only  13%  of  black 
children  and  16%  of  Hispanic  children  were  reading 
at  proficient  level,  with  42%  and  46%,  respectively, 
reading  at  or  above  basic  level. 

We  are  still  struggling  with  how  to  change  poor 
readers’  outcomes.  Longitudinal  evidence  suggests 
that  many  children  who  fail  to  acquire  adequate  read¬ 
ing  skills  in  early  elementary  school  will  never 
acquire  the  skills  necessary  to  become  proficient 
readers  (Lonigan,  2006).  Although  historical  efforts 
to  improve  reading  ability  focused  on  holistic 
approaches  involving  overall  cognitive  development, 
current  efforts  have  honed  in  on  identifying  specific 
practices  that  will  directly  improve  reading  outcomes. 
With  the  passage  of  the  No  Child  Left  Behind 
(NLCB)  Act  in  the  United  States  and  its  academic 
cornerstone,  Reading  First ,  the  demand  for  scientifi¬ 
cally  based,  high-quality  and  effective  reading 
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instruction  has  increased  significantly.  Consequently, 
the  recent  resurgence  in  educational  research  has 
sought  to  explicate  better  the  underlying  processes 
involved  in  learning  to  read,  thereby  identifying  prac¬ 
tices  that  are  most  effective  in  teaching  children  to  read. 

Earlier  Efforts:  Compensatory  Education 

It  has  long  been  recognized  that  children  from  impov¬ 
erished  or  minority  families  are  at  risk  for  academic 
failure  (Ramey  et  al.,  1983),  particularly  with  regard 
to  becoming  proficient  readers  (Rauh  et  al.,  2003).  In 
the  mid- 19th  century,  religious  and  philanthropic 
groups  provided  daycare  to  children  of  immigrant 
workers  founded  on  the  presumption  that  the  typical 
poor  or  immigrant  family  was  not  able  to  adequately 
provide  for  their  young  children  (Ramey  and  Camp¬ 
bell,  1984).  These  same  influences  were  seen  again 
during  the  compensatory  education  movement, 
inspired  by  the  civil  rights  movement  of  the  1960s  and 
1970s  and  seeking  to  compensate  for  social  and  eco¬ 
nomic  inequalities  (Kagan,  2002).  Bom  of  the  move¬ 
ment  were  preschool  programs  such  as  the  Perry  Pre¬ 
school  Project  and  Project  Head  Start.  It  was  hoped 
that  these  early  childcare  settings  would  replicate  the 
environments  that  more  affluent  young  children  expe¬ 
rienced  before  coming  to  school,  thus  providing  the 
disadvantaged  children  with,  among  other  things, 
social,  language,  and  early  literacy  skills. 

Although  follow-up  data  of  children  enrolled  in 
Head  Start  or  similar  programs  suggest  that  early  inter¬ 
vention  influences  children  in  educationally  meaning¬ 
ful  ways  (Hebbeler,  1985),  these  effects  all  but  disap¬ 
pear  by  the  time  children  reach  the  middle  elementary 
school  years  (Lee  et  al.,  1990).  Children  at  risk  for 
academic  failure  (and,  specifically,  reading  failure) 
need  more  than  an  enriching  preschool  environment 
to  develop  the  skills  necessary  to  become  proficient 
readers.  Children  at  risk  for  reading  failure  need  rig¬ 
orous  and  consistent  support  throughout  elementary 
school  to  acquire  the  skills  necessary  to  become  pro¬ 
ficient  readers. 

Current  Efforts:  National  Reading 
Panel,  NCLB,  and  Reading  First 

In  response  to  the  National  Assessment  of  Educational 
progress  report  of  1996,  which  showed  that  overall 
40%  of  fourth  graders  in  the  United  States  did  not 
demonstrate  reading  ability  at  even  the  basic  level. 
Congress  mandated  the  National  Institute  of  Child  and 
Human  Development  (NICHD)  to  establish  the 
National  Reading  Panel  (NRP).  The  main  task  of  the 
NRP  was  to  review  current  educational  research  and 


determine  the  most  effective  strategies  and  practices 
for  teaching  reading.  In  2000,  the  NRP  released  its 
report,  emphasizing  the  necessity  of  utilizing  reading 
instruction  with  firm  scientific  grounding  to  improve 
reading  outcomes.  In  2002,  in  response  to  the  NRP 
report.  Congress  passed  the  No  Child  Left  Behind 
(NCLB)  Act  and  the  Reading  Lirst  initiative,  mandat¬ 
ing  exclusive  use  of  reading  curricula  based  on  scien¬ 
tific  research  for  those  schools  receiving  federal  grants. 
To  develop  scientifically  grounded  reading  curricula, 
researchers  have  first  sought  to  elucidate  the  compo¬ 
nents  essential  for  reading  proficiency:  decoding, 
vocabulary,  and  comprehension. 

Current  Understandings  of  Reading 

Much  of  the  existing  instructional  effort  funded  by 
Reading  Lirst  has  targeted  basic  reading  skills,  partic¬ 
ularly  word-level  decoding,  for  children  in  kindergar¬ 
ten  through  third  grade.  Of  serious  concern  is  the  iden¬ 
tification  of  strategies  for  children  with  reading 
disabilities,  particularly  those  children  with  dyslexia. 
Lor  most  individuals  with  dyslexia,  the  primary  prob¬ 
lem  appears  to  be  decoding  rather  than  comprehension 
(Torgesen,  2002).  Extensive  research  now  suggests 
that  dyslexia  derives  from  phonological  as  opposed  to 
visual  difficulties  (Schatschneider  and  Torgesen, 
2004);  consequently,  one  focus  of  current  reading 
instruction,  which  now  is  a  routine  part  of  all  major 
basic  reading  instruction,  is  promoting  the  develop¬ 
ment  of  phonological  awareness. 

Phonological  awareness  refers  to  an  awareness  of 
and  access  to  the  sound  structure  of  one’s  oral  language 
(Anthony  and  Lrancis,  2005;  Jorm  and  Share,  1983; 
Wagner  and  Torgesen,  1987).  The  pronunciations  of 
words  can  be  represented  as  strings  of  phonemes.  All 
spoken  English  words  can  be  represented  using  a  set  of 
approximately  40  phonemes.  Of  the  10  trillion  possible 
combinations  of  40  phonemes,  only  a  relatively  small 
number  actually  occur  in  spoken  language,  and  many 
of  these  combinations  occur  in  multiple  words  (Wagner 
et  al.,  1997).  Thus,  cat ,  rat ,  and  hat  each  consist  of  three 
phonemes;  the  first  is  different  and  the  latter  two  are 
identical  in  the  three  words.  These  facts  are  represented 
by  their  spellings.  Each  has  a  different  initial  letter  and 
identical  medial  and  final  letters.  To  a  child  with  pho¬ 
nological  awareness,  the  English  writing  system  will 
appear  to  be  a  sensible  way  of  representing  spoken 
words  in  print.  A  child  lacking  such  awareness  will  find 
the  English  writing  system  to  be  much  more  arbitrary. 

Two  streams  of  converging  evidence  support  the 
view  that  phonological  awareness  plays  a  causal  role 
in  learning  to  read.  First,  causal  modeling  of  longitu¬ 
dinal  correlational  data  support  such  a  role.  Wagner 


543 


Henny  P.A.  Boshuizen  et  al. 


and  colleagues  (1997)  reported  a  5-year  longitudinal 
study  of  216  children  who  were  followed  from  kinder¬ 
garten  through  fourth  grade.  The  results  supported  a 
bidirectional  model  of  causal  relations:  Phonological 
awareness  plays  a  causal  role  in  the  development  of 
beginning  reading  skills;  conversely,  the  development 
of  reading  skills  plays  a  causal  role  in  the  subsequent 
development  of  phonological  awareness. 

The  second  stream  of  support  for  reciprocal  causal 
relations  between  phonological  awareness  and  word- 
level  reading  comes  from  intervention  studies  and 
studies  of  illiterate  adults  and  poor  readers.  Support 
for  a  causal  influence  of  phonological  awareness  on 
subsequent  word  reading  comes  from  studies  that 
train  phonological  awareness  and  then  look  for  effects 
on  word-level  reading  (for  a  review  of  this  literature, 
see  the  Report  of  the  National  Reading  Panel , 
NICHD,  2000).  In  a  longitudinal  study  by  Torgesen 
and  colleagues  (1999)  children  with  poor  phonologi¬ 
cal  processing  skills  were  instructed  in  the  Auditory 
Discrimination  in  Depth  Program  (Lindamood  and 
Lindamood,  1984).  The  children  were  provided  with 
explicit  instruction  in  labeling  the  articulatory  ges¬ 
tures  associated  with  each  phoneme,  followed  by  pho¬ 
neme  tracking  activities  such  as  using  colored  blocks 
and  letters  to  represent  the  phonemes  in  words. 
Results  showed  that  when  children  with  poor  phono¬ 
logical  processing  skills  were  provided  with  intensive 
word-level  instruction,  their  phonological  processing, 
phonemic  awareness,  and  word-level  reading  skills 
improved  significantly  more  than  those  children  pro¬ 
vided  with  balanced  text  and  word-level  instruction 
and  those  children  receiving  instruction  that  mim¬ 
icked  normal  classroom  activities. 

Attention  now  is  turning  to  the  relatively  neglected 
topic  of  how  to  improve  reading  comprehension 
(Biemiller  and  Slonim,  2001).  One  approach  that  has 
been  taken  to  improving  comprehension  has  been  to 
focus  on  improving  vocabulary  and  other  oral  language 
skills.  Oral  language  skills  in  general,  and  vocabulary 
in  particular,  are  essential  for  effective  reading  compre¬ 
hension;  although  decoding  is  essential  because  children 
need  to  be  able  to  read  the  words  they  encounter,  chil¬ 
dren  also  need  to  understand  the  words  they  have  read. 

Biemiller  and  Slonim  (2001)  estimate  that  children 
acquire  as  many  as  2.2  new  root  words  a  day  from  age 
one  through  the  second  grade.  Upon  entry  into  school, 
children  begin  to  learn  words  through  either  direct  or 
indirect  instruction  (Graves,  1987).  Direct  or  explicit 
instruction  implies  that  students  are  taught  information 
about  words  and  word  meanings.  Although  this  method 
is  effective  in  imparting  word  meanings  essential  for 
the  successful  reading  of  a  particular  story  or  under¬ 
standing  of  a  new  concept,  it  is  impossible  for  instruc¬ 


tors  to  teach  all  the  words  children  need  to  acquire  to 
be  competent  readers  overall  (Sternberg,  1987).  Indi¬ 
rect  instruction  implies  that  word  meanings  are 
acquired  by  students  through  reading;  unknown  word 
meanings  are  derived  from  contextual  cues.  Beck  et  al. 
(2002)  have  developed  a  robust  approach  to  vocabulary 
instruction  that  provides  students  with  rich,  varied, 
contextual,  and  repeated  experience  with  words  (Beck 
et  al.,  2002,  p.  2)  by  utilizing  direct  and  explicit  meth¬ 
ods  of  vocabulary  instruction  in  addition  to  indirect  and 
contextual  instruction.  Rather  than  targeting  words  that 
most  children  will  acquire  without  instruction  ( clock , 
baby ,  walk),  this  instruction  targets  words  that  are  nec¬ 
essary  for  understanding  higher  level  texts  (words  such 
as  coincidence,  absurd ,  fortunate) .  Although  findings 
are  mixed  concerning  the  most  effective  means  of 
vocabulary  instruction.  Beck  and  McKeown  (1991,  p. 
805)  summarized  the  general  findings: 

First,  all  instructional  methods  produce  better  word  learn¬ 
ing  than  no  instruction.  Second,  no  one  method  has  been 
shown  to  be  consistently  superior.  Third,  there  is  advan¬ 
tage  for  methods  that  use  a  variety  of  techniques.  Fourth, 
there  is  advantage  from  repeated  exposures  to  the  words 
to  be  learned.  The  simple  version  of  these  findings  is  that 
people  tend  to  learn  what  they  are  taught,  and  more 
attention  to  what  is  being  taught  is  useful. 

Stahl  and  Fairbanks  (1986)  conducted  a  meta-anal- 
ysis  on  vocabulary  instruction  as  it  relates  to  reading 
comprehension  and  vocabulary  scores.  With  regard  to 
vocabulary,  their  analyses  suggest  that  instruction  has 
effect  sizes  of  1.37  (SD=  .76, /V=  21 ,/;  <  .01)  for  word- 
specific  contextual  vocabulary  measures,  1.70  (SD  = 
1 .42,  N  =  55, p  <  .01)  for  definitional  measures,  and  .26 
(SD  =  .29,  N  =  17 ,  p  <  .01)  for  global  measures  of 
vocabulary,  with  the  greatest  effect  sizes  being  related 
to  instruction  that  was  mixed  (direct  and  indirect). 

More  recently,  the  National  Reading  Panel 
(NICHD,  2000)  attempted  to  conduct  a  meta-analytic 
review  but  did  not  find  a  sufficient  number  of  studies 
that  met  the  vocabulary  subgroups’  criteria  for  scien¬ 
tific  rigor.  In  spite  of  its  inability  to  perform  a  meta¬ 
analysis  on  vocabulary  instruction,  the  National  Read¬ 
ing  Panel  summarized  several  key  trends  for  successful 
vocabulary  instruction  (NICHD,  2000),  which  are  sim¬ 
ilar  to  those  outlined  by  Beck  and  McKeown  (1991). 
Successful  vocabulary  instruction  should  include  both 
direct  and  indirect  instruction  and  should  involve  mul¬ 
tiple  exposures  in  rich  and  varied  context;  the  tasks 
should  be  restructured  when  necessary  to  ensure  stu¬ 
dents’  understanding.  No  single  method  of  vocabulary 
instruction  is  recommended  but  rather  a  variety  of 
techniques  should  be  utilized  to  ensure  that  the  stu¬ 
dents  are  actively  engaged. 
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As  previously  stated,  comprehension  is  the  desired 
outcome  of  reading.  Decoding  allows  children  to  read 
the  words  on  the  page  and  vocabulary  knowledge 
allows  children  to  understand  the  words  they  have  read, 
but  comprehension  goes  beyond  these  two  skills  and 
requires  children  to  put  together  the  words  into  strings 
of  ideas  and  concepts.  Although  the  connection 
between  vocabulary  and  reading  comprehension  has 
long  been  accepted  (correlations  range  from  0.3  to  0.8), 
researchers  are  continuing  to  define  this  relationship 
(Tannenbaum  et  al.,  2006).  Anderson  and  Freebody 
( 1979)  proposed  three  possible  hypotheses.  The  general 
aptitude  hypothesis  suggests  that  the  two  are  related 
because  both  are  manifestations  of  general  intelligence; 
the  general  knowledge  hypothesis  suggests  that  they 
are  related  because  they  are  both  manifestations  of  con¬ 
ceptual  knowledge;  and  the  instrumentalist  hypothesis 
suggests  that  vocabulary  and  reading  comprehension 
are  causally  related  so  improvements  in  vocabulary 
ability  will  positively  influence  reading  comprehension. 
This  last  hypothesis  is  the  most  interesting  to  research¬ 
ers  because  it  suggests  that  it  is  possible  to  influence 
reading  comprehension  through  vocabulary  instruction 
(Stahl  and  Fairbanks,  1986). 

Beck  and  colleagues  have  suggested  that  vocabu¬ 
lary  instruction  can  influence  reading  comprehension 
(Beck  and  McKeown,  1991;  Beck  et  al.,  1987,  2002) 
when  words  are  learned  in  varied,  repeated,  and  mean¬ 
ingful  ways  (Tannenbaum  et  al.,  2006).  In  the  meta¬ 
analysis  conducted  by  Stahl  and  Fairbanks  (1986)  in 
which  they  also  analyzed  vocabulary  instruction,  the 
results  lend  support  to  the  instrumental  hypothesis. 
The  analyses  yielded  effect  sizes  of  0.97  (SD  =  .81,  A 
-A\,p<  .01,  range  0-3.07)  and  0.30  (SD  =  .22,  N  = 
15, p  <  .01)  for  the  influence  of  vocabulary  instruction 
on  global  measures  of  comprehension. 

In  addition  to  indirectly  influencing  reading  com¬ 
prehension  through  vocabulary  instruction,  teaching 
methods  have  sought  to  enhance  reading  comprehen¬ 
sion  through  direct  instruction  in  cognitive  strategies. 
The  development  of  cognitive  strategies  by  readers  can 
be  broken  down  into  three  stages:  active  awareness  of 
processes  involving  comprehension,  instruction  in 
strategies  useful  for  aiding  comprehension,  and  prac¬ 
tice  and  eventual  independence  in  utilizing  strategies 
for  comprehension  (NICHD,  2000).  Among  those 
strategies  with  support  from  empirical  studies  are  men¬ 
tal  imagery,  graphic  organizers,  summarization,  and 
most  notably  question  generation.  Interestingly,  like 
vocabulary,  no  single  strategy  is  the  answer  to  the 
problem  of  comprehension.  The  National  Reading 
Panel  notes  that  mixed,  or  multiple  strategies  for  com¬ 
prehension  have  “considerable  scientific  support” 
(NICHD,  2000,  p.  46). 


Concluding  Remarks 

In  conclusion,  a  great  deal  of  research  has  been  carried 
out  on  models  of  instruction  for  development  of  reading 
and  other  language  skills.  Much  of  existing  knowledge 
is  limited  to  beginning  reading,  particularly  word-level 
decoding,  although  developments  are  being  made  to 
better  understand  vocabulary  ability,  reading  compre¬ 
hension,  and  the  relation  between  the  two.  As  progress 
is  made  in  explaining  further  the  underlying  processes 
of  learning  to  read,  corresponding  improvements  in 
developing  effective  reading  instruction  will  follow. 


LEARNING  MATHEMATICS: 

THE  PROBLEM  OF  LEARNING 
ABSTRACT  KNOWLEDGE 

Koeno  Gravemeijer 

Introduction  to  Learning  Mathematics 

Difficulty  with  learning  mathematics  is  usually 
explained  by  referring  to  the  gap  between  the  student’s 
personal  knowledge  and  the  abstract  formal  mathemat¬ 
ical  knowledge  that  has  to  be  acquired.  In  this  section, 
it  will  be  argued  that  the  problem  is  not  simply  in  the 
gap  that  has  to  be  bridged.  The  problem  is  that,  for  the 
student,  there  is  nothing  on  the  other  side  of  the  bridge. 
The  abstract  formal  mathematical  knowledge  of  the 
expert  does  not  feature  into  the  student’s  experiential 
reality.  This  point  may  be  clarified  by  looking  at  the 
genesis  of  mathematical  knowledge. 

If  one  looks  at  the  history  of  mathematics,  one  may 
discern  a  repeating  pattern  within  which  processes  or 
procedures  develop  into  objects.  An  example  of  such  a 
development  may  be  found  in  counting.  Counting  first 
emerges  as  a  procedure  to  establish  how  many  there 
are  of  something.  Gradually,  however,  numbers  get  an 
independent  quality,  as  objects  in  and  of  themselves. 
We  may  observe  a  similar  shift  with  (algebraic)  func¬ 
tions.  Functions  first  come  to  the  fore  as  prescriptions 
for  how  a  given  input  is  transformed  into  the  corre¬ 
sponding  output.  Later,  functions  are  defined  as  ordered 
number  pairs  and  are  conceived  of  as  mathematical 
objects  with  certain  characteristics;  that  is  to  say,  math¬ 
ematicians  experience  them  as  real  objects  that  they 
can  act  upon  and  reason  with— just  as  most  adults 
experience  numbers  as  real  objects.  With  the  construal 
of  various  mathematical  objects,  mathematicians  create 
a  body  of  knowledge  that  they  experience  as  real. 

Mathematics  teachers,  too,  will  have  construed 
mathematics  as  an  external  body  of  knowledge  that 
is  experientially  real  for  them.  As  a  consequence. 
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teachers  and  textbook  authors  may  take  their  own 
abstract  mathematical  knowledge  for  an  external 
body  of  knowledge  that  can  be  explained  to  students. 
The  difference  between  the  abstract  knowledge  of  the 
teachers  and  the  experiential  knowledge  of  the  stu¬ 
dents,  however,  constitutes  a  serious  source  of  mis- 
communication. 

Different  Frameworks  of  Reference 

At  a  certain  age,  young  children  do  not  understand  the 
question  "How  much  is  4  plus  4?”  even  though  they 
may  very  well  understand  that  4  apples  and  4  apples 
equals  8  apples.  The  explanation  for  this  phenomenon 
is  that,  for  them,  a  number  is  still  tied  to  countable 
objects  as  in  “4  apples.”  In  this  case,  “4”  is  more  like 
an  adjective  than  a  noun  for  them.  At  a  higher  level, 
the  number  4  will  be  associated  with  various  number 
relations,  such  as:4  =  2  +  2  =  3+  1  =  5-  1  =  8  +  2, 
and  so  on.  At  this  higher  level,  numbers  have  become 
mathematical  objects  that  derive  their  meaning  form  a 
network  of  number  relations  (Van  Hiele,  1973). 

When  elementary-school  teachers  talk  about  num¬ 
bers,  they  may  very  well  be  talking  about  mathematical 
objects  that  are  not  part  of  the  students’  experiential 
realities;  thus,  teachers  and  students  speak  different 
languages— without  being  aware  of  it.  He  or  she  talks 
about  numbers  as  mathematical  objects  that  exist 
within  a  network  of  numerical  relations.  The  teacher 
may,  for  instance,  reason  that  “7  +  6  equals  13  because 
7  +  3  =  10, 6  =  3  +  3,  and  10  +  3  equals  13”.  Students 
who  have  not  yet  construed  the  necessary  network  of 
numerical  relations  and  think  of  numbers  as  adjectives 
cannot  follow  this  line  of  reasoning  and  have  to  revert 
to  copying  and  memorizing. 

Models 

To  accommodate  the  abstract  character  of  mathemat¬ 
ics,  instructional  designers  try  to  devise  tactile  or 
visual  models  (embodiments)  that  represent  mathe¬ 
matical  relationships  and  concepts  to  students  in  a 
readily  apprehensible  form.  The  underlying  idea  is  that 
these  external  representations  will  facilitate  the  pro¬ 
cess  of  making  connections  with  the  represented  math¬ 
ematical  relationships  and  concepts.  In  this  respect,  the 
word  transparent  is  used,  which  suggests  that  students 
can  look  through  the  models  and  see  the  mathematics. 
This  would  enable  the  students  to  construct  internal 
mental  representations  that  mirror  those  embodied  in 
the  external  representations.  The  most  well-known  tac¬ 
tile  models  that  fit  this  approach  are  the  Dienes  blocks, 
which  are  meant  to  concretize  the  decimal  system  (see 
Figure  41 .1). 


Figure  41.1  Dienes  blocks. 


Students  are  expected  to  see  one  big  block  as  con¬ 
sisting  of  1000  tiny  cubes,  a  flat  slice  as  100,  and  a 
rod  as  10  tiny  blocks.  Research,  however,  shows  that 
this  interpretation  is  not  self-evident  (Labinowics, 
1985).  Cobb  et  al.  (1992)  argue  that  the  problematic 
character  of  the  Dienes  blocks  lies  in  its  assumption 
that  instructional  representations  are  the  primary 
source  of  the  students’  mathematical  knowledge.  For 
us  as  adults,  it  is  self-evident  what  these  instructional 
representations  signify,  but  this  is  not  the  case  for 
young  students.  The  plausibility  of  the  usefulness  of 
representations  such  as  those  resides  in  the  fact  that 
mathematics  educators  experience  mathematical  con¬ 
structs,  such  as  tens,  ones,  and  hundreds,  as  object-like 
entities  that  can  be  pointed  to  and  spoken  about.  This 
feeling  is  due  not  only  to  their  individual  mathematical 
sophistication  but  also  to  their  experience  of  being  able 
to  talk  and  reason  about  these  objects  unproblemati- 
cally  while  interacting  with  others. 

From  the  perspective  of  teachers  and  instructional 
designers,  it  makes  perfect  sense  to  try  to  develop 
transparent  models;  they  see  their  knowledge  of  the 
decimal  system  reflected  in  the  blocks.  For  students, 
however,  the  Dienes  blocks  are  nothing  but  just 
wooden  blocks.  We  cannot  expect  the  students  to  see 
more  sophisticated  mathematics  in  the  blocks  than  the 
mathematics  they  have  already  acquired.  This  problem 
is  known  as  the  learning  paradox  (Bereiter,  1985): 
How  would  it  be  possible  for  the  students  to  recognize 
mathematical  relationships  that  are  developmentally 
more  advanced  than  what  they  already  know?  The 
instructional  consequence  is  that  when  students  do  not 
see  what  there  is  to  be  seen,  the  teacher  does  not  have 
many  options  other  than  to  spell  out  the  correspon¬ 
dences  between  the  blocks  and  the  algorithm  in  detail. 

What  Makes  Mathematics  Difficult? 

In  short,  we  may  conclude  that  the  difficulty  of  learn¬ 
ing  mathematics  does  not  lie  in  the  formal,  abstract 
character  of  mathematics  as  such.  The  actual  problem 
seems  to  reside  in  the  fact  that  teachers  and  the  text¬ 
book  authors  mistake  their  own  more  abstract  math¬ 
ematical  knowledge  for  an  external  body  of  knowl¬ 
edge  that  can  be  explained  to  the  students;  however, 
the  gap  between  the  knowledge  of  the  teachers  and 
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the  experiential  reality  of  the  students  is  too  big  to 
make  this  work.  We  saw  that  instructional  representa¬ 
tions  cannot  bridge  this  gap,  because  what  those  mate¬ 
rials  signify  depends  on  the  eye  of  the  beholder.  We 
may  conclude,  therefore,  that  it  is  the  tradition  of 
ignoring  where  mathematics  comes  from  that  makes 
mathematics  so  difficult  to  learn.  One  might,  of  course, 
counter  that  reality  shows  that  (at  least  some)  people 
appear  to  have  learned  mathematics  in  spite  of  this 
form  of  instruction.  We  may  reason,  however,  that  their 
actual  learning  process  will  have  been  different  from 
absorbing  or  memorizing  an  alien  body  of  knowledge. 
According  to  Freudenthal  (1991),  they  may  have  been 
able  to  reconstruct  mathematics  in  a  manner  similar  to 
a  reinvention  process.  For  these  people  the  adage 
learning  first,  understanding  later  may  be  true;  how¬ 
ever,  Freudenthal  (1991)  wondered  whether  that  holds 
true  for  others  who  might  be  better  served  by  what 
Freudenthal  calls  guided  reinvention . 

Alternative 

Freudenthal  (1973, 1991)  observed  that  the  end  prod¬ 
uct  of  the  mathematical  activity  of  many  generations 
of  mathematicians  is  taken  as  a  starting  point  for  the 
instruction  of  young  students.  He  called  this  an  anti- 
didactical  inversion.  An  alternative  is  to  create  oppor¬ 
tunities  for  students  to  reinvent  mathematics .  In  rela¬ 
tion  to  this,  he  spoke  of  “mathematics  as  a  human 
activity.”  Just  as  the  activity  of  mathematicians 
resulted  in  mathematics  as  we  know  it  now,  the  activ¬ 
ity  of  students  can  result  in  the  construction  of  math¬ 
ematics.  This  approach,  therefore,  offers  an  alterna¬ 
tive  for  teaching  students  mathematics  as  a  ready¬ 
made  product. 

We  may  elaborate  Freudenthal’ s  point  further.  For 
him— as  a  mathematician— mathematics  is  primarily 
an  activity.  An  activity  that  he  denoted  as  “mathema- 
tizing”  or  organizing.  In  relation  to  this,  he  spoke  of 
the  activity  of  organizing  subject  matter  to  make  it 
more  mathematical.  This  may  concern  both  organizing 
matter  from  reality  to  make  it  accessible  for  mathe¬ 
matical  means  and  organizing  mathematical  matter  to 
make  it  more  mathematical.  We  may  relate  “more 
mathematical”  in  this  context  to  characteristics  such 
as  general,  exact, brief,  and  sure,  which  suggests  math¬ 
ematical  activities  such  as  generalizing,  formalizing, 
curtailing,  and  proving.  Freudenthal  (1973)  argued  that 
students  can  reinvent  mathematics  by  mathematizing, 
although  he  also  acknowledged  that  the  students  can¬ 
not  simply  reinvent  the  mathematics  that  took  the 
brightest  mathematicians  eons  to  develop.  This  is  why 
he  proposes  guided  reinvention.  Teachers  and  text¬ 
books  have  to  help  the  students  along,  while  trying  to 


make  sure  that  the  students  experience  learning  math¬ 
ematics  as  a  process  of  inventing  mathematics  for 
themselves.  To  accomplish  this,  a  reinvention  route  has 
to  be  developed;  therefore,  teachers  need  the  help  of 
instructional  designers,  who  in  turn  may  be  supported 
by  researchers.  Offering  this  kind  of  support  has  been 
the  mission  of  the  Freudenthal  Institute  in  the  Nether¬ 
lands  over  the  past  decades.  This  has  resulted  in  what 
is  referred  to  as  a  domain-specific  instruction  theory 
for  realistic  mathematics  education  (RME). 

Emergent  Modeling 

According  to  this  theory,  guided  reinvention  may  be 
supported  by  emergent  modeling  (Gravemeijer,  1999, 
2004).  The  emergent  modeling  approach  circumvents 
the  learning  paradox  in  a  way  that  fits  with  Meira’s 
(1995)  historical  analysis.  On  the  basis  of  that  histor¬ 
ical  analysis,  he  suggested  a  dialectic  process  of  sym¬ 
bolizing  and  meaning  making,  in  which  both  the  sym¬ 
bolizations  and  the  corresponding  meaning  co-evolve. 
Historically,  symbols  and  models  did  not  materialize 
out  of  thin  air;  they  are  the  result  of  a  long  process  of 
inventing,  adjusting,  and  refining.  So,  one  might  con¬ 
clude  that,  instead  of  trying  to  help  students  to  make 
connections  with  ready-made  mathematics,  we  should 
try  to  help  students  construe  mathematics  in  a  more 
bottom-up  manner. 

This  recommendation  fits  with  the  idea  of  emer¬ 
gent  modeling.  The  emergent  modeling  approach  takes 
its  point  of  departure  in  the  activity  of  modeling.  Mod¬ 
eling  in  this  conception  is  an  activity  of  the  students, 
who  are  asked  to  solve  a  contextual  problem.  The 
students  then  model  the  problem  to  solve  it  with  the 
help  of  that  model.  Such  a  modeling  activity  might 
involve  making  drawings,  diagrams,  or  tables,  or  it 
could  involve  developing  informal  notations  or  using 
conventional  mathematical  notations.  The  conjecture 
is  that  acting  with  these  models  will  help  the  students 
reinvent  the  more  formal  mathematics. 

Initially,  the  models  come  to  the  fore  as  context- 
specific  models.  The  models  refer  to  concrete  or  para¬ 
digmatic  situations  that  are  experientially  real  for  the 
students.  Initial  models  should  allow  for  informal  strat¬ 
egies  that  correspond  with  situated  solution  strategies  at 
the  level  of  the  situation  that  is  defined  in  the  contextual 
problem.  Then,  while  the  students  gather  more  experi¬ 
ence  with  similar  problems,  their  attention  may  shift 
toward  the  mathematical  relations  and  strategies.  This 
helps  them  to  further  develop  those  mathematical  rela¬ 
tions  which  enables  them  to  use  the  model  in  a  different 
manner:  The  model  becomes  more  important  as  a  base 
for  reasoning  about  these  mathematical  relations  than 
as  a  way  of  representing  a  contextual  problem.  In  this 
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Figure  41.2  Tenstrip. 

manner,  the  model  begins  to  become  a  referential  base 
for  more  formal  mathematics.  In  short,  a  model  of 
informal  mathematical  activity  develops  into  a  model 
for  more  formal  mathematical  reasoning. 

This  approach  has  been  elaborated  successfully  for 
a  variety  of  topics,  varying  from  data  analysis  (Cobb, 
2002)  and  calculus  (Doorman,  2005)  to  addition  and 
subtraction  up  to  100  (Stephan  et  al.,  2003).  We  will 
take  the  latter  as  an  easily  accessible  example  and 
briefly  describe  the  instructional  sequence  that  aims  at 
developing  a  framework  of  number  relations  that  stu¬ 
dents  can  use  for  flexible  strategies  for  addition  and 
subtraction  up  to  100.  The  starting  point  of  that 
sequence  is  in  measuring  lengths  by  iterating  a  given 
measurement  unit,  quickly  followed  by  iterating  units 
of  ten  and  one.  While  coordinating  units  of  ten  and 
one,  the  students  begin  to  become  aware  of  useful 
number  relations ,  such  as  48  =  50  -  2 ,  and  48  +  10  =  58. 

Next,  a  paper  strip  of  ten  units  that  can  be  used  for 
both  iterating  units  of  ten  and  reading  individual  units 
is  introduced  as  a  measurement  device  (see  Figure 
41 .2);  subsequently,  a  ruler  is  construed  by  combining 
ten  of  the  so-called  tenstrips.  Measuring  with  this  ruler 
builds  on  the  earlier  activity  of  iterating  units  of  ten 
and  one.  New  activities  follow  that  involve  comparing, 
incrementing,  and  decrementing  lengths.  Students  may 
begin  using  multiples  of  ten  as  benchmarks  instead  of 
counting  individual  units;  for  example,  the  difference 
between  48  and  75  can  be  found  by  using  50,  60,  and 
70  as  benchmarks,  while  reasoning  that  48  +  2  =  50, 
50  +  20  =  70,  70  +  5  =  75,  thus  the  difference  is  2  + 
20  +  5  =  27.  Next,  these  arithmetical  strategies  are 
represented  by  jumps  on  an  empty  number  line  (see 
Figure  41.3a).  Later  on,  when  the  students  have 
become  more  flexible  with  number  relations,  they  may 
reason:  75  -48  =  75  -50  +  2,  which  they  may  elucidate 
with  jumps  on  the  number  line  (see  Figure  41.3b). 

The  model  then  derives  its  meaning  from  the  math¬ 
ematical  relations  and  has  become  a  model  for  math¬ 
ematical  reasoning.  In  retrospect,  we  may  observe  a 
shift  from  numbers  that  initially  are  bound  to  measures 
(e.g.,  37  inches)  to  numbers  as  entities  in  and  of  them¬ 
selves  that  derive  their  meaning  from  a  network  of 
number  relations  (e.g.,  37  =  30  +  7  =  40  -  3  =  20  + 
17  =  47  -  10).  The  sequence  also  gives  a  flavor  of  how 
the  emergent  modeling  approach  can  be  used  in 
instructional  design.  First,  the  designer  should  try  to 
describe  the  goals  in  terms  of  a  framework  of  mathe- 
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Figure  41.3  (a)  Jumps  on  the  empty  number  line;  (b)  illustrat¬ 
ing  75  -  48  =  75  -  50  +  2. 

matical  relations;  this  informs  the  designer  about  the 
relations  that  have  to  be  focused  on.  Second,  the 
designer  has  to  try  to  design  a  series  of  submodels  that 
may  support  a  “model  of’  to  “model  for”  transition  in 
such  a  manner  that  each  new  submodel  builds  on  ear¬ 
lier  experiences  with  preceding  submodels. 

Conclusions 

Abstract  mathematical  knowledge  is  difficult  to  trans¬ 
mit  to  students  because  it  concerns  knowledge  on  a 
different  level  of  understanding.  The  actual  problem, 
however,  is  that  we  tend  to  think  of  abstract  mathe¬ 
matical  knowledge  as  an  external  body  of  knowledge 
that  can  be  explained  to  the  students.  Instead  of  trying 
to  help  students  to  make  connections  with  a  body  of 
mathematical  knowledge  that  “is  not  there”  in  the 
experiential  realities  the  students,  we  may  want  to  try 
to  help  them  construct  new  mathematical  knowledge 
by  building  on  what  they  already  know.  Some  of  the 
strategies  that  realistic  mathematics  education  offers 
for  this  alternative  approach  are  guided  reinvention  and 
emergent  modeling. 

An  important  aspect  of  this  approach  is  that  it 
creates  opportunities  for  students  to  develop  mathe¬ 
matical  knowledge  that  is  grounded  in  their  experien¬ 
tial  reality.  More  importantly,  it  leaves  the  connection 
with  those  sources  open  which  allows  students  to  fold 
back  to  more  concrete  levels  of  understanding  if  they 
run  into  problems.  The  RME  approach,  however,  does 
not  offer  an  easy  solution  to  the  problem  of  how  to 
teach  abstract  mathematics.  First,  realistic  mathemat¬ 
ics  education  requires  instructional  design.  Teachers 
need  instructional  sequences  or,  better,  the  rationales 
or  local  instruction  theories  that  underpin  such 
sequences,  together  with  resources  that  offer  sets  of 
potentially  productive  activities.  Second,  teacher 
development  is  necessary.  Within  the  conception  of 
teaching  as  helping  students  in  making  connections 
with  mathematics  as  a  ready-made  body  of  knowledge, 
the  central  teaching  activities  will  be  those  of  giving 
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directions  and  explaining.  To  change  to  the  alternative 
approach,  teachers  will  have  to  make  a  shift  from  giv¬ 
ing  directions  and  explanations  to  helping  students  to 
invent  mathematics.  This  presents  them  with  the  diffi¬ 
cult  task  of  building  on  the  input  and  ideas  of  the 
students  while  at  the  same  time  working  toward  given 
conventional  goals. 

Helping  students  invent  implies  that  teachers  will 
have  to  share  some  of  their  authority.  The  teacher, 
though,  remains  the  authority  in  the  classroom  but  in 
a  different  role.  The  teacher  establishes  the  rules 
regarding  what  mathematics  is  and  what  it  means  to 
learn  mathematics  in  his  or  her  classroom.  Moreover, 
the  teacher  decides  on  the  choice  of  instructional  activ¬ 
ities,  frames  topics  for  discussion,  and  orchestrates 
whole-class  discussions  in  such  a  manner  that  they 
contribute  to  the  construction  of  the  intended  mathe¬ 
matics.  In  doing  so,  these  teachers  will  have  to  strike 
a  balance  between  guidance  and  invention  or  reinven¬ 
tion.  In  short,  it  is  the  teacher  who  is  the  one  who 
shapes  the  curriculum  innovation  that  is  implied  here. 


LEARNING  SCIENCE: 

PROVOKING  EXPLORATION 
OF  PHENOMENA  IN  NATURE 

Geerdina  Maria  van  der  Aalsvoort 

Introduction  to  Learning  Science 

Science  learning  suggests  that  students  become  more 
familiar  with  concepts  related  to  biology,  physics, 
chemistry,  or  astronomy  and  develop  conceptual 
knowledge  about  the  domains.  Science  learning  is 
strongly  dependent  on  science  teaching.  Although 
early  exposures  to  natural  phenomena  supposedly  trig¬ 
ger  a  child’s  interest  in  concepts  related  to  science,  a 
certain  level  of  cognitive  development  is  required  for 
the  child  to  notice  natural  phenomena,  and  this  level 
of  development  is  especially  necessary  for  the  child  to 
be  able  to  pose  questions  related  to  the  underlying 
mechanisms  of  the  phenomena. 

From  a  constructivist  point  of  view,  growing 
understanding  of  concepts  is  part  of  the  development 
of  cognitive  systems  that  take  place  over  time.  Piaget 
(DeLisi  and  Golbeck,  1999)  evolved  a  constructivist 
theory  of  cognitive  functioning  and  development  that 
attempts  to  account  for  the  growth  and  development 
of  existing  cognitive  systems  and  application  of 
already  formed  cognitive  systems.  Piaget  found  that 
children  develop  their  concepts  of  natural  phenomena 
through  a  process  of  finding  a  so-called  equilibrium 


between  new  skills  that  exert  pressure  on  concepts 
already  formed  and  those  already  established.  In  the 
child’s  struggle  to  relate  those  experiences,  he  or  she 
may  either  fail  to  account  for  aspects  of  the  environ¬ 
ment  or  be  overly  influenced  by  environmental  forces. 
The  child’s  own  activity,  attention,  and  interest  deter¬ 
mine  whether  assimilation  or  accommodation  takes 
place  when  constructing  conceptual  models. 

Perret-Clermont  et  al.  (2004), based  on  Vygotsky’s 
point  of  view,  emphasized  the  dependence  of  construc¬ 
tion  of  knowledge  on  outside  influences.  They  noted 
that  a  child  learns  when  teachers  bridge  the  gap 
between  the  child’s  available  modes  of  thought  and 
new  ideas  by  working  jointly  in  the  zone  of  proximal 
development.  In  this  way,  the  teacher,  as  “outside  influ¬ 
ence,”  guides  the  child  toward  more  advanced  forms 
of  thought. 

Especially  in  the  case  of  science,  learning  is  con¬ 
sidered  to  be  something  that  is  transmitted  in  school 
(Nunes  and  Bryant,  2004).  The  development  of  sci¬ 
entific  reasoning  comes  about  when  children  learn  to 
use  the  logical  aspects  of  a  concept,  understand  sit¬ 
uations  that  give  meaning  to  a  concept,  and  become 
sensitive  to  the  way  a  problem  is  represented.  Con¬ 
ceptual  mastery  of  scientific  concepts  requires  being 
confronted  with  phenomena  and  finding  solutions  to 
understand  a  phenomenon  by  using  logical  thinking. 
The  phenomena  provide  a  means  for  cultivating  or 
practicing  the  construction  of  explanations.  Explana¬ 
tion  generation  is  something  that  can  be  promoted  in 
science  classrooms .  Whereas  current  theories  recog¬ 
nize  that  there  are  general  ways  in  which  a  child’s 
intellect  changes  as  the  child  grows  older,  accounting 
for  the  solution  of  specific  problems  requires  an  addi¬ 
tion  to  or  a  change  of  currently  available  knowledge 
(Nunes  and  Bryant,  2004). 

From  a  constructivist  approach,  new  information 
is  built  upon  existing  knowledge  structures.  This 
approach  defines  conceptual  change  within  a  child’s 
knowledge  system  as  follows.  Early  in  their  lives,  and 
over  a  period  of  time,  children  know  a  great  deal  about 
separate  science  issues.  These  disparate  elements  are 
all  organized  in  complex  ways.  When  new  information 
is  to  be  learned  that  comes  into  conflict  with  a  learner’s 
prior  knowledge  system,  a  major  reorganization  is 
required.  Teachers  can  function  as  facilitators  to  guide 
the  process  of  learning  science  and  intentionally  pro¬ 
voke  such  reorganization  by  the  students  in  their 
classes  (Ohlsson  et  al.,  2000).  There  is  no  guarantee, 
however,  that  instruction  and  reorganization  are  imme¬ 
diately  successful.  Science  learning  is  a  construction 
process  of  new  conceptual  identities  and  is  not  merely 
a  replacement  of  naive  knowledge  with  scientific 
knowledge  (Kumpulainen  et  al.,  2003).  Students  need 
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to  develop  the  appropriate  cognitive  structures  before 
they  can  master  scientific  concepts,  as  there  is  a  dis¬ 
tinction  between  the  child’s  initial  exploratory  frame¬ 
work  prior  to  systematic  instruction  and  misconcep¬ 
tions  that  may  occur  after  instruction  (Vosniadou  and 
Verschaffel,  2004). 

Consequences  for  Learning  Astronomy 

Based  on  Piaget’s  theory  of  development  of  concepts, 
many  studies  have  been  undertaken  that  investigate  the 
development  of  concepts  of  natural  phenomena  such  as 
those  related  to  astronomy  (Vosniadou  and  Brewer, 
1992, 1994;  Vosniadou  et  al.,  2005).  The  majority  of  the 
first-grade  children  included  in  these  studies  have  been 
found  to  have  formed  initial  models  of  the  day  and  night 
cycle.  Moreover,  most  of  the  participating  children 
showed  that  they  had  discovered  that  the  alternation  of 
day  and  night  constitutes  a  phenomenon  that  requires 
an  explanation.  They  also  showed  that  they  had  learned 
to  form  so-called  mental  models  of  natural  phenomena 
such  as  the  day  and  night  cycle.  A  mental  model  refers 
to  a  particular  kind  of  representation  that  is  analogous 
to  the  states  of  the  world  it  represents.  It  can  be  manip¬ 
ulated  mentally,  it  allows  predictions  about  the  out¬ 
comes  of  causal  states  in  the  world,  and  it  provides 
physical  phenomena  (Vosniadou  and  Brewer,  1994). 

To  investigate  development  concerning  conceptual 
changes,  an  empirical  approach  can  be  sought  that 
explores  how  young  children  develop  science  concepts 
after  they  are  exposed  to  materials  that  evoke  spontane¬ 
ous  discovery  learning  about  natural  phenomena.  Infor¬ 
mation  collected  with  respect  to  such  learned  concepts 
can  then  be  used  to  rethink  what  could  be  successful 
elements  of  science  teaching  in  formal  education.  We 
know,  for  example,  from  the  study  of  Ohlsson  et  al. 
(2000)  that  kindergarten  children  (mean  age  5  years) 
profited  from  being  exposed  to  a  game  that  included 
virtual  traveling  in  space  from  an  asteroid  to  Earth.  They 
had  significantly  more  correct  answers  on  questions 
about  the  shape  of  the  Earth  immediately  after  exposure 
to  the  game.  This  result  was  maintained  after  a  delay  of 
4  months,  when  the  same  questions  were  posed  again. 
Findings  like  these  suggest  that  discovery  learning  is  a 
fruitful  way  to  promote  young  children’s  learning  of 
conceptual  understanding  of  natural  phenomena  while 
at  the  same  increasing  their  technology  literacy  (Cam¬ 
eron  and  Dwyer,  2005;  Hertzog  and  Klein,  2005). 

A  Field  Study 

Based  on  the  findings  of  Ohlsson  et  al.  (2000),  we 
studied  children’s  conceptual  development  by  using 
PICCO,  a  computer  program  of  particular  natural  phe¬ 


nomena  (Kangassalo,  1997).  PICCO  offers  virtual 
opportunities  to  explore  the  Earth,  the  planets  in  space, 
the  meaning  of  time  with  respect  to  day  and  night,  and 
the  seasons.  Seven  actions  can  be  activated  on  the 
screen:  looking  at  a  city  in  a  virtual  landscape;  chang¬ 
ing  dates,  which  allows  seasons  to  change;  turning 
hands  on  the  clock  so  day  turns  into  night  and  vice 
versa ;  using  a  microscope  or  binoculars;  using  a  dic¬ 
tionary;  using  a  rocket  that  simulates  travel  into  space; 
and  using  a  compass  that  allows  the  user  to  change  the 
angle  of  the  landscape  on  the  screen. 

Because  virtual  traveling  in  space  is  possible  by 
launching  the  rocket,  children  can  study  the  position  of 
the  Earth  in  relationship  to  the  moon,  the  sun,  and  other 
planets  in  space.  This  allows  the  child  to  see  the  Earth 
from  outer  space  and  to  notice  that  the  Earth  is  round. 
Moreover,  the  change  of  seasons  can  be  elicited  by 
manipulating  the  date,  which  causes  the  landscape  to 
change  from  images  related  to  spring,  summer,  autumn, 
or  winter.  Turning  the  hands  on  the  clock  can,  for  exam¬ 
ple,  cause  day  to  become  night;  as  the  landscape  dark¬ 
ens,  the  birds  stop  singing.  The  visual  experiences  on 
the  screen  can  be  elicited  any  time  the  child  activates 
the  game.  There  are  neither  time  limits  nor  rules  for 
using  the  buttons  that  activate  the  elements  on  the 
screen.  The  children’s  mental  models  of  the  Earth  and 
their  ideas  of  the  day  and  night  cycle  and  the  seasons 
may  differ  from  the  images  presented  on  the  screen. 
Actions  that  are  allowed  by  PICCO  are  intended  to 
evoke  a  cognitive  conflict  in  the  child’s  conceptual 
understanding  of  these  phenomena  of  nature.  As  a  con¬ 
sequence  of  this  conflict,  the  child’s  understanding  of 
a  particular  phenomenon  is  put  under  stress,  which 
should  evoke  an  imbalance  in  their  conceptual  knowl¬ 
edge  related  to  the  natural  phenomenon.  This  imbal¬ 
ance  in  turn  may  promote  deeper  understanding  of  the 
phenomenon  observed  and  elicit  understanding  of  the 
phenomenon  on  a  higher  conceptual  level. 

Results 

We  used  PICCO  to  study  the  broader  question  of 
whether  young  students  would  profit  from  exposure  to 
a  program  that  allowed  investigation  of  natural  phe¬ 
nomena  in  a  game-like  environment  (for  details  of  the 
study,  see  Van  der  Aalsvoort  et  al.,  2006).  The  program 
was  installed  on  a  computer  in  one  classroom  in  each 
of  two  different  schools  and  was  available  for  use  over 
4  subsequent  weeks.  The  teachers  offered  time  and 
opportunity  for  their  children  to  use  the  program  at  least 
once  a  week.  After  4  weeks,  PICCO  was  removed  from 
the  computer.  The  participants  were  interviewed  both 
before  and  after  their  exposure  to  PICCO  regarding 
their  knowledge  and  understanding  of  the  phenomena 
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covered  by  the  program.  The  sample  involved  20  sec¬ 
ond-grade  students  from  two  regular  education  pri¬ 
mary  schools.  Ten  children  from  each  class  partici¬ 
pated.  They  had  been  selected  as  the  first  ten  on  a  list 
of  students  in  their  class.  We  collected  intelligence 
scores  as  a  child  characteristic ,  as  well  as  information 
about  the  socioeconomic  status  of  the  child’s  home. 
The  mean  scores  on  IQ  were  equal  between  the 
schools,  but  the  scores  related  to  socioeconomic  status 
were  remarkably  higher  in  school  2  than  in  school  1 . 
As  a  consequence,  we  tested  the  data  for  each  school 
separately. 

The  methodology  involved  three  steps  that  were 
designed  to  ensure  that  differences  in  the  mental  mod¬ 
els  of  the  children  after  the  intervention  could  be  inter¬ 
preted  as  results  of  their  exposure  to  the  PICCO  game. 
The  first  step  was  to  compare  the  number  of  correct 
answers  between  the  schools  based  on  the  interview 
with  each  student  before  and  after  the  intervention. 
The  children  in  school  2  only  showed  a  significantly 
higher  number  of  correct  responses  to  the  interview 
questions  about  natural  phenomena  after  the  exposure 
to  the  game. 

The  second  step  involved  analyses  of  the  answers 
of  the  children  to  the  interview  questions  to  reveal 
specific  changes  that  may  have  resulted  from  their 
involvement  in  the  gaming.  Three  levels  of  conceptual 
knowledge  were  distinguished  with  regard  to  the  Earth, 
day  and  night  cycle,  and  seasons  by  combining 
answers  from  the  interview  into  those  themes.  The 
proportion  of  answers  categorized  as  representing  a 
scientific  model  of  the  shape  of  the  Earth  improved 
from  20  to  40%  in  the  post-test  responses  of  children 
in  school  1 ,  whereas  the  post-test  responses  of  children 
in  school  2  changed  from  80  to  90%.  Students  from 
school  2  gave  answers  that  were  related  to  a  scientific 
model  of  the  day  and  night  cycle  in  20%  of  their 
answers  after  exposure  to  PICCO,  whereas  none  of  the 
students  from  school  1  reached  that  level.  We  attribute 
the  huge  differences  between  the  pretest  (maintained 
in  the  post-test)  scores  of  the  children  in  these  groups 
to  socioeconomic  status,  as  the  children  from  school 
2  came  from  a  significantly  higher  SES  level.  Thus, 
the  children  in  school  2,  despite  IQ  scores  at  a  similar 
level  as  those  in  school  1,  came  from  a  higher  SES 
level  and  obtained  dramatically  higher  scores  on  their 
pre-  and  post-intervention  interview  responses. 

In  the  final  step,  we  analyzed  the  children’s  explo¬ 
ration  pathways.  The  explorations  pathways  consist  of 
the  visual  pattern  reflected  in  the  number  of  times  and 
the  order  in  which  actions  were  undertaken  during 
each  PICCO  session.  We  expected  that  the  children 
would  intentionally  push  buttons  to  achieve  sequences 
in  actions  on  the  screen  which  would  suggest  that  they 


were  processing  the  information  on  a  conceptual  level. 
Three  main  sequences  were  anticipated.  The  first  was 
related  to  the  shape  of  the  Earth:  The  child  used  buttons 
in  a  sequence  of  launching  a  rocket,  viewing  Earth 
from  outer  space  as  a  round  object,  and  then  traveling 
back  to  earth.  The  second  sequence  referred  to  the  day 
and  night  cycle:  The  child  moved  the  hands  of  the 
clock  and  saw  that  the  landscape  changed  from  day 
into  night  and  vice  versa.  The  third  sequence  related 
to  season:  The  child  changed  the  date  on  the  calendar 
so  the  landscape  changed  from  one  season  into  another 
and  then  changed  the  date  once  again.  An  example  is 
a  student  we  shall  call  William.  William  took  part  in 
ten  sessions  of  PICCO.  One  sequence  related  to  Earth 
was  found.  Moreover,  three  times  sequences  related  to 
the  day  and  night  cycle  and  to  the  seasons  were 
detected.  His  score  on  the  interview  after  the  interven¬ 
tion  had  improved  on  the  theme  of  Earth  only. 

The  final  analysis  showed  a  nonsignificant  rela¬ 
tionship  between  the  higher  order  learning  processes 
detected  in  the  children’s  exploration  paths  and  the 
answers  they  gave  during  the  interviews.  We,  therefore, 
are  not  able  to  conclude  that  intentional  actions  with 
immediate  feedback  on  the  screen  related  to  the  deep¬ 
ening  of  the  children’s  conceptual  development.  The 
results  do  allow  us,  however,  to  conclude  that  the  stu¬ 
dents  from  school  1  profited  mainly  with  respect  to 
their  knowledge  of  the  shape  of  the  Earth  as  a  natural 
phenomenon,  whereas  the  students  from  school  2 
improved  primarily  with  respect  to  their  knowledge  of 
the  day  and  night  cycle  and  the  seasons. 

In  summary,  the  findings  related  to  the  use  of 
PICCO  in  these  second-grade  classrooms  revealed  that 
the  students  profited  from  exposure  to  a  program  that 
allows  investigation  of  natural  phenomena  in  a  game¬ 
like  way:  however,  the  children  in  each  school  made 
improvements  in  different  content  areas  and  at  differ¬ 
ent  levels.  The  students  from  school  1  improved  with 
respect  to  their  conceptual  understanding  of  the  shape 
of  the  Earth.  In  school  2,  this  knowledge  was  already 
present  in  most  students.  Among  the  school  2  students, 
the  most  improvement  was  found  with  regard  to  their 
understanding  of  the  day  and  night  cycle. 

One  main  factor  that  seems  to  relate  to  these  find¬ 
ings  is  that  of  SES;  it  seems  that  the  higher  the  stu¬ 
dents’  socioeconomic  status  and  the  more  developed 
their  conceptual  knowledge  about  natural  phenomena, 
the  less  they  profited  from  a  game  that  intends  to  allow 
discovery  learning  about  these  phenomena.  The  inter¬ 
vention  findings  also  suggest,  however,  that  although 
home  environment  may  contribute  to  less  developed 
conceptual  developmental  about  natural  phenomena 
improvement  of  the  conceptual  knowledge  can  be 
derived  within  a  very  short  time. 
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Discussion 

The  findings  of  the  study  have  several  implications.  We 
present  two  that  may  improve  our  understanding  of  how 
the  construction  of  mental  models  can  be  elicited  by 
intervening  in  the  child’s  surroundings  (e.g.,  the  class¬ 
room)  and  what  this  can  mean  for  educational  planning. 
The  first  implication  concerns  the  relationship  between 
socioeconomic  status  of  the  participants  and  exposure 
to  materials  that  aim  at  eliciting  development  of  con¬ 
ceptual  models  (Kohnstamm,  1967;  Szechter  and  Liben, 
2004).  A  study  about  solving  scientific  problems  by  9- 
to  12-year-old  children  in  proximity  with  their  parents 
revealed  that  the  parents  of  the  children  assumed  the 
role  of  evidence  interpretation  but  did  not  engage  in 
collaborative  discussions  within  which  the  child  could 
develop  scientific  reasoning  (Gleason  and  Schauble, 
2000).  The  study  alerts  us  with  a  concern  that  offering 
effective  assistance  appears  to  be  related  to  verbaliza¬ 
tions  of  scientific  problem  solving.  If  this  is  the  case, 
then  children  less  experienced  in  verbalizing  their  own 
explanations  may  be  at  a  disadvantage  in  showing  their 
understanding  of  natural  phenomena.  This  concern  can¬ 
not  be  accounted  for  in  our  findings,  because  when  we 
interviewed  the  students  we  offered  them  paper  and 
pencil  and  play  dough  so  they  could  show  the  shape  of 
the  Earth  and  the  position  of  the  sun  nonverbally  as  well 
as  verbally.  The  children  involved  were  not  disadvan¬ 
taged  in  showing  their  conceptual  knowledge  of  natural 
phenomena.  A  second  concern  relates  to  the  way  the 
parents  in  the  previous  studies  assisted  their  child  in 
exploring  natural  phenomena.  The  differences  between 
the  schools  in  our  study  may  have  related  to  the  parents’ 
views  about  their  activities  with  their  children  at  home; 
this  may  have  made  it  less  interesting  for  the  children 
to  explore  natural  phenomena  as  they  were  too  unfamil¬ 
iar  with  the  type  of  information  provided  by  the  screen. 

The  second  implication  of  the  results  of  this  study 
relates  to  the  use  of  artifacts.  Vosniadou  et  al.  (2005) 
discussed  the  specific  meaning  of  artifacts  for  scientific 
reasoning.  They  stated  that  children  need  extensive 
instruction  with  the  artifact  to  learn  how  to  use  it  and 
to  grasp  the  conceptual  meaning  behind  it.  Our  expe¬ 
rience  with  the  PICCO  materials  revealed  that  the  chil¬ 
dren  could  grasp  concepts  through  reference  to  arti¬ 
facts,  such  as  the  Earth  in  space  being  portrayed  on  a 
flat  screen  as  a  round  shape.  An  alternative  explanation 
of  the  results  of  the  study  is  that  adjusting  to  thoughts 
aimed  at  helping  students  grasp  phenomena  of  nature 
while  looking  at  the  screen  does  not  automatically 
teach  the  child  to  verbalize  this  conceptual  knowledge. 
This  raises  an  important  question:  What  type  of  expe¬ 
rience  does  a  child  need  to  have  to  develop  an  under¬ 
standing  of  natural  phenomena? 


The  third  and  final  issue  that  we  wish  to  raise  with 
respect  to  discovery  learning  concerns  the  use  of 
microgenetic  analysis  to  gain  insight  into  learning  pro¬ 
cesses  related  to  conceptual  development  within  gam¬ 
ing.  Although  Hertzog  and  Klein  (2005)  suggested  that 
the  use  of  technology  such  as  PICCO  may  be  a  good 
vehicle  for  channeling  students’  creativity  and  critical 
thinking,  it  is  unclear  which  learning  processes  add  to 
the  formation  of  conceptual  knowledge  about  natural 
phenomena.  Another  question  is  whether  or  not  virtual 
discoveries  overrule  actual  experiences,  especially 
when  parent  involvement  may  be  the  mediating  vari¬ 
able  between  exposure  to  gaming  and  learning  from 
it.  We  suggest,  therefore,  that  the  microgenetic  work¬ 
ing  method  should  be  applied  more  often  to  attempt 
to  interpret  children’s  behavior  during  the  course  of 
gaming.  Variations  in  the  level  or  kind  of  pathways 
may  provide  insight  into  dynamic  variations  among 
children  (Granott  and  Parziale,  2002;  Steenbeek  and 
Van  Geert,  2002;  Tunteler  and  Resing,  2004). 

Conclusions  about  Science  Learning 

What  can  we  conclude  from  the  findings  with  respect 
to  science  learning  and  science  teaching?  Our  findings 
showed  that  the  higher  the  socioeconomic  status  of 
students  was,  the  more  they  profited  from  the  oppor¬ 
tunity  to  learn  from  a  program  that  allowed  investiga¬ 
tion  of  natural  phenomena.  As  a  consequence  a  main 
message  would  be  that  schools  need  to  offer  a  learning 
environment  that  favors  the  needs  of  children  of  fam¬ 
ilies  of  low  socioeconomic  status.  We  suggest,  how¬ 
ever,  first  observing  the  children’s  everyday  lives  at 
home  and  deciding  afterward  which  type  of 
approaches  or  enrichment  programs  would  best  suit 
specific  groups  of  children  (Tudge  and  Hogan,  2005). 


LEARNING  LAW:  THE  PROBLEMS 
WITH  ONTOLOGY  AND  REASONING 

Fleurie  Nievelstein,  Tamara  van  Gog,  and  Frans  J.  Prins 

Introduction  to  Learning  Law 

Before  addressing  the  difficulties  law  students  experi¬ 
ence  and  describing  research  on  effective  instructional 
interventions  and  necessary  future  research  in  this 
area,  we  shortly  describe  both  the  common  law 
(Anglo-Saxon)  and  the  civil  law  (Continental)  sys¬ 
tems,  along  with  providing  a  discussion  of  the  cogni¬ 
tive  processes  the  different  law  systems  evoke  and  the 
teaching  methods  currently  used. 
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Common  Law 

Common  law  is  largely  uncodified;  therefore,  prior 
cases  must  be  reviewed  to  determine  the  appropriate 
rules  (Sinclair,  1971).  It  is  characterized  by  the  fact 
that  judges  utilize  specific  knowledge  of  previously 
experienced,  concrete  problem  situations  (cases)  and 
the  outcomes  of  those  situations  to  build  their  judg¬ 
ments.  This  type  of  reasoning  in  common  law  is  called 
case-based  reasoning,  which  is  inductive  in  nature. 
Law  students  and  lawyers  should  be  able  to  recognize 
structural  similarities  between  the  current  and  prior 
cases  that  are  likely  to  differ  on  context-specific  (sur¬ 
face)  features  and  to  make  connections  between  those 
cases  to  infer  a  general  solution.  Case-based  reasoning 
is  a  cyclic  and  integrated  process  of  solving  a  problem, 
learning  from  this  experience,  and  solving  a  new  prob¬ 
lem  by  analogy  (Aamodt  and  Plaza,  1994). 

The  method  of  case-based  teaching  has  become 
the  primary  mode  of  legal  pedagogy  in  the  United 
States  and  the  United  Kingdom.  Thousands  of  legal 
cases  are  systematically  organized  and  available  for 
use  by  legal  educators.  Students  have  to  construct 
meaning  from  facts  that  may  at  first  seem  simple  but 
grow  increasingly  complex  if  they  explore  them  at 
some  depth  when  studying  a  particular  case.  Through 
case-based  teaching,  students  develop  skills  in  analyt¬ 
ical  thinking  and  reflective  judgment  by  reading  and 
discussing  complex,  real-world  examples  that  are  sim¬ 
ilar  to  situations  earlier  encountered  in  legal  practice 
(Aamodt  and  Plaza,  1994;  Carter  and  Unklesbay,  1989; 
Taylor,  2000) . 

Civil  Law 

Civil  law,  in  contrast,  is  codified  and  contains  a  whole 
set  of  rules  that  can  be  applied  and  interpreted  by 
judges.  The  judge’s  task  is  to  analyze  the  case,  search 
for  legal  rules,  and  interpret  these  rules  with  the  pur¬ 
pose  to  substantiate  a  solution  for  a  particular  case. 
Civil  law  relies  more  on  rule-based  reasoning,  which 
is  deductive  in  nature.  Rather  than  working  from  spe¬ 
cific  pieces  of  information  toward  a  general  conclu¬ 
sion,  one  is  working  from  general  rules  or  assertions 
toward  one  specific  conclusion  (Chalmers,  1976; 
Maughan,  1996).  Legal  problems  are  analyzed  in  the 
light  of  generally  accepted  rules,  from  which  a  solution 
can  be  deduced. 

A  variation  on  the  case-based  teaching  approach 
that  is  often  used  in  common  law  is  the  problem-based 
method  of  teaching  (Admiraal  et  al.,  1999;  Teich, 
1986),  in  which  students  analyze  and  solve  legal  prob¬ 
lems  by  finding  and  applying  relevant  principles  of 
law.  In  the  Netherlands,  Crombag  et  al.  (1977)  devel¬ 


oped  a  systematic  approach  to  problem  solving  that 
aims  at  solving  civil  cases  step  by  step.  This  didactic 
model  represents  an  ideal,  linear  sequence  of  opera¬ 
tions  an  experienced  lawyer  should  consider  to  reason 
about  a  certain  case.  To  solve  a  legal  case,  seven  steps 
have  to  be  taken:  (1)  formulation  of  the  problem,  (2) 
selection  of  legal  facts,  (3)  selection  of  relevant  law 
and  rules,  (4)  analysis  and  interpretation  of  these  rules, 
(5)  application  of  the  rules  on  the  case,  (6)  evaluation 
of  the  construction,  and  (7)  final  formulation  of  a  judg¬ 
ment.  During  case  analysis,  these  phases  are  not 
strictly  separated  but  form  a  cyclic  process. 

Difficulties  Learning  Law 

The  differences  between  the  two  law  systems  as  well 
as  the  corresponding  reasoning  approaches  are  obvi¬ 
ous;  however,  for  novice  learners  who  have  to  solve 
law  problems  in  either  law  system,  similar  difficulties 
arise.  Difficulties  arise,  on  the  one  hand,  because  a 
novice’s  knowledge  base  is  not  yet  well  developed 
and,  on  the  other  hand,  because  the  domain  entails 
some  very  complex  concepts  and  skills  that  must  be 
mastered. 

Knowledge  Structure 

To  solve  a  legal  problem,  available  information  must 
be  checked,  legally  interpreted,  and  completed,  if  nec¬ 
essary,  and  relevant  legal  information  (previous  cases 
or  rules)  must  be  selected.  Facts  of  meager  significance 
have  to  be  distinguished  from  the  ones  that  do  matter. 
This  is  even  more  complicated  because,  depending  on 
the  perspective  taken  (e.g.,  lawyer  of  the  plaintiff  or 
defendant),  different  facts  are  of  different  importance. 
From  research  conducted  on  surface  and  structural 
problem  features,  it  is  known  that  novices  are  often 
distracted  by  surface  features  (contextual)  and  have 
problems  identifying  the  structural  features  that  play 
a  crucial  role  in  determining  possible  solutions 
(Holyoak  and  Koh,  1987).  Effective  legal  reasoning 
requires  identification  and  interpretation  of  structural 
features  from  a  legal  perspective.  In  other  words,  ade¬ 
quate  determination  of  the  significance  of  facts 
requires  knowledge  of  rules  of  law  and  knowledge  of 
the  underlying  policies  of  a  case  (Vandevelde,  1996). 

Based  on  their  experience,  experts  have  formed 
scripts  that  help  them  to  recognize  and  solve  routine 
problems  (Custers  et  al.,  1998).  Findings  from  research 
on  expertise  differences  suggest  that  experts  in  a 
domain  use  mental  frameworks  or  schemata  that  con¬ 
tain  declarative  knowledge  (concepts  and  principles) 
to  understand  and  frame  a  problem  and  procedural 
knowledge  to  apply  principles  to  the  problem  (Chi, 
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2005;  Dufresne  et  al.,  1992;  Schank  and  Abelson, 
1977).  Crombag  et  al.  (1977)  compared  the  knowledge 
structures  of  students  and  an  expert  with  regard  to  one 
concept  of  private  law,  the  tort.  Participants  had  to 
order  29  different  private  law  concepts  around  this 
main  concept.  They  found  that  the  expert’s  conceptual 
system  of  this  legal  concept  consisted  of  specific  clus¬ 
ters  and  elements,  which  could  be  interpreted  as  fea¬ 
tures  of  a  tort  script.  In  contrast,  students’  knowledge 
structures  seemed  incomplete,  less  organized,  and  less 
easily  activated  than  the  expert’s  script.  Experts  are 
also  known  to  encapsulate  lower  level,  detailed  con¬ 
cepts  under  higher  level  concepts  (i.e.,  they  reach  a 
higher  level  of  abstraction)  (Boshuizen  and  Schmidt, 
1992).  During  work  on  routine  problems,  those 
abstract,  higher  level  concepts  are  automatically  acti¬ 
vated;  the  lower  level,  detailed  concepts  are  only  acti¬ 
vated  if  necessary  for  specific,  nonroutine  tasks.  In 
sum,  law  students  have  to  develop  scripts  and  encap¬ 
sulated  concepts  that  will  assist  them  in  recognizing 
structural  features  and  relevant  facts  in  cases. 

Alexander’s  (2003)  model  of  domain  learning 
(MDL)  describes  the  quantitative  and  qualitative 
knowledge  shifts  that  will  take  place  during  three 
stages  of  domain-specific  expertise  development:  (1) 
acclimation,  in  which  beginning  students,  who  have 
limited  and  fragmented  knowledge  that  cannot  easily 
be  activated,  become  familiar  with  the  domain;  (2) 
competence,  in  which  fragmented  knowledge  changes 
into  a  more  extensive,  coherent,  and  principled  whole; 
and  (3)  proficiency  or  expertise,  which  is  characterized 
by  a  very  extensive  and  well- structured  expert  knowl¬ 
edge  base.  In  this  final  stage,  individuals  are  able  to 
contribute  new  knowledge  to  the  domain,  and  as  a 
result  the  domain  keeps  on  growing  (Alexander,  2003). 

Ontology 

Apart  from  gaining  knowledge  of  cases  and  rules  and 
the  development  of  scripts,  novices  in  law  must  at  the 
same  time  familiarize  themselves  with  the  official  lan¬ 
guage  of  law  (Lindahl,  2004).  They  need  to  master  the 
ontology  — that  is,  a  shared  and  agreed  upon,  explicit, 
formal  representation  of  the  domain  (Bench-Capon 
andVisser,  1997;  Visser  and  Bench-Capon,  1998).  This 
can  be  complicated,  because  many  concepts  that  are 
routinely  used  in  everyday  informal  life  often  have  a 
very  specific  formal  meaning  that  may  only  partially 
coincide  with  the  everyday  meaning  (for  a  description 
of  similar  problems  in  the  domain  of  science,  see  Chi, 
2005;  Chi  and  Roscoe,  2002).  The  following  examples 
of  an  expert’s  and  a  novice’s  definition  of  the  concept 
of  owner  illustrate  this  point.  Asked  for  a  definition  of 
owner,  the  expert  says:  “Well,  an  owner  has  a  subjec¬ 


tive  right  of  ownership  on  certain  goods.  It  is  the  fullest 
right  someone  can  have  concerning  goods  and  s/he  is 
allowed  to  do  everything  s/he  wants  with  regard  to  the 
good.”  A  novice  says:  “An  owner  is  someone  who 
owns  something;  for  example,  if  you  have  a  dog,  it  is 
your  dog”  (Nievelstein  et  al.,  2005). 

Intermediate  Concepts 

An  additional  problem  for  learners  is  that  in  the 
domain  of  law  the  function  of  certain  concepts  can 
vary  depending  on  the  described  circumstances  (Ash¬ 
ley  and  Aleven,  1991).  Legal  knowledge  seems  to  be 
hierarchically  ordered,  and  several  concepts  in  such  a 
hierarchical  network  function  as  intermediate  concepts 
(Ashley  and  Briininghaus,  2003).  An  intermediate  con¬ 
cept  determines  the  relationship  between  legal 
grounds,  on  the  one  hand,  and  legal  consequences,  on 
the  other  hand,  and  can  have  different  functions 
depending  on  the  described  legal  context  (Lindahl, 
2003).  The  term  ownership ,  for  example,  can  lead  to 
a  certain  justification  ground  in  one  condition,  whereas 
the  same  term  can  lead  to  another  justification  ground 
in  another  situation.  Ownership  regarding  animals 
denotes  other  rights  and  responsibilities  than  owner¬ 
ship  regarding  a  house,  for  example. 

An  example  of  a  teaching  tool  making  use  of  inter¬ 
mediate  legal  concepts  is  an  artificial  model  for  pre¬ 
dicting  and  explaining  case  outcomes  developed  by 
Ashley  and  Briininghaus  (2003).  The  model  relies  on 
a  large  database  of  different  cases.  Every  case  repre¬ 
sents  a  dispute  between  a  plaintiff  and  a  defendant  and 
is  represented  by  factors  that  either  strengthen  a  plain¬ 
tiff’s  claim  or  a  defendant’s  response.  The  issue  of 
every  dispute  is  represented  as  an  intermediate  legal 
concept  and  is  placed  on  top  of  a  hierarchical  tree. 
Depending  on  both  the  circumstances  and  the  perspec¬ 
tive  taken  (i.e.,  the  role  of  the  plaintiff  or  defendant), 
the  intermediate  concept  fulfills  a  different  role  and 
leads  to  other  legal  consequences.  The  model  com¬ 
pares  a  new  case  with  cases  contained  in  the  database 
that  share  similar  structural  features  (i.e.,  comparable 
issues).  Whether  a  new  case  is  won  by  the  plaintiff  or 
by  the  defendant  is  predicted  and  explained  based  on 
these  comparisons 

Adversarial  Reasoning 

Legal  argumentation  plays  a  pivotal  role  in  law  and  is 
therefore  an  important  aspect  of  legal  education  (Carr, 
2003).  An  argument  can  be  defined  as  an  assertion  with 
an  accompanying  justification  (Kuhn,  1991).  One  dif¬ 
ficulty  of  argumentation  in  law  is  that  reasoning  is 
often  adversarial— that  is,  performed  in  a  context  of 
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debate  and  disagreement  that  requires  that  students 
learn  to  take  multiple  perspectives  to  place  their  own 
client  in  an  optimal  position.  A  legal  argument  that  is 
acceptable  in  itself  could  be  overturned  by  counterar¬ 
guments  and  exceptions.  The  aim  is  to  place  oneself 
in  an  optimal  position  compared  to  an  opponent  by 
anticipating,  understanding,  and  counteracting  the 
opponent’s  actions  (Thagard,  1992).  Although  rules, 
laws,  and  precedents  are  available,  the  adversarial  con¬ 
text  implies  that  their  application  is  never  a  matter  of 
straightforward  verification. 

Supporting  Law  Students 

The  above  section  shows  that,  when  learning  law,  dif¬ 
ficulties  arise  related  to  knowledge  development  in 
general  (acquiring  domain-specific  knowledge  and 
scripts)  and  to  the  domain  of  law  specifically  (devel¬ 
oping  an  ontology,  understanding  intermediate  con¬ 
cepts,  acquiring  argumentation  and  perspective-taking 
skills).  The  question  is  how  we  can  design  law  educa¬ 
tion  in  such  a  way  that  it  will  support  law  students, 
especially  early  on  in  their  educational  trajectory  (nov¬ 
ices),  to  overcome  those  difficulties. 

Research  on  Instructional 
Formats  in  Law 

Much  research  on  instructional  design  has  been  con¬ 
ducted  in  more  structured  domains  than  law;  however, 
some  instructional  formats  that  have  been  found  to  be 
effective  in  those  domains  have  also  been  investigated 
in  law  education.  In  this  section,  we  discuss  the  fol¬ 
lowing  instructional  formats:  (1)  worked-out  exam¬ 
ples,  (2)  a  tutoring  system  that  elicits  adversarial  rea¬ 
soning,  (3)  a  model  for  supporting  argumentation,  and 
(4)  a  method  using  cooperative  learning. 

The  effectiveness  of  presenting  students  with 
worked-out  examples  and  process  worksheets  has  been 
theoretically  argued  for  and  has  been  studied  in  (spe¬ 
cific  phases  of)  learning  how  to  prepare  a  plea  (Hum¬ 
mel  and  Nadolski,  2002;  Hummel  et  al.,  2004,  2006; 
Nadolski  et  al.,  2001).  Hummel  et  al.  (2004)  found 
that  students  scored  higher  on  a  training  plea  (i.e., 
retention)  after  having  studied  worked-out  examples 
but  found  no  effect  of  worked-out  examples  on  imme¬ 
diate  and  delayed  transfer.  Process  worksheets  posi¬ 
tively  affected  delayed  transfer. 

Ashley  and  Aleven  (1991)  developed  a  tutoring 
system  that  provided  practice  and  feedback  opportu¬ 
nities  for  multiple  perspective  taking  and  the  develop¬ 
ment  of  argumentation  skills,  which  are  both  necessary 
for  adversarial  reasoning.  This  system  teaches  students 
how  to  make  arguments  by  comparing  and  contrasting 


cases.  Each  case  is  based  on  a  dispute  between  a  plain¬ 
tiff  and  a  defendant.  By  facing  students  with  questions, 
alternative  interpretations,  rules,  counter  moves,  and 
examples  of  the  most  effective  or  the  best  move,  the 
system  teaches  them  to  distinguish  structural  case  fea¬ 
tures  and  to  reason  in  an  adversarial  context  where 
they  have  to  make  and  respond  to  arguments. 

Carr  (2003)  compared  arguments  of  two  groups  of 
second-year  law  students  on  a  final  exam.  One  group 
was  trained  to  create  arguments  for  legal  problems 
with  Toulmin’s  (1958)  model  of  argument  in  a  com¬ 
puter-supported  setting,  while  the  other  group  had  to 
create  arguments  for  the  same  problems  without  using 
this  tool.  The  hypothesis  that  students  trained  with  the 
model  of  argument  would  have  higher  scores  on  the 
final  exam  was  not  supported.  One  explanation  for  his 
findings  could  be  that  second-year  students  have  not 
yet  acquired  enough  domain  knowledge  to  benefit 
from  the  tool,  because  to  express  an  argument  in  a 
structured  way  knowledge  of  the  domain  is  requisite 
(Kuhn,  1991).  More  advanced  students,  then,  might 
have  benefited  from  the  tool. 

Henderson  and  Martin  (2002)  studied  cooperative 
learning  in  family  law.  Students  had  to  work  on  a 
relatively  general  assignment:  reviewing  cases  and 
writing  a  family  law  position  paper.  Their  findings 
showed  that,  for  this  assignment,  cooperative  learning 
resulted  in  improved  individual  and  group  achieve¬ 
ment,  in  more  critical  thinking  about  family  law,  and 
in  better  understanding  of  the  problems  in  law. 

Suggestions  for  Further 
Educational  Research  in  Law 

The  worked-out  examples  used  by  Hummel  et  al. 
(2004,  2006)  can  be  called  product  oriented  (van  Gog 
et  al.  2004),  as  they  focused  learners  on  solution  steps 
to  attain  an  end  product  (solution).  These  product- 
oriented,  worked-out  examples  did  not  affect  transfer. 
As  Hummel  et  al.  (2004)  noted,  process-oriented, 
worked-out  examples  might  have  improved  transfer 
but  were  not  investigated  in  that  study.  Process-ori¬ 
ented,  worked-out  examples  provide  both  the  solution 
steps  and  the  strategic  and  domain-principled  knowl¬ 
edge  used  in  selecting  the  steps.  In  other  words,  the 
systematic  approach  to  problem  solving  and  the 
accompanying  heuristics,  as  well  as  the  principled 
domain  knowledge  that  experts  use  in  their  reasoning 
process  toward  the  solution,  are  explicated  in  combi¬ 
nation  with  the  worked-out  solution  steps  provided 
(van  Gog  et  al.,  2004).  Using  such  process-oriented 
worked-out  examples  in  law  might  be  very  effective, 
as  they  essentially  provide  students  with  expert  models 
of  problem  solving  and  reasoning. 
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An  interesting  question  that  Ashley  and  Briining- 
haus  (2003)  did  not  address  was  whether  people  rec¬ 
ognize  intermediate  concepts  and  structural  features  of 
a  case  more  quickly  and  argue  better  when  they  are 
trained  with  the  principles  of  the  case-based  prediction 
algorithm.  Another  technique  that  might  assist  in  learn¬ 
ing  to  argue  and  in  learning  to  understand  the  different 
functions  of  intermediate  concepts  is  the  use  of  concept 
maps,  which  has  proven  effective  for  conceptual  learn¬ 
ing  in  science  education  (Roth,  1990;  Van  Boxtel  et  al., 
2000).  Students  in  this  domain  learned  to  assign  mean¬ 
ing  to  concepts  by  making  concept  maps  from  different 
points  of  view.  Using  this  technique  in  combination 
with  cooperative  learning  might  be  effective  in  law. 
During  group  sessions,  students  could  explain  different 
concept  arrangements  to  fellow  students  and  challenge 
each  other’s  opinions  about  their  explanations. 

Maughan  (1996)  contended  that  practice  in  authen¬ 
tic,  real-life  contexts  is  important,  especially  for  the 
acquisition  of  argumentation  and  adversarial  reasoning 
skills.  Fully  authentic  training  contexts  are  often  dif¬ 
ficult  to  achieve;  however,  authenticity  can  be  con¬ 
ceived  of  as  a  multidimensional  construct  (Gulikers  et 
al.,  2004),  and  high  authenticity  on  some  of  these 
dimensions  might  be  more  important  for  learning  cer¬ 
tain  skills  than  others.  A  study  of  which  dimensions 
should  have  a  high  degree  of  authenticity  may  generate 
guidelines  for  the  construction  of  training  situations 
(e.g.,  moot  courts;  see  Taylor,  2000).  Gulikers  et  al. 
have  also  shown  that  the  importance  of  authenticity  of 
different  dimensions  varies  with  expertise  levels; 
hence,  it  may  very  well  be  that  not  so  authentic  cog¬ 
nitive  tutors  (Ashley  and  Aleven,  1991)  are  especially 
effective  in  the  early  stages  of  argumentation  skill 
acquisition  and  that  practice  in  contexts  with  a  higher 
degree  of  authenticity  becomes  increasingly  important 
with  increasing  skill  development. 

It  may  also  be  helpful  to  investigate  ways  to  sup¬ 
port  law  teachers,  as  they  will  in  turn  support  students. 
Research  on  expert-laypeople  communication  has 
shown  that  experts  often  experience  difficulties  with 
accurately  estimating  novices’  level  of  knowledge  and 
adapting  their  explanations  to  that  level  (Bromme  et 
al.,  2001;  Niickles  et  al.,  2005).  Although  law  teachers 
should  be  trained  in  recognizing  students’  knowledge 
levels  and  adapting  to  them,  university  teachers  in 
particular  are  also  domain  experts,  so  expert-laypeople 
communication  problems  might  also  apply  to  law 
teachers.  If  this  is  the  case,  educational  research  should 
also  be  targeted  toward  improving  teachers’  under¬ 
standing  of  development  of  ontology  and  conceptual 
understanding,  on  the  one  hand,  and  providing  them 
with  tools  to  accurately  assess  the  level  of  development 
of  their  students,  on  the  other  hand. 


Discussion 

This  section  has  shown  that,  even  though  common  law 
and  civil  law  evoke  somewhat  different  learning  and 
teaching  processes,  students  experience  similar  difficul¬ 
ties  in  learning  law.  Those  difficulties  are  largely  related 
to  specific  facets  of  this  domain  (ontology,  intermediate 
concepts,  and  adversarial  reasoning)  that  make  knowl¬ 
edge  and  skill  acquisition  more  complex  than  in  other 
professional  domains .  A  major  challenge  for  educational 
researchers  and  instructional  designers  lies  in  identifying 
techniques  that  adequately  support  students’  integrated 
knowledge  and  skill  development  at  different  stages  in 
their  educational  trajectories.  Surprisingly,  compared  to 
other  complex  professional  domains  such  as  medicine, 
research  on  learning  and  instruction  in  the  domain  of 
law  is  sparse.  Some  interesting  previous  studies  have 
been  described  here,  and  an  attempt  has  been  made  to 
identify  promising  directions  for  future  research  in  this 
domain  that  will  result  in  instructional  formats  and  prin¬ 
ciples  that  will  help  students  overcome  the  difficulties 
they  encounter  and  foster  their  expertise  development  as 
much  as  possible  during  their  formal  education. 

LEARNING  MEDICINE:  A  CONTINUUM 
FROM  THEORY  TO  PRACTICE 

Tim  Dornan,  Albert  Scherpbier,  and  John  Spencer 

Introduction  to  Learning  Medicine 

The  medical  profession  is  founded  on  a  moral  code. 
Its  primary  educational  task  is  to  encourage  members 
of  successive  generations  to  uphold  the  code  and  to 
pass  the  code  on  to  learners  coming  after  them.  Per¬ 
petuation  of  this  value  system  gives  education  a  central 
place  in  the  profession  which  has  been  affirmed  repeat¬ 
edly  from  Hippocrates  (Lyons  and  Petrucelli,  1987)  to 
the  present  time  (General  Medical  Council,  1999). 
That  much  is  easy,  but  changes  in  the  nature  of  clinical 
practice,  educational  practice,  and  the  social  order  are 
making  medical  education  an  increasingly  complex 
domain.  This  section  concentrates  on  undergraduate 
medical  education.  It  links  special  features  of  the 
domain  to  factors  that  are  changing  it  and  recently 
devised  solutions.  It  uses  examples  from  the  United 
Kingdom  to  illustrate  general  points. 

Special  Features  of  the  Domain 

General  Description 

Entrants  come  fresh  from  high  school  or  as  mature 
students  with  life  experience  and  perhaps  a  prior 
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TABLE  41.1 

Disciplines  Taught  in  Medical  School 

More  theoretical  disciplines;  for  example: 

Physiology,  anatomy,  and  chemistry  of  the  body  tissues  and  systems 
Chemistry  of  drugs  and  their  actions 
Sciences  of  individual  and  social  behavior 
More  applied  disciplines;  for  example: 

Pathology 
Public  health 

Individual  disciplines  into  which  medicine  is  organized;  for  example: 
Internal  medicine 
Surgery 

General  practice 


degree.  Medical  training  takes  4  to  6  years,  depending 
on  the  country  and  the  student’s  previous  education. 
High  academic  standards  are  required  for  admission, 
and  students  must  be  highly  motivated  because  medi¬ 
cal  study  is  arduous.  Entrants  with  no  science  back¬ 
ground  may  be  required  to  take  an  access  course  to 
ground  them  in  the  bioscientific  vocabulary  of  medi¬ 
cine.  In  addition  to  being  ambitious,  medical  students 
tend  to  be  altruistically  motivated.  At  the  beginning, 
they  learn  biomedical  science  and  perhaps  behavioral 
and  social  science  and  humanities  but  have  minimal 
workplace  experience.  In  later  years,  the  focus  shifts 
from  theory  to  practice,  and  they  gain  extensive  clin¬ 
ical  workplace  experience.  After  graduating,  new  doc¬ 
tors  take  responsibility,  under  supervision,  for  the  care 
of  patients  but  have  to  undergo  postgraduate  training 
before  they  can  practice  unsupervised.  The  aim  of 
undergraduate  medical  education,  then,  is  to  produce 
graduates  who  can  enter  the  workforce  and  receive  the 
postgraduate  training  that  will  make  them  independent 
practitioners. 

Subject  Matter  and  Learning  Outcomes 

The  wide,  long,  and  deep  competences  (Hamilton, 
1999)  that  underpin  medical  practice  place  a  huge 
burden  on  learners.  They  are  drawn  from  the  disci¬ 
plines  listed  in  Table  41.1.  The  U.K.  General  Medical 
Council's  definition  of  the  duties  of  a  doctor  is  shown 
in  Table  41 .2  (General  Medical  Council,  1993).  It  takes 
technical  competence  as  a  given  and  concentrates  on 
the  social  responsibilities  of  a  doctor;  however,  most 
of  a  medical  student’s  education  is  taken  up  with 
acquiring  technical  rather  than  social  competence,  and 
knowledge  acquisition  takes  pride  of  place. 

The  espoused  knowledge  of  the  domain  and 
method  of  learning  vary  in  character  from  one  medical 
discipline  to  another.  Anatomy,  for  example,  is  often 


TABLE  41.2 

The  Duties  of  a  Doctor,  as  Defined  by  the  U.K. 
General  Medical  Council 

Doctors  should: 

Make  the  care  of  their  patient  their  first  concern. 

Treat  every  patient  politely  and  considerately. 

Respect  patients’  dignity  and  privacy. 

Listen  to  patients  and  respect  their  views. 

Give  patients  information  in  a  way  they  can  understand. 

Respect  the  rights  of  patients  to  be  fully  involved  in  decisions  about 
their  care. 

Keep  their  professional  knowledge  and  skills  up  to  date. 
Recognize  the  limits  of  their  professional  competence. 

Be  honest  and  trustworthy. 

Respect  and  protect  confidential  information. 

Make  sure  their  personal  beliefs  do  not  prejudice  their  patients’  care. 
Act  quickly  to  protect  patients  from  risk  if  they  have  good  reason 
to  believe  that  they  or  a  colleague  may  not  be  fit  to  practice. 
Avoid  abusing  their  position  as  a  doctor. 

Work  with  colleagues  in  the  ways  that  best  serve  patients’  interests. 

Source:  General  Medical  Council,  Duties  of  a  Doctor,  General 
Medical  Council,  London,  1993. 


rote  learned  in  preparation  for  tests  of  factual  recall 
because  the  true  science  of  human  form  is  tied  up  in 
sciences  of  human  embryology  and  evolution  that  stu¬ 
dents  simply  do  not  have  time  to  study  in  any  depth. 
Biochemistry  and  physiology  are  taught  as  experimen¬ 
tal  sciences,  although  it  is  left  to  students  and  curric¬ 
ulum  planners  to  find  clinical  relevance  in  them 
because  science  teachers  are  not  medical  practitioners. 
In  addition  to  the  official  curriculum  (Coles,  1998),  an 
unofficial  curriculum  covers  a  very  different  type  of 
know-how,  including  such  things  as  time  management, 
how  to  behave  in  practice  settings,  role  and  hierarchy, 
and  career  development  (Sinclair,  1997).  Turning  to 
the  skills  domain,  medical  students  must  acquire  the 
study  skills  of  a  learner  and  the  psychomotor  and  com¬ 
munication  skills  of  practice.  Table  41.3  shows  a  clas¬ 
sification  of  clinical  skills  (Doman  et  al.,  2003).  There 
is  also  a  tacit  curriculum  of  complex  social  skills 
required  for  success  in  the  workplace.  The  term  atti¬ 
tudes  is  used  loosely  to  cover  the  several  types  of 
learning  listed  in  Table  41.4  that  sustain  the  learner 
during  their  studies  and  help  them  socialize  profes¬ 
sionally.  These  social  and  emotional  aspects  of  medical 
education  are  further  considered  later  in  this  section. 

Doctors-in-waiting  have  to  integrate  all  of  this 
learning  into  safe,  fluent,  and  spontaneous  professional 
performance;  a  medical  emergency  could,  for  example. 
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TABLE  41.3 

A  Classification  of  Clinical  Skills 


Type  of  Skill 
Consultation 
Physical  examination 
Image  interpretation 


Procedure 

Therapeutic 

Management 

Laboratory 

Patient  and  staff  safety 


Description 

Effective  transaction  with  a  patient  to  provide  medical  care 
Examining  a  patient  to  make  a  diagnosis 
Examining  a  radiograph  or  electrocardiogram  systematically, 
detecting  abnormalities,  and  relating  those  findings  to 
knowledge  of  health  and  disease 
Incrementing  a  patient’s  body  for  diagnostic  or  therapeutic 
purposes;  for  example,  taking  blood 
The  procedural  and/or  intellectual  act  of  safely  administrating 
a  complex  treatment  like  a  blood  transfusion 

For  example,  coordinating  the  various  services  that  will  look 
after  a  patient  when  they  are  discharged  from  hospital 

Handling  specimens  safely  and  conducting  simple  analyses  on 
body  fluids 

Hand  washing  and  other  ways  of  preventing  transmission  of 
communicable  diseases 


TABLE  41.4 

Learning  Outcomes  Often  Described  as  Attitudes 

Knowledge  of: 

The  professional  value  system 

How  the  legal  and  ethical  frameworks  of  medicine  have  to  be  put  into  practice 
Personal  qualities,  such  as: 

Confidence 

Motivation 

Probity/integrity 

Sense  of  professional  identity 

Self-awareness 


call  for  knowledge  of  the  different  uniforms  of  the 
nursing  hierarchy,  knowledge  of  the  biochemical 
effects  of  a  patient’s  drugs,  the  intellectual  skill  of 
applying  the  information  to  analyze  the  problem,  and 
the  emotional  and  social  skills  of  moderating  interac¬ 
tions  between  the  patient  and  nurse  in  the  heat  of  the 
emergency. 

Educational  Processes 

Traditionally,  medical  students  have  acquired  knowl¬ 
edge  by  attending  lectures  and  practical  classes  and  by 
learning  from  books.  When  the  emphasis  of  their  learn¬ 
ing  shifts  from  preclinical  theory  to  clinical  practice, 
they  learn  from  practitioners  in  practice  settings  (Sin¬ 
clair,  1997).  Some  attitudinal  learning  is  projected  onto 
medical  students  by  didactic  teachers,  acquired  as 
knowledge,  or  embedded  in  the  skills  they  learn,  but 


an  important  influence  on  attitudinal  development  is 
the  process  of  role  modeling  (Elzubeir  and  Rizk,  2001 ; 
Tiberius  et  al.,  2002).  Research  has  consistently  shown 
that  medical  students  and  young  doctors  identify  as 
role  models  practitioners  who  are  clinically  excellent, 
behave  compassionately  toward  patients,  teach  dili¬ 
gently,  and  focus  on  doctor-patient  communication 
and  patients’  psychosocial  problems  (Wright  et  al., 
1998).  Skills  such  as  conducting  physiological  exper¬ 
iments  in  the  early  curriculum  are  a  means  to  the  end 
of  becoming  knowledgeable.  Skills  learned  later  in 
workplaces,  such  as  obtaining  a  complete  history  and 
conducting  a  full  examination  to  replicate  the  behavior 
of  a  doctor,  are  both  a  means  and  an  end.  Practical 
skills  are  not  just  something  doctors  have  to  be  able 
to  perform  but  are  a  means  for  students  to  integrate 
into  the  practicing  clinical  team,  which  is  vital  to  their 
clinical  learning  (Dornan,  2006). 
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TABLE  41.5 

Pressures  toward  Graduate  Entry 


Widened  participation 


Need  for  more  doctors 

Personal  attributes 


Fitness  for  medical  study 


To  be  equitable  and  balance  the  social  mix  of  the  medical  profession, 
there  is  a  wish  to  give  disadvantaged  teenagers  the  chance  to  enter 
the  medical  profession  later  in  life. 

There  is  pressure  to  admit  students  who  can  be  turned  quickly  into 
the  extra  doctors  society  needs. 

Their  greater  life  experience  and  the  motivation  that  brought  them 
into  higher  education  at  this  late  stage  make  them  a  particularly 
valued  type  of  student  for  medical  school. 

Evidence  suggests  that  mature  entrants  make  a  smoother  transition 
from  classroom  to  workplace  learning  and  grow  in  academic 
strength  when  nonmature  entrants  have  reached  a  plateau. 


Today’s  medical  education  was  conceived  at  a  time 
when  most  practice  was  in  hospitals,  where  patients 
were  admitted  for  long  periods  of  time,  often  with 
grossly  apparent  disease  that  medicine  could  not  cure. 
The  tradition  was  for  medical  students  to  be  attached 
to  a  senior  clinician  and  his  postgraduate  trainee  doc¬ 
tors  based  in  a  single  hospital  ward.  Traditional  clinical 
apprentices  functioned  as  the  most  junior  members  of 
medical  teams,  admitting  patients  to  their  teacher’s 
ward,  carrying  out  menial  tasks,  and  providing  an  extra 
pair  of  hands  for  surgical  operations.  Medicine  has  a 
rich  assessment  tradition.  Knowledge  was  traditionally 
assessed  through  essays,  viva  voce  examinations,  and 
cross-examination  at  the  bedside.  Practical  compe¬ 
tence  was  tested  by  asking  questions  about  a  patient 
the  candidate  had  spent  some  time  examining  or  by 
observing  the  candidate  examining  several  patients 
with  externally  detectable  evidence  of  disease  (so- 
called  long  and  short  case  exams,  respectively).  Such 
assessments  were  often  adversarial  in  nature  and  con¬ 
fronted  students  with  exotic  manifestations  of  disease 
rather  than  commonplace  situations  they  would  have 
to  deal  with  in  practice. 

Special  Themes  in  Medical  Education 

Medical  education  has  to  attend  to  medical  students’ 
social  development  as  much  as  equip  them  with  spe¬ 
cific  competences.  It  has  to  usher  them  through  a  pro¬ 
cess  of  moral  development  (Branch,  2000),  when  their 
idealism  is  challenged  and  they  feel  pressure  to  “adopt 
a  ritualized  professional  identity”  (Lempp  and  Seale, 
2004).  It  must  help  them  learn  how  to  handle  strong 
emotions  stemming  as  much  from  the  process  of  pro¬ 
fessional  acculturation  as  from  contact  with  disease 
and  distress,  sustain  their  motivation,  build  the  confi¬ 
dence  that  enables  them  to  interact  with  practitioners. 


and  develop  a  sense  of  professional  identity  (Doman, 
2006).  There  are  strong  elements  of  risk  in  medical 
education.  Junior  medical  students  are  painfully  aware 
that  blurting  out  the  word  “cancer”  within  a  patient’s 
hearing  can  do  irrevocable  harm  to  the  object  of  their 
motivation,  as  could  any  failings  in  the  practical  tasks 
of  medicine.  Students  could  contract  a  potentially  fatal 
communicable  disease  from  a  patient,  fail  in  the  com¬ 
petitive  world  of  practice,  or  become  the  object  of  a 
lawsuit. 

Pressures  for  Change  in  Medical  Education 

The  demand  for  more  doctors  is  overcrowding  medical 
schools  and  putting  pressure  on  traditional  clinical 
apprenticeship.  Undergraduate  entry  systems  are  being 
pushed  toward  mature/graduate  entry  by  the  various 
factors  listed  in  Table  41.5.  Having  more  mature  stu¬ 
dents,  in  its  turn,  pushes  medical  schools  to  adopt 
systems  of  education  that  are  more  respectful  of  learn¬ 
ers’  existing  attributes.  Medical  practice  in  developed 
countries  is  changing  in  many  ways  that  impact  on 
medical  education  (see  Table  41.6). 

Medicine  is  caught  in  a  more  general  tension 
between  academia’s  traditional  view  of  education  as 
scholarship  and  the  profession’s  increasing  call  for 
doctors  who  are  fit  for  what  Schon  described  as  the 
“swampy  lowlands”  of  uncertainty  and  real-world 
practice  (Schon,  1983).  It  has  also  been  strongly 
affected  by  the  self-directed  learning  movement 
(Knowles,  1975),  which  challenged  both  traditional 
didacticism  and  the  hierarchical  relationship  between 
clinical  master  and  apprentice  and  resulted  in  wide¬ 
spread  adoption  of  learner-centered  methods.  Medi¬ 
cal  education  as  it  was  practiced  until  very  recently 
gave  students  a  fairly  pure  grounding  in  science  fol¬ 
lowed  by  opportunities  to  put  theory  into  practice  in 
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TABLE  41.6 

Changes  in  the  Nature  of  Practice  That  Affect  Medical  Education 


Change 

Shorter  lengths  of  hospital  stay  and  more  care  out  of  hospital 

Stronger  emphasis  on  disease  prevention  and  health  promotion,  as  well 
as  on  long-term  management  of  chronic  disease 

Strong  emphasis  on  patient  safety  and  mitigation  of  clinical  risk 

A  change  in  medicine’s  traditionally  privileged  status  with  the  rise  of 
professions  allied  to  medicine  and  more  flexible  use  of  skill  mix 

Empowerment  of  lay  people 

Increasingly  technical,  specialized  nature  of  practice  and  proliferation 
of  clinical  disciplines 

Reappraisal  of  the  nature  of  medical  professionalism  in  the  context  of 
a  new  social  compact  between  the  profession  and  society 

Advances  in  science  such  as  the  “new  genetics”  and  molecular  biology 


Effect  on  Education 

Favors  community-based  education  and  new  methods  and  locations  or 
settings  for  teaching  in  hospitals 

Changes  the  subject  matter  and  focus  of  medical  curricula 

Eliminates  learning  by  trial  and  error  and  makes  it  more  difficult  for 
learners  to  cross  the  threshold  into  practice 

Changes  the  emphasis  of  medical  education  towards  interprofessional 
and  team  learning;  provides  a  rationale  for  interprofessional  education 

Shifts  the  subject  matter  from  learning  outcomes  that  matter  primarily 
to  pedagogues  and  calls  for  explicit  training  in  patient-centered 
medical  practice 

Increases  curriculum  content;  makes  it  more  difficult  to  learn 
holistically  and  to  cover  a  full  breadth  of  curriculum  content  in  any 
one  discipline 

Increases  curriculum  content,  particularly  in  areas  such  as  ethics  and 
communication,  and  the  underpinning  social  and  humanistic  sciences 

Increases  curriculum  content 


apprenticeship  attachments;  however,  the  many  forces 
listed  in  Table  41.7  are  broadening  and  changing  the 
epistemology  of  medical  education. 

As  developed  societies  have  become  more  egali¬ 
tarian  and  lay  people  more  empowered,  doctors’  priv¬ 
ileged  social  position  and  sometimes  insensitive  or 
paternalistic  behavior  have  been  challenged.  The  result 
has  been  to  restate  medicine’s  value  system  within  a 
new  framework  that  emphasizes  the  subservient  place 
of  the  medical  profession  (General  Medical  Council, 
2002;  Working  Party  of  the  Royal  College  of  Physi¬ 
cians,  2005). 

Solutions  and  Unanswered  Questions 

While  changes  in  practice  have  eroded  the  traditional 
relationship  between  master  and  apprentice,  educa¬ 
tional  theory  has  shifted  its  focus  from  teachers  to 
learners  and  the  way  they  regulate  their  learning  (Ten 
Cate  et  al.,  2004).  This  new  focus  is  a  two-edged  sword. 
The  egalitarian  relationship  between  a  group  of  self- 
directed  learners  and  a  problem-based  learning  facili¬ 
tator  is  an  effective  way  of  teaching  the  declarative 
knowledge  of  medicine  (Prince  et  al.,  2003);  however, 
other  types  of  learning  depend  on  a  relationship 
between  learner  and  teacher  that  is  “inherently  asym¬ 
metrical”  (Tiberius  et  al.,  2002).  To  acquire  psychomo¬ 
tor  skills,  medical  students  have  to  be  trained  by  mas¬ 
ters  (Patrick,  1992).  To  acquire  clinical  expertise,  they 
must  socialize  into  communities  of  practice  in  which 


their  teachers  are  masters  (Swanwick,  2005;  Wenger, 
1998).  We  found  that  a  move  toward  self-directed 
learning  in  the  workplace  disenfranchised  teachers  and 
left  learners  confused  and  purposeless  (Dornan  et  al., 
2005a,b).  Research  is  needed  into  the  roles  students 
should  adopt  within  communities  of  clinical  practice. 
Patients  are  increasingly  recognized  as  active  partici¬ 
pants  rather  than  passive  recipients  of  medical  care,  so 
their  role  in  contemporary  medical  education  has  also 
to  be  researched  (Spencer  et  al.,  2000). 

Outcome-Based  Education  and  Subject  Matter 

Until  recently,  medical  curricula  would  have  been 
described  in  terms  of  the  disciplines  that  taught  them. 
Now,  they  have  explicit  learning  outcomes,  usually 
expressed  as  competences.  Although  it  is  self-evidently 
important  to  have  clear  goals,  the  supposition  that  med¬ 
ical  expertise  can  be  reduced  to  a  set  of  competences 
is  questionable.  Experience  and  tacit  knowledge  have 
clearly  defined  places  in  clinical  expertise  (Schmidt  et 
al.,  1990;  Wyatt,  2001)  but  are  difficult  to  express  as 
competences.  Even  separating  educational  process  and 
product  is  questionable  when  students  learn  by  form¬ 
ing  relationships  with  practitioners.  The  current 
emphasis  on  professionalism  ensures  that  the  subject 
matter  of  medical  curricula  is  expressed  in  wider  terms 
than  bioscience.  Social  awareness,  ethical  behavior, 
good  communication,  and  effective  resource  manage¬ 
ment  are  all  to  be  expected  of  21st-century  doctors.  To 
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TABLE  41.7 

Epistemological  Changes  and  Their  Effect  on  Medical  Education 


Changes  in  the  Epistemology  of  Medical  Education 

A  state  of  maturity  in  science  that  changes  the  doctor  from  a  pioneer 
of  untested  solutions  to  implementer,  within  social  settings,  of  mature 
technology 

The  demand  in  societies  for  doctors  to  show  social  qualities  that  are 
neither  theoretically  nor  practically  grounded  in  biomedical  science 

The  rise  of  behavioral  and  social  sciences  that  are  theoretically  coherent 
and  have  the  power  to  explain  and  provide  solutions  to  a  variety  of 
issues  of  health  and  disease 

Entrant  practitioners’  experience  of  finding  themselves  poorly  prepared 
for  their  professional  roles  by  a  theory-before-practice,  science-based 
education 

Acknowledgment  that  modern  medicine  has  many  faces:  medicine  as 
science,  medicine  as  humanity,  medicine  as  practical  know-how,  and 
medicine  as  social  savoir  fair e 


Effects 

Medical  education  must  adapt  to  the  role  of  doctor  as  a  social 
rather  than  scientific  agent  of  change 

Medical  education  must  choose  to  either  provide  an 
education  that  is  not  grounded  in  theory  or  adopt  new 
foundation  disciplines,  including  humanities 

Provide  new  subject  matter  for  a  medical  student’s 
theoretical  education 

Shift  at  least  in  the  sequence,  and  perhaps  in  the  theoretical 
content,  of  medical  curricula 

Explicit  adoption  of  a  wide  epistemology  of  practice  rather 
than  giving  science  preeminence  and  leaving  other  aspects 
of  learning  to  the  unofficial  curriculum 


fulfill  their  social  responsibilities,  medical  students 
must  acquire  emotional  learning  outcomes  such  as  con¬ 
fidence,  motivation,  and  a  professional  identity  (Dor- 
nan,  2006)  .  In  addition,  there  are  introspective  qualities 
that  are  difficult  to  define  because  they  are  so  individ¬ 
ual  yet  are  easy  to  recognize.  One  author  coined  the 
term  mindfulness  to  describe  the  insight  that  allows  a 
practitioner  to  generalize,  tackle  future  challenges, 
incorporate  new  behaviors  and  attitudes,  express  com¬ 
passion,  and  “be  present”  (Epstein,  1999).  Mindfulness 
cannot  be  taught,  but  it  can  be  developed  through  ques¬ 
tioning,  reflection,  mentoring,  and  role  modeling— lit¬ 
erally,  an  education  or  a  “bringing  out.”  Better  defini¬ 
tion  of  the  personal  qualities  of  a  doctor,  the  ways  in 
which  those  qualities  can  be  developed,  and  their  rela¬ 
tion  to  constructs  such  as  emotional  intelligence  are 
important  topics  for  future  research. 

Curriculum  Design 

Integration  is  a  dominant  theme  in  contemporary  med¬ 
ical  education.  Rather  than  a  discontinuous  sequence 
of  educational  experiences,  each  preparing  for  the 
next,  contemporary  medical  education  seeks  progres¬ 
sively  to  shape  the  student  entrant  as  a  doctor.  Abrupt 
transitions  delay  students'  learning,  including  their 
moral  development  (Branch,  2000).  Progressive  expo¬ 
sure  to  workplaces  alongside  theoretical  education 
helps  them  learn  practically,  cognitively,  socially,  and 
emotionally.  Vertical  integration,  it  can  be  argued,  is  a 
socially  responsive  curriculum  design  (Littlewood  et 
al.,  2005).  There  is  also  a  call  for  curricula  to  be 


horizontally  integrated  — that  is,  cover  a  logical 
sequence  of  subject  matter  organized  around  body  sys¬ 
tems  rather  than  individual  disciplines.  This  works  less 
well  for  workplace  learning  than  for  the  theory  of 
medicine  .  The  expertise  that  teachers  are  keen  to  share 
with  students  is  so  bound  up  in  their  disciplinary  prac¬ 
tice  that  horizontal  integration  of  workplace  learning 
causes  serious  confusion  (Patel  et  al.,  2002).  The  rel¬ 
ative  merit  of  integrated  vs.  discipline-based  work¬ 
place  learning  is  an  important  topic  for  future  research. 

Pedagogic  Methods  and  Assessment 

Problem-based  learning  has  been  one  of  medical  edu¬ 
cation’s  most  important  contributions  to  higher  educa¬ 
tion.  Originally  conceived  as  a  way  to  teach  transferable 
problem-solving  skills  (Barrows,  1984),  it  is  a  proven 
effective  means  of  shifting  regulation  of  education  from 
teacher  to  learner  (Schmidt,  1993).  Critics  have  pointed 
out  that  problem-based  learning  does  not  make  medical 
students  more  knowledgeable  than  traditional,  didactic 
approaches  (Berkson,  1993),  overlooking  evidence  that 
it  develops  more  appropriate  learning  skills,  builds 
social  skills,  and  is  more  enjoyable  (Albanese  and 
Mitchell,  1993;  Vernon  and  Blake,  1993). 

Concern  for  the  safety  of  the  student  and  patient 
led  to  the  development  of  various  simulation  tech¬ 
niques  and  the  widespread  introduction  of  clinical 
skills  laboratories.  Interpersonal  communication  is 
trained  and  tested  using  simulated  patients  (Van  Dalen 
et  al.,  1999).  Psychomotor  skills  are  taught  using  ana¬ 
tomical  models,  while  more  complex  procedures  use 
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high-fidelity  simulation  technologies  (Issenberg  et  al., 
2005).  How  to  promote  the  transfer  of  simulation  edu¬ 
cation  to  practice  settings  is  another  important  topic 
for  future  research. 

Most  research  into  clinical  workplace  learning  is 
so  contextually  bound  that  there  is  a  need  for  new, 
more  generalizable  conceptions.  We  have  offered 
experience-based  learning  as  one  such  conception. 
According  to  this  model,  the  central  condition  for 
workplace  learning  is  supported  participation  in  prac¬ 
tice.  As  learners  become  more  experienced,  they 
progress  along  a  scale  from  observer  to  participant, 
culminating  in  performance  for  the  patients’  benefit. 
Supported  participation  results  in  emotional  and  prac¬ 
tical  learning,  which  reinforce  one  another.  Learning 
is  self-reinforcing  because  successful  learning  makes 
it  easier  for  the  learner  to  be  a  participant.  The  quality 
of  supported  participation  is  strongly  influenced  by 
qualities  of  the  learning  environment,  predominant 
among  which  is  the  relationship  between  practitioners 
and  learners.  This  theory  provides  a  basis  for  continu¬ 
ing  research  into  clinical  workplace  learning. 

Much  intensive  research  into  assessment  has  been 
conducted.  In  the  domain  of  medical  knowledge, 
progress  testing  deserves  special  mention  (Van  der 
Vleuten  et  al.,  1996).  Introduced  as  a  complement  to 
problem-based  learning  and  a  way  of  discouraging 
students  from  “cramming  and  forgetting,”  this  tech¬ 
nique  tests  students’  longitudinal  growth  of  knowledge 
referenced  to  their  peer  group.  Another  form  of  assess¬ 
ment  developed  in  medicine  is  the  Objective  Struc¬ 
tured  Clinical  Examination  (OSCE)  (Harden  et  al., 
1975).  The  OSCE  assesses  students  on  a  variety  of 
tasks,  covering  a  wide  range  of  subject  matter.  It  has 
been  widely  adopted  as  a  way  of  reducing  bias  and 
overcoming  the  problem  of  content  specificity  in  prac¬ 
tical  examinations. 

Evaluation  — measuring  the  quality  of  educa¬ 
tion— is  so  closely  linked  to  assessment  that  the  two 
words  are  synonymous  in  U.S.  English;  however,  the 
assumption  that  examination  performance  is  a  valid 
measure  of  the  quality  of  a  curriculum  is  questionable 
for  two  main  reasons.  First,  students  learn  to  the  test. 
They  strive  to  overcome  the  deficiencies  of  poor  cur¬ 
ricula  by  studying  privately,  very  likely  at  the  cost  of 
learning  outcomes  that  are  not  measured.  Second, 
using  assessment  to  measure  curriculum  quality  sup¬ 
poses  that  the  essential  qualities  of  a  curriculum  are 
assessable.  Because  pass-fail  decisions  have  to  be 
made  on  reliable  data,  assessments  have  tended  to 
focus  on  knowledge  or  skills  that  can  be  tested  in 
highly  controlled  conditions.  So,  another  important 
field  of  contemporary  research  is  authentic  assessment, 
which  takes  assessment  into  workplace  settings. 


Conclusion 

Medicine  as  a  learning  domain  is  characterized  by  risk 
to  the  learner  and  patient,  a  knowledge -rich  education 
that  has  to  prepare  learners  to  handle  complex  prob¬ 
lems  immediately  after  qualification,  strong  emotional 
forces,  and  a  close  professional  relationship  between 
teacher  and  learner  that  gives  education  a  high  priority 
in  the  value  system  of  the  profession. 
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Part  V 

Design  and  Development 


This  fifth  part  of  the  Handbook  was  led  by  M.  David 
Merrill  and  focuses  on  research  that  pertains  to  pro¬ 
fessional  practice.  The  emphasis  in  this  part  of  the 
Handbook  is  on  research  that  examines  what  instruc¬ 
tional  technologists  do,  how  they  work,  and  the  tools 
they  use.  This  part  of  the  Handbook  consists  of  11 
chapters  covering  these  topics:  (1)  instructional 


design  competencies,  (2)  task  analysis,  (3)  perfor¬ 
mance  assessment,  (4)  evaluation  models  and  meth¬ 
ods,  (5)  change  agency,  (6)  design  languages,  (7) 
design  and  development  teams,  (8)  user-centered 
design,  (9)  design  and  development  tools,  (10)  design 
and  development  artifacts,  and  (11)  systems  design 
for  change. 
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ABSTRACT 

This  chapter  provides  an  analysis  of  current  competen¬ 
cies  (i.e.,  knowledge,  skills,  and  attitudes)  that  are  nec¬ 
essary  to  achieve  effective  instructional  design  and 
development  outcomes  with  specific  focus  on  contem¬ 
porary  technology-mediated  educational  applications. 
Referring  to  current  competency  standards  and  models, 
such  as  those  devised  by  ibstpi®,  this  chapter  elaborates 
on  those  competencies  essential  for  supporting  the  inte¬ 
gration  of  hardware  and  software  technologies  (mobile 
devices,  personal  blogs,  multiplayer  environments)  into 
collaborative  instructional  networks.  In  addition,  the 
analysis  identifies  recent  models  of  design  that  provide 


a  framework  for  educational  professionals  to  ensure 
that  they  have  the  knowledge,  skills,  and  attitudes  to 
effectively  manage  the  challenges  and  utilize  the  affor- 
dances  of  educational  technology  in  today’s  complex 
social  settings.  By  understanding  these  shifts,  instruc¬ 
tional  designers  will  appreciate  the  expanding  and 
changing  nature  of  their  role. 

KEYWORDS 

Competency :  A  knowledge,  skill,  or  attitude  that 
enables  one  to  effectively  perform  the  activities  of 
a  given  occupation  or  function  to  the  standards 
expected  in  employment. 
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Development  model:  A  recommended  set  of  activities 
or  tasks  that  defines  a  process  for  successful 
instructional  design. 

Instructional  design:  A  purposeful  activity  that  results 
in  a  combination  of  strategies,  activities,  and 
resources  to  facilitate  learning. 

Instructional  designer:  A  person  with  the  competen¬ 
cies  to  design  instruction. 


INTRODUCTION 

The  field  and  practice  of  instructional  design  has  a 
long  and  proud  heritage.  Through  ongoing  research 
and  development,  theories  and  models  for  best  practice 
in  the  creation  of  instructional  interventions  have  been 
established  (Reigeluth,  1999)  and  supported  by  defi¬ 
nitions  and  specifications  of  the  knowledge,  skills,  and 
attitudes  required  to  be  an  effective  instructional 
designer  (Richey  et  al.,  2001).  In  fact,  Reigeluth  ( 1999, 
p.  5)  stated  that  “an  instructional  design  theory  ... 
offers  explicit  guidance  on  how  to  help  people  learn 
and  develop.”  The  competency  standards  articulated 
by  Richey  et  al.  (2001)  provide  a  foundation  from 
which  individuals  can  effectively  implement  an 
instructional  theory.  It  is  important  to  note,  however, 
that  there  are  many  permutations  of  instructional 
design  and  development  teams.  In  terms  of  the  players 
in  the  instructional  design  field,  Richey  and  colleagues 
(2001,  p.  108)  argued  that  the  field  of  instructional 
design  has  a  number  of  established  or  emerging  spe¬ 
cialist  roles.  Four  roles  — analyst,  evaluator,  e-learning 
specialist,  and  project  manager— have  evolved  and  are 
common  in  many  settings: 

•  The  analyst  specializes  in  performance  anal¬ 
ysis  and  training  needs  assessment. 

•  The  evaluator  specializes  in  various  forms 
of  evaluation  and  assessment,  but  especially 
transfer  and  impact  evaluation. 

•  The  e-learning  specialist  specializes  in 
development  of  multimedia  and  electronic 
learning  products,  particularly  Web-based 
learning. 

•  The  project  manager  specializes  in  manag¬ 
ing  internal  or  external  designers  on  one  or 
several  projects. 

More  recent  writing  has  suggested  that  the  teacher 
and  the  learner  are  considered  integral  to  the  overall 
design  process  (Sims  and  Jones,  2003;  Sims  and  Hed- 
berg,  2006).  As  course  conceptualization  and  imple¬ 
mentation  change,  we  face  a  challenge  to  better  under¬ 
stand  the  roles  and  skills  required  to  establish  effective 


learning  environments  for  new  generations  of  both 
teachers  and  learners.  At  the  same  time,  we  are  con¬ 
sistently  presented  with  alternatives  to  existing  models 
of  instructional  design  which  impact  on  both  the  the¬ 
oretical  perspective  and  methods  of  implementation 
(Fox,  2006;  Sims  and  Jones,  2003).  We  also  face  sig¬ 
nificant  social  and  technological  changes  that,  from 
the  educational  perspective,  have  been  popularized  by 
Prensky’s  (2001,  2006)  notions  of  the  digital  immi¬ 
grant  and  digital  native,  as  well  as  changes  in  learners’ 
behaviors  due  to  access  to  online  gaming  and  social 
networking  technologies. 

In  this  chapter,  we  consider  the  impact  of  these 
changes  on  the  underlying  skill  sets  for  the  instruc¬ 
tional  designer  and  also  on  where  and  when  the 
instructional  design  role  should  be  practiced.  It  is  rec¬ 
ognized  that  the  practice  of  instructional  design  is  not 
static  (Irlbeck  et  al.,  2006)  and  that  it  is  essential  to 
consistently  reflect  on  best  practices  to  meet  contem¬ 
porary  learning  environments  and  expectations. 
Understanding  new  perspectives  of  instructional 
design  also  means  appreciating  that  interactions 
between  elements  in  the  system  can  lead  to  the  emer¬ 
gence  of  highly  complex,  intelligent  behavior  (Kays, 
2003);  consequently,  it  is  not  merely  a  case  of  imple¬ 
menting  an  instructional  design  model  but  rather  one 
of  using  behaviors  and  activities  within  the  broader 
instructional  design  system  as  a  means  to  allow  com¬ 
plex  and  intelligent  behaviors  and  higher  level  learning 
to  occur.  The  application  of  emergence  theory  (Kays, 
2003;  Kays  and  Francis,  2003)  to  the  design  of  online 
distance  education  derives  from  viewing  the  e-learning 
environment  and  the  learning  process  itself  as  a  prob¬ 
lem  in  organized  complexity.  Kays  innovatively  linked 
emergence  theory  and  instructional  design  (Kays, 
2003;  Kays  and  Francis,  2003),  and  as  she  argued  in 
Irlbeck  et  al.  (2006,  pp.  177-178): 

The  elements  in  it — students,  instructor,  resource  materi¬ 
als,  environment — interact  spontaneously,  even  randomly, 
and  are  shaped  by  social  processes  of  a  natural  alignment 
of  the  concepts  for  learning  and  dynamic  group  behaviour. 

. .  .In  the  most  pure  application  of  emergence,  the  teacher 
or  “pacemaker”  would  not  exist  in  the  traditional  sense. 
Faculty  in  the  emergent  role  will  need  a  far  different 
attitude  from  the  conventional  role  and  will  become  part 
of  the  collective  rather  than  the  controlling  agent. 

These  perspectives  on  instructional  design  and  the 
competencies  required  of  the  instructional  designer 
gave  rise  to  a  range  of  challenges  with  regard  to 
ensuring  that  individuals  are  well  equipped  with  the 
necessary  competencies  to  establish  learning  envi¬ 
ronments  to  meet  the  needs  of  today’s  learners.  Con¬ 
sequently,  it  is  imperative  for  current  instructional 
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designers  to  expand  their  skill  set  to  meet  these  needs 
and  challenges. 

Reigeluth  (1999)  added  the  subtitle  A  New  Para¬ 
digm  of  Instructional  Theory  to  his  text  on  instruc¬ 
tional  design,  and  since  then  we  have  witnessed  rad¬ 
ical  shifts  in  the  possibilities  for  educational  delivery, 
predicated  on  the  growth  of  online  technologies. 
When  examining  the  role  and  practice  of  instructional 
design,  a  key  issue  to  emerge  is  the  blurring  between 
the  activities  of  the  designer  and  the  user  (teacher, 
learner,  trainer,  trainee).  Whereas  established  models 
often  assume  that  teaching  strategies,  learning  activ¬ 
ities,  and  resource  materials  are  designed  and  devel¬ 
oped  independently  from  the  delivery  environment, 
current  environments  do  allow  those  activities  to  be 
undertaken  by  all  course  participants  (Sims  and 
Jones,  2003).  It  is  pertinent,  therefore,  to  question 
who  is  the  designer,  what  is  his  or  her  role,  and  what 
competencies  are  essential  in  the  design  and  devel¬ 
opment  context?  It  is  also  important  to  consider  how 
instructional  designers  are  gaining  their  skills  and  the 
extent  to  which  certification  programs  are  consistent 
with  the  current  demands  of  society  and  the  learner. 

A  second  factor  that  confronts  today’s  designers  is 
to  manage  the  balance  between  being  knowledgeable 
(in  creating  environments  and  the  expectations  of  the 
learner)  and  implementation  (instructional  and  learn¬ 
ing  goals,  timelines,  and  effective  performance  of  out¬ 
comes).  Effectively  designed  and  implemented  tech¬ 
nology-enhanced  collaborative  learning  systems  and 
interactive  learning  objects  can  improve  teaching  and 
learning  outcomes;  however,  building  these  environ¬ 
ments  may  require  more  time  than  is  allowed  by  devel¬ 
opment  budgets  or  implementation  timelines. 

Another  factor  relates  to  the  shifts  and  changes  in 
the  learner  population,  popularized  by  Prensky’s 
(2001)  differentiation  of  the  digital  native  and  the  dig¬ 
ital  immigrant.  Commentators  are  now  referring  to  the 
millennial  or  new-age  learner  and  the  dynamic  shift 
toward  mobile  and  broadcast  information  access  and 
the  ability  to  learn  constantly  (Carmean,  C.,  pers.  com- 
mun.).  The  implication  for  instructional  design  is  that 
the  next  generation  of  designers,  instructors,  and  edu¬ 
cators  needs  to  develop  competencies  that  expand  upon 
or  even  replace  those  that  have  been  considered  essen¬ 
tial  (Richey  et  al.,  2001).  Certainly  these  competencies 
should  emphasize  that  technologies  are  tools  to  think 
and  learn  with  rather  than  objects  to  engage  with  and 
possibly  learn  from.  Part  of  this  newer  understanding 
of  learning  with  technology  is  its  very  social  nature  . 
The  complexity  of  the  world  now  requires  learners  to 
better  understand  content  from  multiple  perspectives 
and  to  use  tools  such  as  communication  networks, 
modeling,  simulations,  and  hypertexts  to  access  mul¬ 


tiple  perspectives  of  information  that  can  enrich  the 
instructional  and  learning  experience.  One  question  to 
consider  is  whether  these  networks  will  have  the 
sophistication  to  enable  learning  without  the  interven¬ 
tion  of  the  traditional  instructional  designer  (Kays, 
2003;  Siemens,  2004). 

These  are  the  challenges  of  the  new  instructional 
designer:  to  understand  what  makes  a  powerful  learn¬ 
ing  experience,  what  technologies  can  be  integrated  to 
foster  learning  in  these  environments,  and  how  to  do 
it  effectively.  The  emerging  social  technologies  (e.g., 
blogs)  allow  learners  to  collaborate  and  communicate 
informally,  and  hardware  technologies  are  creating 
portable  devices  that  facilitate  the  anytime,  anywhere 
learning  principle.  So,  the  instructional  designer  is 
challenged  with  new  technologies  and  with  learners 
who  are  working  with  content  and  accessing  content 
in  quite  different  ways.  Additionally,  the  ability  of 
learners  to  source  vast  amounts  of  information  neces¬ 
sitates  a  switch  from  content  specification  being  solely 
the  responsibility  of  a  subject-matter  expert.  If  we 
accept  this  development,  we  need  to  consider  how 
content  is  contextualized  within  the  learning  environ¬ 
ment  and  who  the  owner  and  responsible  provider  of 
that  content  is  (Sims,  2006). 

Finally,  it  is  important  to  better  understand  how 
the  established  models  of  design  and  development  can 
be  effectively  applied  or  modified  to  meet  the  ever- 
changing  socioeconomic  conditions  and  environ¬ 
ments;  for  example,  there  is  renewed  interest  in  the 
use  of  games  for  education  and  training,  and  research¬ 
ers  are  examining  how  podcasts  and  blogs  might 
enhance  the  educational  experience.  More  specific  fac¬ 
tors  such  as  social  capital  and  accessibility  are  having 
an  impact  on  the  instructional  process,  and  designs 
catering  to  multicultural  learners  are  in  demand.  Can 
current  instructional  design  models  meet  these  needs? 
Do  instructional  designers  have  the  competencies  to 
address  these  factors?  Are  existing  competencies  valid, 
or  do  they  require  repurposing? 

To  address  these  questions,  this  chapter  uses  the 
validated  set  of  instructional  designer  competencies 
developed  by  the  International  Board  of  Standards  for 
Training,  Performance,  and  Instruction  ( ibstpi ®;  http:// 
www.ibstpi.org/)  to  both  emphasize  the  existing  key 
competencies  for  the  instructional  designer  as  well  as 
introduce  those  specific  knowledge,  skills,  and  atti¬ 
tudes  we  believe  are  necessary  to  meet  the  demands 
of  21st-century  education  and  training.  In  developing 
this  analysis,  we  use  the  definition  of  competency  pro¬ 
vided  by  Richey  and  colleagues  (2001,  p.  32):  “a 
knowledge,  skill,  or  attitude  that  enables  one  to  effec¬ 
tively  perform  the  activities  of  a  given  occupation  or 
function  to  the  standards  expected  in  employment.” 
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COMPETENCY  STANDARDS 

The  field  of  instructional  design  is  well  established.  As 
such,  a  number  of  organizations  have  developed  com¬ 
petency  standards  for  those  who  wish  to  practice 
instructional  design;  for  example,  the  International 
Society  for  Technology  in  Education  (ISTE)  standards, 
although  not  specifically  directed  at  designers,  provide 
relevant  perspectives  on  technology  competencies. 
Similarly,  a  recent  project  conducted  by  the  United 
Nations  Educational,  Scientific,  and  Cultural  Organi¬ 
zation  (UNESCO)  recommended  a  series  of  competen¬ 
cies  for  training  educators  in  information  and  commu¬ 
nication  technology  (ICT).  In  contrast  to  the  UNESCO 
standards,  which  focus  on  competencies  of  integrating 
educational  technologies  into  teaching,  and  the  ISTE 
competencies,  which  are  educational  technology  stan¬ 
dards  and  performance  indicators  for  teachers,  students, 
and  administrators,  ibstpi ®  provides  a  set  of  interna¬ 
tionally  validated  competencies  and  performance  indi¬ 
cators  for  instructional  designers  (Richey  et  al.,  2001). 

At  the  time  of  this  writing,  however,  few  organi¬ 
zations  provide  professional  certification  of  individu¬ 
als  to  perform  instructional  design  tasks  at  an  agreed- 
upon  level.  Numerous  institutions  offer  instructional 
design  programs,  with  the  assumption  that  graduation 
will  provide  a  form  of  certification.  Even  if  formal 
certification  of  instructional  designers  were  to  become 
more  prevalent,  this  would  not  preclude  the  need  to 
continually  investigate  competency  classifications  to 
ensure  their  currency,  relevance,  and  integrity.  The 
questions  of  whether  or  not  instructional  designers 
should  be  certified  and  what  the  standards  might  be 
remain  unanswered. 

Given  the  rigorous  validation  procedures  of  the 
ibstpi ®  competencies  and  our  association  with  ibstpi ® 
as  directors,  we  have  selected  this  approach  to  rein¬ 
force  established  competencies  for  instructional 
designers  and  to  highlight  those  areas  where  further 
work  is  required  to  address  the  shifting  sands  of  the 
instructional  designer’s  work.  In  terms  of  compe¬ 
tency  validation,  a  number  of  assumptions  have  been 
made,  including:  (1)  instructional  design  is  a  process 
that  is  most  commonly  guided  by  systematic  design 
models  and  principles,  and  (2)  instructional  design  is 
most  commonly  aimed  at  the  transfer  of  training  and 
improved  individual  and  organizational  performance. 
These  assumptions  reinforce  one  perspective  of  the 
instructional  designer— that  of  a  person  (or  team)  that 
operates  separately  in  their  endeavors  from  the  imple¬ 
mentation  and  delivery  environment.  The  extent  to 
which  the  ibstpi ®  competencies  are  relevant  to  the 
new-age  instructional  designer  is  the  focus  of  the 
following  section. 


The  ibstpi®  Standards 

The  ibstpi ®  standards  are  clearly  significant  in  defin¬ 
ing  instructional  designer  competencies;  yet,  the 
opportunity  remains  to  extend  these  to  address  the 
issues  of  blurred  roles,  efficiency,  and  the  currency 
of  design  models.  The  ibstpi ®  competencies  focus  on 
four  domains  of  skill:  (1)  professional  foundations, 
(2)  planning  and  analysis,  (3)  design  and  develop¬ 
ment,  and  (4)  implementation  and  management. 
Within  each  of  these  groups,  specific  competency 
statements  are  defined.  The  following  analysis  iden¬ 
tifies  selected  statements  from  these  standards  (itali¬ 
cized)  and  presents  issues  that  must  be  addressed 
with  respect  to  the  way  in  which  instructional  design¬ 
ers  approach  the  design  of  instructional  materials  and 
environments. 

Within  the  professional  foundations  group,  the 
first  competency  identified  and  considered  essential 
for  the  instructional  designer  is  the  ability  to  commu¬ 
nicate  effectively  in  visual ,  oral,  and  written  form. 
Although  no  one  would  dispute  the  importance  of 
communication,  when  one  interacts  with  design 
stakeholders  who  may  exist  within  different  time 
zones  and  geographic  areas  and  whose  communica¬ 
tion  is  dependent  on  collaborative  network  tools ,  then 
certain  forms  of  communication  are  often  disabled 
(e.g.,  visual  or  oral),  and  the  written  form  is  the 
primary  interactive  element.  What  does  this  mean  for 
the  designer?  The  designer’s  communication  skills 
must  extend  to  combinations  of  asynchronous  and 
synchronous  interactions,  and  their  ability  to  present 
instructional  information  must  integrate  key  factors 
pertinent  to  the  virtual  environment.  Even  more  fre¬ 
quently,  instructional  designers  will  have  to  rely  on 
podcasts,  wikis,  and  mobile  phones  to  receive  and 
respond  to  information;  the  traditional  modes  will  be 
superceded  by  those  underpinned  by  these  emerging 
digital  technologies. 

A  second  competency  states  the  importance  to  the 
instructional  designer  of  being  able  to  update  and 
improve  ...  knowledge ,  skills,  and  attitudes  pertaining 
to  instructional  design  and  related  fields .  One  of  the 
key  issues  emerging  from  current  research  and  theory 
of  instruction  design  is  the  importance  of  a  multidis¬ 
ciplinary  approach;  for  example,  Kays  (2003)  empha¬ 
sized  the  importance  of  emergence  theory  as  a  means 
to  better  understand  the  intricacies  and  dynamics  of 
instructional  design,  while  Irlbeck  and  colleagues 
(2006)  argued  the  value  of  a  multidisciplinary 
approach,  extending  the  original  concepts  of  Sims  and 
Jones  (2003).  Design-based  research  is  also  being  used 
to  inform  the  development  and  revision  cycles  of  tech¬ 
nology-enhanced  learning  environments;  this  research 
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methodology  helps  extend  our  knowledge  of  develop¬ 
ing,  implementing,  and  sustaining  effective  technol¬ 
ogy-enhanced  learning  environments  and  to  create  and 
test  theoretical  constructs  of  such  designs.  The  inten¬ 
tion  of  this  research  is  to  inquire  into  the  nature  of 
technology-enhanced  learning  and  create  frameworks 
for  informing  the  use  of  design  elements  to  enhance 
technology-based  environments  (Cobb,  2001;  Collins, 
1992;  Design-Based  Research  Collective,  2003; 
Koszalka  and  Ganesan,  2004). 

A  third  competency  in  this  domain  is  to  identify 
and  resolve  ethical  and  legal  implications  of  design  in 
the  workplace.  Although  clearly  important  due  to  the 
growing  issues  of  plagiarism,  ownership,  copyright, 
and  intellectual  property  rights,  the  question  remains 
as  to  how  effectively  this  resolution  can  be  achieved 
independently  from  the  delivery  environment.  Not 
only  must  designers  consider  these  issues  when 
instructional  materials  are  devised,  but  they  must  also 
allow  for  resolution  within  the  actual  delivery  cycle. 
This  reinforces  the  urgency  of  the  role  of  the  designer 
to  shift  from  an  external  content  focus  to  an  internal 
dynamic  focus.  It  also  emphasizes  the  need  to  identify 
how  all  stakeholders  might  be  involved  in  the  design 
process  (Sims  and  Hedberg,  2006). 

With  respect  to  the  planning  and  analysis  compe¬ 
tencies,  one  of  the  key  skills  for  the  instructional 
designer  is  to  conduct  a  needs  assessment  to  determine 
the  rationale  and  justification  and  outcomes  for  a  pro¬ 
posed  course.  One  of  the  shifting  areas  of  the  design 
framework  is  the  input  required  from  the  many  stake¬ 
holders  who  are  integral  to  the  success  of  technology- 
mediated  projects  (Sims  and  Hedberg,  2006).  Needs 
assessment  input  is  even  more  important  now  as  com¬ 
plex  technology-enhanced  instructional  environments 
emerge;  determining  what  and  who  to  engage  in  the 
needs  analysis  process  is  a  critical  competency  that 
requires  a  set  of  performance  activities  that  inform  the 
design  of  effective  instruction.  Similarly,  the  compe¬ 
tency  to  design  a  curriculum  or  program  is  not  some¬ 
thing  that  can  be  done  in  isolation  from  other  key 
stakeholders  (see  Sims  et  al.,  2002);  therefore,  the 
instructional  designer  has  to  shift  from  creator  of  a 
curriculum  to  one  who  conceptualizes  environments 
where  curricula  can  be  dynamically  created  and  mod¬ 
ified.  This  extends  to  competencies  such  as  select  and 
use  a  variety  of  techniques  for  determining  instruc¬ 
tional  content,  as  we  must  question  whether  this 
remains  the  role  of  the  instructional  designer  or  has 
content  become  too  fragmented  and  complex  to  be 
predetermined  (Sims  et  al.,  2002). 

Within  the  context  of  online  learning,  which  is  a 
key  element  of  current  educational  technology,  this 
argument  extends  to  competencies  such  as  identify  and 


describe  target  population  characteristics  and  analyze 
the  characteristics  of  the  environment .  Examining  each 
of  these  leads  to  questions  such  as  how  easy  is  it  to 
actually  specify  a  target  population  and  to  what  extent 
can  we  assume  homogeneity  in  training  groups?  Cer¬ 
tainly,  there  are  what  might  be  termed  closed  cohorts 
of  learners  (such  as  a  new  intake  of  Marines),  but  the 
same  cannot  be  said  of  the  classroom,  where  ethnic 
diversity  and  socioeconomic  differences  can  result  in 
vast  differences  in  learner  characteristics.  Similarly, 
the  environments  in  which  people  now  learn  cannot 
easily  be  predicted;  for  example,  are  they  learning  in 
the  workplace,  at  home,  or  on  the  road?  We  argue  that 
this  may  no  longer  be  the  instructional  designer’s  role 
to  define,  but  rather  that  role  must  be  to  enable  the 
individual  participants  to  adapt  the  learning  environ¬ 
ment  to  their  individual  and  contextual  needs. 
Although  this  would  appear  to  be  captured  in  the  com¬ 
petency  to  reflect  upon  the  elements  of  a  situation 
before  finalizing  design  solutions  and  strategies,  we 
question  how  well  an  instructional  designer  can 
achieve  this  if  they  are  separated  from  the  teacher, 
learners,  and  delivery  environment. 

Considering  the  design  and  development  compe¬ 
tencies,  there  is  an  underlying  assumption  that  the 
instructional  designer  is  in  a  position  to  achieve  cer¬ 
tain  outcomes  and  that  instructional  materials  can  be 
developed  independently.  As  an  example,  select  and 
use  a  variety  of  techniques  to  define  and  sequence 
the  instructional  content  and  strategies  has  a  sense 
of  predetermination  of  sequence,  whereas  current 
models  of  online  learning  argue  for  more  dynamic 
and  indeterminate  or  fuzzy  aspects  of  content  that 
may  be  relevant  to  a  course  of  study.  The  ability  of 
learners  to  take  on  this  role,  rather  than  the  designers, 
cannot  be  underestimated  (Sims  et  al.,  2002).  This 
also  relates  to  defined  competencies  such  as  develop 
instructional  materials  and  design  instruction  that 
reflects  an  understanding  of  the  diversity  of  learners 
and  groups  of  learners,  where  the  question  must 
again  be  raised  as  to  how  well  the  designer  can  pre¬ 
dict  the  delivery  environment  and  cater  to  that  diver¬ 
sity  effectively.  Rather  than  predicting  how  learners 
will  or  should  respond,  designers  have  the  responsi¬ 
bility  to  create  environments  where  learners  are 
empowered  with  such  choices. 

Given  the  above  discussion,  the  set  of  competen¬ 
cies  related  to  implementation  and  management  of 
instructional  environments  takes  on  new  meaning  and 
importance;  for  example,  the  competency  to  promote 
collaboration,  partnerships,  and  relationships  among 
the  participants  in  a  design  project  has  been  empha¬ 
sized,  but  it  is  argued  the  participants  have  been  iden¬ 
tified  as  having  priority  roles  in  the  process. 
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ENHANCED  COMPETENCIES 

In  the  current  climate  of  education  and  training,  prac¬ 
titioners  are  faced  with  competing  theories  of  learning 
(Driscoll,  2005;  McCarthy,  2000),  debate  and  variation 
in  instructional  design  theory  (Fox,  2006;  Jonassen, 
2006;  Reigeluth,  1999),  shifting  learning  preferences 
and  lifestyles  of  learners  (Prensky,  2006),  and  contin¬ 
ual  technological  change  and  development.  Within  this 
setting  the  way  in  which  people  engage  in  learning 
activities  and  the  roles  of  those  who  devise  those  activ¬ 
ities  and  associated  teaching  strategies  are  being  chal¬ 
lenged.  The  enhanced  communications  and  connectiv¬ 
ity  afforded  by  digital  technology  enable  a  more 
immediate  response  to  learning  needs  which  in  turn 
challenges  the  existing  roles  and  competencies  of 
instructional  designers.  In  the  following  discussion,  we 
present  a  set  of  enhanced  competencies  that  are  situ¬ 
ated  over  and  above  those  already  deemed  essential 
(Richey  et  al.,  2001)  and  which  demonstrate  the  need 
for  those  in  the  instructional  design  field  to  reflect  on 
expanding  their  current  skill  sets. 

We  have  presented  a  case  that  factors  are  emerging 
that  require  a  reassessment  of  the  competencies  for 
those  involved  in  instructional  design.  In  fact,  if  we 
subscribe  to  the  shift  toward  learner-centered  environ¬ 
ments  as  proposed  by  Reigeluth  (1999)  and  integrate 
this  shift  with  the  critical  factors  impacting  on  learning 
environments,  then  we  can  propose  that  the  overall  role 
of  the  designer,  in  the  context  of  institutional  stake¬ 
holders,  must  be  scrutinized  and  debated.  As  new  mod¬ 
els  are  established  to  support  the  growth  of  collabora¬ 
tive  and  networked  learning  (Crawford,  2004;  Sims 
and  Jones,  2003),  it  becomes  imperative  to  reposition 
the  roles  and  skill  sets  of  instructional  designers  to  the 
extent  that  the  term  instructional  design  might  even 
be  replaced  by  learner/learning  design  (Sims,  2006). 

We  propose  that  instructional  design  competencies 
be  expanded  to  apply  equally  to  learners,  instructors, 
and  technical  support  staff,  rather  than  being  identified 
solely  with  the  instructional  designer.  To  use  a  simple 
analogy,  we  are  suggesting  that  the  role  of  the  instruc¬ 
tional  designer  should  shift  toward  that  of  an  architect 
rather  than  a  builder,  with  the  latter  skills  being  assigned 
to  teachers  and  learners.  Competency  sets,  such  as  those 
defined  by  ibstpi ®,  will  form  the  basis  for  a  new  layer 
of  competencies,  and  one  question  raised  is  whether 
they  should  continue  to  be  referred  to  as  instructional 
design  competencies,  as  the  potential  shifts  in  skill 
requirements  identified  in  this  analysis  could  change 
the  ways  we  perceive  education  and  training. 

This  discussion  also  suggests  that  the  control  the 
instructional  designer  has  on  the  overall  educational 
process  may  be  diminishing.  Certainly,  some  would 


argue  that  social  network  environments  allow  the  par¬ 
ticipants  to  create  and  develop  the  design  (Siemens, 
2004;  Webb  and  Sims,  2006).  Based  on  current  tech¬ 
nologies  and  social  changes,  it  would  appear  that  the 
designers  of  educational  materials  will  no  longer  be 
creating  predefined  and  complete  courses  but  rather 
conceptualizing  shells  in  which  a  multitude  of  activi¬ 
ties  and  participants  can  interact  in  a  dynamic  and 
changing  learning  environment. 

Another  question  is  whether  or  not  the  word 
instruction  best  represents  what  is  occurring  in  educa¬ 
tional  and  workplace  settings.  Would  it  be  preferable 
to  think  about  an  interactive  architect  or  the  learning 
environment  architect  rather  than  an  instructional 
designer?  For  example,  if  massive  multiuser  online 
learning  environments  become  established,  it  is  feasi¬ 
ble  that  the  environment  itself  will  become  the  teacher 
and  that  interactions  between  participants  and  the  envi¬ 
ronment  will  define  the  learning  experience.  The  com¬ 
petencies  to  achieve  this  are  at  a  different  level  than 
those  that  currently  apply  to  the  established  roles  of 
instructional  designers. 

Following  from  this  and  considering  the  work  of 
Kays  (2003)  and  the  relationship  between  emergence 
theory  and  instructional  design,  we  face  the  potential 
of  learning  outcomes  emerging  from  learning  environ¬ 
ments  over  which  the  instructional  designer  has  no 
control— and  the  factors  that  govern  the  emergence  of 
such  occurrences  are  complex  and  uncertain.  Thus, 
when  considering  existing  sets  of  competencies  for  the 
instructional  designer,  we  also  must  be  very  aware  that 
significant  social  and  technological  changes  are 
impacting  the  way  we  teach  and  the  way  we  learn.  As 
a  consequence,  it  is  essential  that  those  who  practice 
instructional  design  build  new  understandings  of  emer¬ 
gent  learning  environments  to  ensure  that  their  practice 
is  current  and  relevant. 


CONCLUDING  REMARKS 

In  this  chapter,  we  have  posed  key  questions  regarding 
who  an  instructional  designer  is  and  the  key  compe¬ 
tencies  required  for  him  or  her  to  perform  their  role 
effectively,  ibstpi ®  (Richey  et  al.,  2001)  captures  the 
key  competencies  of  how  we  have  traditionally  defined 
the  instructional  design  professional,  and  as  technol¬ 
ogy  is  becoming  more  popular  and  easier  to  use  this 
role  is  becoming  less  aligned  to  an  individual  and  more 
distributed  to  those  who  facilitate  technology- 
enhanced  instruction  and  the  learners  who  participate 
in  such  environments.  The  strength  of  the  existing 
competencies  are  that  they  define  how  instruction  must 
be  tied  to  learning,  regardless  of  who  is  designing. 
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developing,  implementing,  or  evaluating.  And, 
although  these  competencies  suggest  a  strong  founda¬ 
tion  for  design  professionals,  questions  are  emerging 
in  this  new  age  around  the  explosive  availability  of 
new  technologies  and  their  integration  into  instruction 
and  learning.  We  need  to  consider  new  aspects  of  learn¬ 
ers,  environments,  technology  features,  and  implemen¬ 
tation  coupled  with  emerging  research.  Although  tra¬ 
ditional  learning  and  instructional  theories  can  inform, 
shifts  in  learning  and  instructional  paradigms  help  us 
to  keep  focused  on  the  purpose  of  designing  instruc¬ 
tion:  purposeful  engagement,  social  interactions,  and 
activities. 

It  is  a  time  of  change,  when  social  and  technolog¬ 
ical  forces  are  redefining  what  it  means  to  learn.  It  is 
a  time  of  change,  when  learners  come  with  a  new  set 
of  skills  that  embrace  mobile,  digital,  wireless  tech¬ 
nologies.  It  is  a  time  of  change,  when  the  complexity 
of  collaborative  learning  and  the  wealth  of  information 
accessible  to  individual  learners  can  make  predetermi¬ 
nation  of  content  almost  impossible.  Consequently,  the 
extent  to  which  the  instructional  designer  can  success¬ 
fully  work  in  and  adapt  to  these  conditions  must  be 
considered.  Although  the  current  sets  of  competencies 
remain  important  and  the  skills  of  practicing  instruc¬ 
tional  designers  remain  critical  to  successful  learning 
outcomes,  we  contend  that  the  very  nature  of  instruc¬ 
tional  design,  as  a  role  that  exists  separately  from  the 
delivery  environment,  is  being  challenged  and  that 
repurposing  and  modifying  the  core  competencies  are 
critical  if  the  instructional  design  role  is  to  be  consid¬ 
ered  relevant  to  new  modes  of  learning  and  teaching. 
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ABSTRACT 

This  chapter  presents  an  overview  of  the  current  state 
of  cognitive  task  analysis  (CTA)  in  research  and  prac¬ 
tice.  CTA  uses  a  variety  of  interview  and  observation 
strategies  to  capture  a  description  of  the  explicit  and 
implicit  knowledge  that  experts  use  to  perform  com¬ 
plex  tasks.  The  captured  knowledge  is  most  often 
transferred  to  training  or  the  development  of  expert 
systems.  The  first  section  presents  descriptions  of  a 
variety  of  CTA  techniques,  their  common  characteris¬ 
tics,  and  the  typical  strategies  used  to  elicit  knowledge 
from  experts  and  other  sources.  The  second  section 
describes  research  on  the  impact  of  CTA  and  synthe¬ 
sizes  a  number  of  studies  and  reviews  pertinent  to 
issues  underlying  knowledge  elicitation.  In  the  third 
section,  we  discuss  the  integration  of  CTA  with  train¬ 
ing  design.  Finally,  in  the  fourth  section,  we  present  a 
number  of  recommendations  for  future  research  and 
conclude  with  general  comments. 

KEYWORDS 

Automated  knowledge:  About  how  to  do  something; 
with  repetition,  it  operates  outside  of  conscious 
awareness  and  executes  much  faster  than  conscious 
processes. 


Cognitive  task  analysis:  Interview  and  observation 
protocols  for  extracting  implicit  and  explicit 
knowledge  from  experts  for  use  in  instruction  and 
expert  systems. 

Complex  tasks:  Tasks  where  performance  requires  the 
integrated  use  of  both  controlled  and  automated 
knowledge  to  perform  tasks  that  often  extend  over 
many  hours  or  days. 

Declarative  knowledge:  Knowledge  about  what  or 
why;  hierarchically  structured;  formatted  as  prop¬ 
ositional,  episodic,  or  visuospatial  information  that 
is  accessible  in  long-term  memory  and  consciously 
observable  in  working  memory. 

Subject-matter  expert  (SME):  A  person  with  extensive 
experience  who  is  able  to  perform  a  class  of  tasks 
rapidly  and  successfully. 


INTRODUCTION 

Cognitive  task  analysis  is  the  extension  of  traditional  task 
analysis  techniques  to  yield  information  about  the  knowl¬ 
edge,  thought  processes  and  goal  structures  that  underlie 
observable  task  performance.  [It  captures  information 
about  both]  . . .  overt  observable  behavior  and  the  covert 
cognitive  functions  behind  it  [to]  form  an  integrated 
whole.  (Chipman  et  al.,  2000,  p.  3) 
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Cognitive  task  analysis  (CTA)  uses  a  variety  of  inter¬ 
view  and  observation  strategies  to  capture  a  description 
of  the  knowledge  that  experts  use  to  perform  complex 
tasks.  Complex  tasks  are  defined  as  those  for  which 
their  performance  requires  the  integrated  use  of  both 
controlled  (conscious,  conceptual)  and  automated 
(unconscious,  procedural,  or  strategic)  knowledge  to 
perform  tasks  that  often  extend  over  many  hours  or 
days  (van  Merrienboer  et  al.,  2002).  CTA  is  often  only 
one  of  the  strategies  used  to  describe  the  knowledge 
required  for  performance.  It  is  a  valuable  approach 
when  advanced  experts  are  available  who  reliably 
achieve  a  desired  performance  standard  on  a  target  task 
and  the  goal  is  to  capture  the  cognitive  knowledge  used 
by  them  (Clark  and  Estes,  1999).  Analysts  use  CTA  to 
capture  accurate  and  complete  descriptions  of  cogni¬ 
tive  processes  and  decisions.  The  outcome  is  most 
often  a  description  of  the  performance  objectives, 
equipment,  conceptual  knowledge,  procedural  knowl¬ 
edge,  and  performance  standards  used  by  experts  as 
they  perform  a  task.  The  descriptions  are  formatted  so 
they  can  be  used  as  records  of  task  performance  and 
to  inform  novices  in  a  way  that  helps  them  achieve  the 
performance  goals  in  any  context.  CTA  is  most  often 
performed  before  (or  as  an  integral  part  of)  the  design 
of  instruction,  work,  job  aids,  or  tests.  The  descriptions 
are  then  used  to  develop  expert  systems,  tests  to  certify 
job  or  task  competence,  and  training  for  acquiring  new 
and  complex  knowledge  for  attainment  of  performance 
goals  (Chipman  et  al.,  2000;  Jonassen  et  al.,  1999). 

TYPES  OF  COGNITIVE  TASK 
ANALYSIS  CURRENTLY  IN  USE 

Researchers  have  identified  over  100  types  of  CTA 
methods  currently  in  use,  which  can  make  it  difficult 
for  the  novice  practitioner  to  choose  the  appropriate 
method  (Cooke,  1994).  The  number  and  variety  of  CTA 
methods  are  due  primarily  to  the  diverse  paths  that  the 
development  of  CTA  has  taken.  It  has  origins  in  behav¬ 
ioral  task  analysis,  early  work  in  specifying  computer 
system  interfaces,  and  in  military  applications— each 
with  its  own  demands,  uses,  and  research  base.  Over 
the  past  20  years,  CTA  has  been  increasingly  informed 
by  advances  in  cognitive  science  and  has  become  an 
important  component  for  the  design  of  systems  and 
training  in  many  domains.  The  growing  body  of  liter¬ 
ature  describing  CTA  methods,  applications,  and 
results  mirrors  the  diverse  application  and  development 
of  CTA  methods;  however,  reviews  and  classifications 
are  available  to  guide  those  interested  in  exploring  and 
applying  CTA,  including  a  comprehensive  review  of 
reviews  provided  by  Schraagen  et  al.  (2000). 


Cognitive  Task  Analysis  Families 

Cooke  (1994)  conducted  one  of  the  more  extensive 
reviews  of  CTA.  She  identified  three  broad  families  of 
techniques:  (1)  observation  and  interviews,  (2)  process 
tracing,  and  (3)  conceptual  techniques.  Observations 
and  interviews  involve  watching  experts  and  talking 
with  them.  Process  tracing  techniques  typically  cap¬ 
ture  an  expert’s  performance  of  a  specific  task  via 
either  a  think-aloud  protocol  or  subsequent  recall.  In 
contrast,  conceptual  techniques  produce  structured, 
interrelated  representations  of  relevant  concepts  within 
a  domain. 

Cooke’s  (1994)  three  families  of  techniques  differ 
in  terms  of  their  specificity  and  formality.  Generally, 
observations  and  interviews  are  informal  and  allow 
knowledge  elicitors  much  flexibility  during  elicitation. 
Process  tracing  methods  have  more  structure  and  spec¬ 
ificity,  although  some  analysis  decisions  are  left  to  the 
elicitor.  Conceptual  techniques  are  well  specified  and 
formal,  with  few  judgments  on  the  part  of  the  elicitor. 
As  a  further  comparison,  more  formal  methods  require 
greater  training  on  the  mechanisms  and  produce  more 
quantitative  data  compared  to  the  informal  methods, 
which  focus  on  interview  skills  and  generate  qualita¬ 
tive  output.  Because  different  techniques  may  result  in 
different  aspects  of  the  domain  knowledge,  Cooke  rec¬ 
ommends  the  use  of  multiple  methods,  a  recommen¬ 
dation  often  echoed  throughout  the  CTA  literature 
(Ericsson  and  Simon,  1993;  Russo  et  al.,  1989;  Vos- 
niadou,  1994). 

Wei  and  Salvendy’s  (2004)  review  of  CTA  meth¬ 
ods  introduced  a  fourth  family— formal  models  — 
which  uses  simulations  to  model  tasks  in  the  cognitive 
domain.  Their  review  further  differs  from  others  in  that 
they  provided  practical  guidelines  on  how  to  use  the 
classifications  of  CTA  methods  to  select  appropriate 
techniques  to  accomplish  various  objectives.  One 
guideline,  for  example,  suggests  that  when  tasks  or 
jobs  do  not  have  a  defined  domain,  observations  and 
interviews  are  especially  useful  in  the  initial  phase  of 
CTA  to  generate  a  more  explicit  context  and  identify 
boundary  conditions. 


VARIETIES  OF  CTA  METHODS 
AND  THEIR  APPLICATIONS 

These  reviews  provide  a  starting  point  to  explore  the 
numerous  varieties  of  CTA  methods  and  their  applica¬ 
tions.  We  examine  the  overall  CTA  process  and  describe 
in  depth  some  methods  that  have  particular  application 
to  instructional  design.  Although  there  are  many  variet¬ 
ies  of  CTA  methods,  most  knowledge  analysts  follow  a 
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five-stage  process  (Chipman  et  al.,  2000;  Clark,  2007; 
Coffey  and  Floffman,  2003;  Cooke,  1994;  Crandall  et 
al.,  2006;  Hoffman  et  al.,  1995;  Jonassen  et  al.,  1999). 
The  five  common  steps  in  most  of  the  dominant  CTA 
methods  are  performed  in  the  following  sequence: 

•  Collect  preliminary  knowledge. 

•  Identify  knowledge  representations. 

•  Apply  focused  knowledge  elicitation  meth¬ 
ods. 

•  Analyze  and  verify  data  acquired. 

•  Format  results  for  the  intended  application. 

The  following  sections  contain  descriptions  of  com¬ 
mon  CTA  methods  and  brief  explanations  of  each  type 
as  it  is  used  during  each  stage  of  the  general  process. 

Collect  Preliminary  Knowledge 

In  this  initial  stage,  the  analyst  identifies  the  sequence 
of  tasks  that  will  become  the  focus  of  the  CTA.  Ana¬ 
lysts  attempt  to  become  generally  familiar  with  the 
knowledge  domain  and  identify  experts  to  participate 
in  the  knowledge  elicitation  process.  Although  knowl¬ 
edge  analysts  and  instructional  developers  do  not  need 
to  become  subject-matter  experts  (SMEs)  themselves, 
they  should  be  generally  familiar  with  the  content, 
system,  or  procedures  being  analyzed.  If  possible,  two 
or  more  subject-matter  experts  should  be  selected  to 
participate  in  the  process  (Chao  and  Salvendy,  1994; 
Fee  and  Reigeluth,  2003).  Although  specific  criteria 
for  identifying  experts  may  change  depending  on  cir¬ 
cumstances,*  all  SMEs  must  have  a  solid  record  of 
successful  performance  at  the  tasks  being  analyzed. 
Experts  are  most  often  interviewed  separately  to  avoid 
premature  consensus  regarding  the  knowledge  and 
skills  necessary  for  effective  performance.  Techniques 
typically  used  during  this  phase  include  document 
analysis,  observation,  and  interviews  (structured  or 
unstructured) .  The  analyst  uses  the  results  of  this  stage 
to  identify  the  knowledge  types  and  structures  involved 
in  performing  the  tasks. 

Document  Analysis 

Analysts  often  begin  their  reviews  by  collecting  any 
available  written  resources  describing  the  tasks  and 
related  subject  matter.  This  can  include  a  wide  variety 
of  documents,  such  as  promotional  literature,  bro- 


*  See  extensive  discussions  of  appropriate  definitions  of  expertise  and 
criteria  for  identifying  experts  in  Cooke  (1992),  Dawes  (1994),  Ericsson 
and  Smith  (1991),  Glaser  and  Chi  (1988),  Mullin  (1989),  and  Sternberg 
and  Horvath  (1998). 


chures,  manuals,  employee  handbooks,  reports,  glossa¬ 
ries,  course  texts,  and  existing  training  materials.  These 
documents  are  analyzed  for  orientation  on  the  tasks, 
preparation  of  the  in-depth  analysis,  and  confirmation 
of  preliminary  ideas  (Jonassen  et  al.,  1999).  This  ori¬ 
entation  prepares  analysts  for  subsequent  task  analysis 
activities;  for  example,  the  information  elicited  during 
structured  interviews  may  be  more  robust  when  ana¬ 
lysts  are  already  familiar  with  experts’  terminology. 
Documentation  analysis  also  allows  comparison  of 
existing  materials  on  a  procedure  with  accounts  of 
expert  practitioners  to  identify  any  immediate  discrep¬ 
ancies  between  doctrine  and  typical  implementation. 

Observations 

Observation  is  one  of  the  most  frequently  used  and 
most  powerful  tools  of  knowledge  elicitation.  It  can 
be  used  to  identify  the  tasks  involved,  possible  limita¬ 
tions  and  constraints  for  subsequent  analysis,  and 
available  information  necessary  to  perform  the  task.  It 
also  allows  analysts  to  compare  an  expert’s  description 
of  the  task  with  actual  events.  In  many  CTA  systems, 
analysts  will  unobtrusively  observe  experts  while  they 
are  performing  the  tasks  under  examination  to  expand 
their  understanding  of  the  domain.  Analysts  observe 
and  record  the  natural  conditions  and  actions  during 
events  that  occur  in  the  setting  (Cooke,  1994). 
Although  definitive  identification  of  an  expert’s  mental 
operations  cannot  be  accomplished  through  observa¬ 
tion,  analysts  may  note  the  occasions  on  which  it 
seems  that  experts  must  make  decisions,  assess  situa¬ 
tions,  or  engage  in  analysis. 

Unstructured  Interviews 

“The  most  direct  way  to  find  out  what  someone  knows 
is  to  ask  them”  (Cooke,  1999,  p.  487).  In  addition  to 
observation,  unstructured  interviews  are  also  common 
early  in  the  CTA  process  to  provide  an  overview  of 
the  domain  and  to  raise  issues  and  questions  for  explo¬ 
ration  in  subsequent  structured  interviews.  In  unstruc¬ 
tured  interviews,  analysts  may  not  dictate  the  content 
or  sequence  of  conversation.  In  other  instances,  how¬ 
ever,  they  may  ask  an  expert  to  focus  on  a  task,  event, 
or  case  with  instructions  to  “tell  me  everything  you 
know  about....” 

Identify  Knowledge  Representations 

Using  the  information  collected  during  the  preliminary 
stage,  analysts  examine  each  task  to  identify  subtasks 
and  types  of  knowledge  required  to  perform  the  task. 
Most  CTA  approaches  are  organized  around  knowledge 
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representations  appropriate  for  the  task,  such  as  con¬ 
cept  maps,  flow  charts,  semantic  nets,  and  so  forth. 
These  representations  provide  direction  and  order  to 
latter  stages  in  the  CTA  process  because  knowledge 
elicitation  methods  map  directly  to  knowledge  types. 
Some  are  best  used  to  elicit  procedural  knowledge, 
while  others  are  more  successful  for  capturing  declar¬ 
ative  knowledge  (Chipman  et  al.,  2000).  A  learning 
hierarchy  is  one  example  of  a  method  to  organize  the 
types  of  knowledge  required  to  perform  a  task. 

Learning  Hierarchy  Analysis 

A  learning  hierarchy  analysis  represents  the  content  of 
skills  ordered  from  more  complex  problem-solving 
skills  at  the  top  to  simpler  forms  of  learning  (Gagne, 
1962,  1968;  Jonassen  et  al„  1999),  so,  for  example, 
problem  solving  is  followed  by  rule  learning,  which  is 
followed  by  concepts.  Thus,  the  basic  idea  is  that  peo¬ 
ple  can  only  learn  rules  if  they  have  already  mastered 
the  prerequisite  concepts  necessary  to  learn  the  rules. 
Analyzing  a  learning  hierarchy  begins  by  identifying 
the  most  complex  (highest)  learning  outcome  and  then 
determining  the  underlying  skills  that  must  be  mas¬ 
tered  to  achieve  the  target  outcome.  A  hierarchy  of 
skills  is  represented  as  a  chart  of  tasks  for  each  intel¬ 
lectual  skill  that  is  acquired  to  progress  to  increasingly 
complex  skills.  The  learning  hierarchy  constructed  at 
this  stage  of  the  CTA  process  provides  the  guide  to 
structure  the  next  stage  of  knowledge  elicitation  by 
identifying  the  information  that  must  be  captured  from 
the  SMEs.  Thus,  it  reflects  the  reiterative  nature  of  the 
CTA  process,  in  which  the  details  of  the  knowledge, 
skills,  and  cognitive  strategies  necessary  for  complex 
learning  are  revealed,  refined,  and  confirmed. 

Apply  Focused  Knowledge  Elicitation  Methods 

During  knowledge  elicitation,  the  analyst  applies  var¬ 
ious  techniques  to  collect  the  knowledge  identified  in 
the  prior  stage.  Past  research  indicates  that  different 
elicitation  methods  yield  different  types  of  knowledge 
and  that  knowledge  is  rarely  articulated  without  being 
the  focus  of  elicitation  (Crandall  et  al.,  2006;  Hoffman 
et  al.,  1998).  Analysts  attempt  to  choose  methods 
appropriate  to  the  targeted  knowledge  type  as  deter¬ 
mined  by  the  knowledge  representations  identified  for 
each  task;  consequently,  most  elicitation  efforts  entail 
multiple  techniques.  Among  the  many  types  of  knowl¬ 
edge  elicitation  methods,  variations  of  structured  and 
semi-structured  interviews  are  most  commonly 
involved  in  CTA  because  they  are  relatively  easy  to 
use  and  require  less  training  than  more  formal  methods 
such  as  protocol  analysis  (Ericsson  and  Simon,  1993) 


or  the  use  of  repertory  grids  (Bradshaw  et  al.,  1993). 
It  is  the  variation  in  these  specific  techniques  that 
defines  the  major  differences  between  specific  CTA 
models.  Although  the  methods  may  differ  in  focus, 
they  share  a  common  purpose  in  capturing  the  condi¬ 
tions  and  cognitive  processes  necessary  for  complex 
problem  solving.  Following  are  descriptions  of  two 
CTA  models  that  have  been  documented  to  effectively 
elicit  experts’  knowledge  in  a  manner  that  is  particu¬ 
larly  effective  for  instruction  (Crandall  and  Gretchell- 
Reiter,  1993;  Velmahos  et  al.,  2004). 

Concepts,  Processes,  and  Principles 

Gathering  concepts,  processes,  and  principles  (CPPs) 
(Clark,  2004,  2007)  involves  a  multi-stage  interview 
technique  that  captures  the  automated  and  unconscious 
knowledge  acquired  by  experts  through  experience  and 
practice.  Multiple  SMEs  describe  the  same  procedure, 
followed  by  cycles  of  expert  self-review  and  peer 
review.  The  initial,  semi- structured  interview  begins 
with  a  description  of  the  CTA  process  by  the  analyst. 
The  SME  is  then  asked  to  list  or  outline  the  perfor¬ 
mance  sequence  of  all  key  subtasks  necessary  to  per¬ 
form  the  larger  task  being  examined.  SMEs  are  also 
asked  to  describe  (or  help  the  interviewer  locate)  at 
least  five  authentic  problems  that  an  expert  should  be 
able  to  solve  if  they  have  mastered  the  task.  Problems 
should  range  from  routine  to  highly  complex  whenever 
possible  .  The  resulting  sequence  of  tasks  becomes  the 
outline  for  the  training  to  be  designed  or  the  job 
description  produced  after  the  CTA  is  completed. 
Starting  with  the  first  subtask  in  the  sequence,  the 
analyst  asks  a  series  of  questions  to  collect: 

•  The  sequence  of  actions  (or  steps)  necessary 
to  complete  the  subtask 

•  The  decisions  that  have  to  be  made  to  com¬ 
plete  the  subtask,  when  each  must  be  made, 
the  alternatives  to  consider,  and  the  criteria 
used  to  decide  among  the  alternatives 

•  All  concepts,  processes,  and  principles  that 
are  the  conceptual  basis  for  the  experts’ 
approach  to  the  subtask 

•  The  conditions  or  initiating  events  that  must 
occur  to  start  the  correct  procedure 

•  The  equipment  and  materials  required 

•  The  sensory  experiences  required  (e.g.,  the 
analyst  asks  if  the  expert  must  smell,  taste, 
or  touch  something  in  addition  to  seeing  or 
hearing  cues  in  order  to  perform  each  sub¬ 
task) 

•  The  performance  standards  required,  such  as 
speed,  accuracy,  or  quality  indicators 
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The  interview  is  repeated  for  each  SME;  each  inter¬ 
view  is  recorded  and  transcribed  verbatim  for  later 
analysis. 

Critical  Decision  Method 

The  critical  decision  method  (CDM)  (Klein  et  al., 
1989)  is  a  semi-structured  interview  method  that  uses 
a  set  of  cognitive  probes  to  determine  the  bases  for 
situation  assessment  and  decision  making  during  crit¬ 
ical  (nonroutine)  incidents  (for  a  full  procedural 
description,  see  Hoffman  et  al.,  1998).  CDM  is  based 
on  the  concept  of  expert  decision  making  as  the  rec¬ 
ognition  of  cue  patterns  in  the  task  environment  with¬ 
out  conscious  evaluation  of  alternatives;  thus,  situa¬ 
tional  awareness  plays  a  dominant  role  in  experts’ 
selection  of  courses  of  action.  The  speed  with  which 
such  decisions  are  made  suggests  that  experts  uncon¬ 
sciously  assess  feasible  goals,  important  cues,  situa¬ 
tional  dynamics,  courses  of  action,  and  expectancies. 
To  elicit  this  knowledge,  CDM  uses  a  retrospective, 
case-based  approach  with  elicitation  occurring  in  mul¬ 
tiple  sweeps  to  gather  information  in  progressively 
deepening  levels  of  detail. 

The  technique  begins  by  selecting  a  critical  inci¬ 
dent  from  the  expert’s  task  experience  that  was  unusual 
in  some  way.  The  experts  involved  provide  unstruc¬ 
tured  accounts  of  the  incident,  from  which  a  timeline 
is  created.  Next,  the  analyst  and  the  experts  identify 
specific  points  in  the  chronology  at  which  decisions 
were  made.  These  decision  points  are  defined  as 
instances  when  other  reasonable  alternative  courses  of 
action  were  possible.  The  decision  points  are  then 
probed  further  using  questions  that  elicit:  (1)  the  per¬ 
ceptual  cues  used  in  making  the  decision,  (2)  prior 
knowledge  that  was  applied,  (3)  the  goals  considered, 
(4)  decision  alternatives,  and  (5)  other  situation  assess¬ 
ment  factors.  The  reports  are  recorded  and  transcribed 
verbatim. 

Analyze  and  Verify  Data  Acquired 

As  noted  above,  CTA  methods  vary  in  structure,  for¬ 
mality,  and  results.  Because  the  knowledge  elicitation 
techniques  described  here  are  less  formal,  they  require 
that  the  analyst  code  and  format  the  results  for  verifi¬ 
cation,  validation,  and  applicability  for  use  in  their 
intended  application.  When  conducting  interviews 
with  experts,  practitioners  recommend  recording  the 
interviews  and  transcribing  them  for  review  at  a  later 
time,  rather  than  trying  to  take  detailed  notes  during 
the  interview,  which  may  distract  from  the  process. 
Transcripts  may  be  coded  to  summarize,  categorize, 
or  synthesize  the  collected  data. 


Following  coding,  the  formatted  output  is  pre¬ 
sented  to  the  participating  SMEs  for  verification, 
refinement,  and  revision  to  ensure  that  the  representa¬ 
tions  of  tasks  and  their  underlying  cognitive  compo¬ 
nents  are  complete  and  accurate  .  Once  the  information 
in  the  formatted  output  has  been  verified  or  revised  by 
the  expert,  the  analyst  should  then  compare  it  with  the 
output  of  other  experts  to  verify  that  the  results  accu¬ 
rately  reflect  the  desired  knowledge  representation. 

The  analysis  stage  in  CPP  (Clark,  2004,  2007) 
begins  with  the  analyst  preparing  a  summary  of  the 
interview  in  a  standard  format  that  includes  the  task, 
a  list  of  subtasks,  and  the  conditions,  standards,  equip¬ 
ment,  and  materials  required.  For  each  subtask,  the 
analyst  then  writes  a  procedure  that  includes  each 
action  step  and  decision  step  required  to  perform  the 
task  and  gives  the  procedure  to  the  SME  to  review.  To 
verify  the  individual  CTAs,  the  analyst  gives  each  sub¬ 
ject-matter  expert’s  product  to  one  of  the  other  SMEs 
and  asks  that  person  to  edit  the  document  for  accuracy 
and  efficiency  (that  is,  to  determine  the  fewest  steps 
necessary  for  a  novice  with  appropriate  prior  knowl¬ 
edge  to  perform  the  task).  In  the  final  stage,  the  analyst 
edits  the  individual  CTAs  into  one  formatted  descrip¬ 
tion  of  how  to  accomplish  all  tasks.  After  final  approval 
by  the  SMEs,  this  final,  formatted  document  provides 
the  information  for  the  instructional  design  process. 
Clark  (2007)  provides  the  format  of  the  protocol. 

The  critical  decision  method  prescribes  no  single 
method  for  coding  the  transcripts  that  are  transcribed 
verbatim  from  the  recorded  interviews,  as  each  specific 
research  question  defines  how  the  transcripts  are  coded 
(Klein  et  al.,  1989).  The  coding  scheme,  however, 
should  be  domain  relevant  and  have  cognitive  func¬ 
tionality;  in  other  words,  it  should  tag  information  that 
represents  perceptual  cues,  decision  points,  and  situa¬ 
tional  assessments.  A  sample  of  a  coded  protocol  can 
be  found  in  Hoffman  and  colleagues  (1998). 

Format  Results  for  the  Intended  Application 

The  results  of  some  highly  structured  CTA  methods 
(e.g.,  cognitive  modeling)  are  readily  applied  to  expert 
systems  or  computer-assisted  tutoring  applications. 
For  less  formal  CTA  methods,  such  as  those  described 
here,  the  results  must  be  translated  into  models  that 
reveal  the  underlying  skills,  mental  models,  and  prob¬ 
lem-solving  strategies  used  by  experts  when  perform¬ 
ing  highly  complex  tasks.  Further,  these  models  inform 
the  instructional  design  of  curriculum,  training,  and 
other  performance  applications.  The  concepts,  pro¬ 
cesses,  and  principles  (Clark,  2004, 2007)  method  gen¬ 
erates  a  description  of  the  conceptual  knowledge,  con¬ 
ditions,  and  a  detailed  list  of  the  actions  and  decisions 
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necessary  to  perform  a  task.  These  products  can  be 
incorporated  into  an  instructional  design  system.  Sim¬ 
ilarly,  products  resulting  from  the  application  of  the 
critical  decision  method  have  been  used  for  a  variety 
of  instructional  applications,  including  building  and 
evaluating  expert  systems  and  identifying  training 
requirements.  CDM  can  provide  case  studies  and 
information  regarding  which  aspects  of  a  task  depend 
on  explicit  knowledge  and  which  depend  on  tacit 
knowledge  (Klein  et  al.,  1989). 

CURRENT  RESEARCH  EVIDENCE 
FOR  THE  IMPACT  OF 
COGNITIVE  TASK  ANALYSIS 

Modem  CTA  evolved  from  a  behavioral  approach  to 
analyzing  performance.  As  the  understanding  of  occu¬ 
pational  demands  evolved  from  a  focus  on  physical 
performance  to  a  focus  on  cognitive  performance,  evi¬ 
dence  suggested  that  key  aspects  of  performance 
entailed  knowledge  that  was  not  directly  observable 
(Ryder  and  Redding,  1993;  Schneider,  1985).  Appli¬ 
cations  of  behavioral  task  analysis  to  training  resulted 
in  incomplete  descriptions  that  led  to  decision  errors 
during  job  performance  (Schraagen  et  al.,  2000).  Early 
versions  of  CTA  were  designed  to  capture  the  deci¬ 
sions  and  analyses  that  could  not  be  directly  observed 
as  well  as  the  deeper  conceptual  knowledge  that  served 
as  the  basis  for  analytical  strategies  and  decisions 
(Clark  and  Estes,  1999).  Thus,  training  shifted  from 
the  reinforcement  of  associations  between  perceptual 
stimuli  and  behaviors  to  the  development  of  declara¬ 
tive  and  procedural  knowledge. 

Research  evidence  indicates  that  the  accurate  iden¬ 
tification  of  experts’  cognitive  processes  can  be 
adapted  into  training  materials  that  are  substantially 
more  effective  than  those  developed  through  other 
means  (Merrill,  2002;  Schaafstal  et  al.,  2000;  Velma- 
hos  et  al.,  2004).  When  content  is  inaccurate  or  incom¬ 
plete,  any  instruction  based  on  that  knowledge  will  be 
flawed  (Clark  and  Estes,  1996;  Jonassen  et  al.,  1999). 
Such  flaws  interfere  with  performance  and  with  the 
efficacy  of  future  instruction  (Lohman,  1986;  Schwartz 
and  Bransford,  1998).  Resulting  misconceptions  resist 
correction,  despite  attempts  at  remediation  (Bargh  and 
Ferguson,  2000;  Chinn  and  Brewer,  1993;  Thorley  and 
Stofflet,  1996). 

Declarative  Knowledge  and  CTA 

Declarative  knowledge  is  hierarchically  structured, 
propositional,  episodic,  visuospatial  information  that 
is  accessible  in  long-term  memory  and  consciously 


observable  in  working  memory  (Anderson,  1983; 
Anderson  and  Lebiere,  1998;  Gagne  et  al.,  1992).  This 
type  of  knowledge  supports  performance  through  the 
conceptual  understanding  of  processes  and  principles 
related  to  a  task  and  the  role  that  the  task  plays  within 
its  broader  context  (Gagne,  1982).  SMEs  possess 
extensive  declarative  knowledge  of  their  domains  in 
the  form  of  principled  frameworks  of  abstract, 
schema-based  representations.  These  frameworks 
allow  experts  to  analyze  complex  problems  efficiently 
(Glaser  and  Chi,  1988;  Zeitz,  1997).  These  elaborate 
schemas  enable  experts  to  retain  and  recall  informa¬ 
tion,  events,  and  problem  states  with  a  high  degree  of 
accuracy  (Cooke  et  al.,  1993;  Dochy  et  al.,  1999; 
Ericsson  and  Kintsch,  1995).  Further,  broad,  princi¬ 
pled  understandings  of  their  domains  facilitate  skill 
transfer  to  solve  related  novel  and  complex  problems 
(Gagne  and  Medsker,  1996;  Hall  et  al.,  1995;  van 
Merrienboer,  1997). 

When  communicated  to  novices,  the  organization 
of  experts’  knowledge  also  impacts  training  outcomes. 
In  an  examination  of  experts’  instructions  to  novices, 
Hinds  et  al.  (2001)  found  that  trainees  who  received 
explanations  from  experts  performed  better  on  transfer 
tasks  than  trainees  who  received  their  explanations 
from  non-experts.  The  experts  provided  explanations 
that  were  significantly  more  abstract  and  theoretically 
oriented  than  those  of  the  non-experts,  so  learners  in 
the  expert-to-novice  instructional  condition  were  able 
to  solve  transfer  problems  more  quickly  and  effectively 
than  their  counterparts  in  the  non-expert-to-novice 
instructional  condition. 

Conceptual  knowledge  alone,  however,  is  insuffi¬ 
cient  for  generating  effective  performance.  The  non¬ 
expert  instructors  in  the  study  provided  more  concrete, 
procedural  explanations,  which  facilitated  higher  per¬ 
formance  by  trainees  when  they  attempted  to  perform 
the  original  target  task.  The  abstractions  provided  by 
the  experts  lacked  key  details  and  process  information 
necessary  for  optimal  performance.  This  finding  is 
consistent  with  many  others  in  the  training  literature, 
suggesting  that  the  most  effective  learning  occurs 
when  all  necessary  information  is  available  to  the 
learner  in  the  form  of  instruction  and/or  prior  knowl¬ 
edge  (for  a  review,  see  Kirschner  et  al.,  2006). 

Findings  from  a  variety  of  studies  indicate  that 
without  CTA  to  facilitate  knowledge  elicitation, 
experts  in  many  fields  unintentionally  misrepresent  the 
conceptual  knowledge  on  which  they  base  their  per¬ 
formance.  In  a  study  by  Cooke  and  Breedin  (1994), 
for  example,  expert  physicists  attempted  to  predict  the 
trajectories  of  various  objects  and  provided  written 
explanations  of  the  methods  by  which  they  reached 
their  conclusions;  however,  when  the  researchers 
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attempted  to  replicate  the  physicists’  predictions  on 
the  basis  of  the  explanations  provided,  they  were 
unable  to  attain  the  same  results.  The  calculated  tra¬ 
jectories  were  significantly  different  from  those  pro¬ 
vided  by  the  experts. 

In  a  similar  study,  expert  neuropsychologists  eval¬ 
uated  hypothetical  patient  profiles  to  determine  their 
theoretical  levels  of  intelligence  (Kareken  and  Will¬ 
iams,  1994).  Participants  first  articulated  the  relation¬ 
ships  between  various  predictor  variables  (e.g.,  educa¬ 
tion,  occupation,  gender,  and  age)  and  intelligence. 
They  then  estimated  IQ  scores  on  the  basis  of  values 
for  the  predictor  variables  they  identified;  however, 
their  estimates  differed  significantly  from  the  correla¬ 
tions  they  provided  in  their  explanations  of  the  rela¬ 
tionships  among  predictor  variables.  Many  were  com¬ 
pletely  uncorrelated.  Clearly,  the  experts’  performance 
relied  on  processes  that  were  very  different  from  their 
declarative  knowledge  of  their  practice. 

Procedural  Knowledge  and  CTA 

Procedural  knowledge  is  required  for  all  skilled  per¬ 
formance.  Skill  acquisition  often  begins  with  learning 
declarative  knowledge  about  discrete  steps  in  a  proce¬ 
dure;  yet,  the  development  of  automaticity  occurs  as 
we  practice  those  procedures.  The  automatization  pro¬ 
cess  involves  learning  to  recognize  important  environ¬ 
mental  cues  that  signal  when  the  skill  is  to  be  applied 
and  the  association  of  the  cues  to  the  discrete  covert 
(cognitive)  and  overt  (action)  steps  required  to  attain 
a  goal  or  subgoal  (Neves  and  Anderson,  1981). 
Through  practice,  these  associations  and  steps  increase 
in  reliability  and  speed  of  performance.  Over  time,  the 
procedures  require  diminishing  levels  of  mental  effort 
or  self-monitoring  to  perform  until  they  utilize  very 
few,  if  any,  cognitive  resources  (Wheatley  and  Wegner, 
2001).  This  consistent,  repeated  mapping  of  condi¬ 
tional  cues  and  steps  manifests  as  an  integrated  if-then 
decision  rule  between  the  cue  (if)  and  the  procedure 
(then)  necessary  to  attain  a  goal  from  a  particular  prob¬ 
lem  state  (Schneider  and  Shiffrin,  1977).  This  repre¬ 
sentation  is  a  production  within  the  ACT-R  cognitive 
model  of  learning  proposed  by  Anderson  (Anderson, 
1995;  Anderson  and  Febiere,  1998). 

During  complex  tasks,  multiple  if-then  produc¬ 
tions  are  strung  together  to  generate  more  sophisticated 
hierarchies  of  performances.  Each  individual  produc¬ 
tion  attains  a  subgoal  that  is  a  component  of  the  overall 
goal.  To  move  from  one  production  to  the  next  in  a 
sequence,  the  new  subgoal  must  be  identified  and  an 
appropriate  production  selected.  For  novices,  the  iden¬ 
tification  and  selection  process  for  nearly  every  sub¬ 
goal  is  a  conscious,  deliberate  decision;  however. 


experts  automate  this  process,  so  they  cannot  con¬ 
sciously  identify  many  of  these  decision  points  (Bless¬ 
ing  and  Anderson,  1996). 

Automaticity  has  two  primary  properties  that  limit 
the  effectiveness  of  unassisted  explanations  by  experts* 
First,  automated  knowledge  operates  outside  of  con¬ 
scious  awareness,  and  executes  much  faster  than  con¬ 
scious  processes  (Wheatley  and  Wegner,  2001).  As  such, 
it  is  not  available  for  introspection  or  accurate  self¬ 
monitoring.  Second,  automated  processes  are  typically 
uninterruptible,  so  they  cannot  be  effectively  changed 
once  they  are  acquired  (Hermans  et  al.,  2000);  conse¬ 
quently,  experts’  unaided  self-reports  of  their  problem¬ 
solving  processes  are  typically  inaccurate  or  incom¬ 
plete**  (Chao  and  Salvendy,  1994;  Feldon,  2004). 

Cues 

Each  element  of  an  if-then  production  has  great  impor¬ 
tance  for  effective  training.  For  learners  to  develop 
effective  procedures,  they  must  attend  to  relevant  cues 
to  determine  correctly  which  subgoals  and  procedures 
are  appropriate.  Thus,  incorporating  experts’  knowl¬ 
edge  of  these  cues  is  important  for  optimal  instruction 
(Fisk  and  Eggemeier,  1988;  Klein  and  Calderwood, 
1991).  Crandall  and  Gretchell-Reiter  (1993) ,  for  exam¬ 
ple,  investigated  the  procedural  knowledge  of  expert 
nurses  specializing  in  neonatal  intensive  care  for  new¬ 
born  or  premature  babies.  The  participants  were  17 
registered  nurses  who  averaged  13  years  of  overall 
experience  and  8.1  years  of  specialization.  Without  a 
formal  knowledge  elicitation  technique,  they 
attempted  to  recall  highly  detailed  accounts  of  critical 
incidents  or  measures  they  had  implemented  that  they 
believed  had  positively  influenced  a  baby’s  medical 
condition.  After  completing  a  free -recall  phase,  the 
researchers  used  CTA  to  identify  additional  relevant 
information  that  the  nurses  did  not  articulate.  Analysis 
of  the  transcripts  revealed  that  the  CTA  probes  elicited 
significantly  more  indicators  of  medical  distress  in  the 
babies  than  were  otherwise  reported.  Before  CTA,  the 
nurses’  explanations  of  the  cues  they  used  were  either 


*  The  literature  on  expertise  has  not  reached  a  consensus  on  the  role 
of  automaticity;  however,  much  empirical  evidence  suggests  that  it  plays 
a  defining  role.  See  Feldon  (in  press)  for  an  extensive  review.  Until  the 
article  reaches  publication,  it  can  be  found  on  the  SpringerLink  website 
under  digital  object  identifier  (DOI)  10. 1007/s  10648-006-9009-0.  A 
prepublication  draft  can  also  be  located  at  http://www.cogtech.usc.edu/ 
recent_publications  .php . 

**  When  experts  attempt  to  solve  novel  problems,  the  elements  of  their 
decision-making  processes  that  are  newly  generated  are  less  likely  to 
be  reported  inaccurately;  however,  preexisting  processes  that  were 
applied  to  those  problems  will  continue  to  be  subject  to  self-report  errors 
(Betsch  et  al.,  1998). 
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omitted  or  articulated  vaguely  as  "highly  generalized 
constellations  of  cues”  (Crandall  and  Gretchell-Reiter, 
1993,  p.  50). 

Comparison  of  the  elicited  cues  to  those  described 
in  the  available  medical  and  nursing  training  literature 
of  the  time  revealed  that  more  than  one  third  of  the 
cues  (25  out  of  70)  used  by  the  expert  nurses  in  the 
study  to  correctly  diagnose  infants  were  absent  from 
that  literature.  These  cues  spanned  seven  previously 
unrecognized  categories  that  were  subsequently 
incorporated  into  standard  training  for  novice  nurses 
entering  neonatal  intensive  care  (Crandall  and  Gam- 
blian,  1991). 

Decision  Points 

In  addition  to  knowing  which  cues  are  important  for 
decision  making,  it  is  also  necessary  to  correctly  iden¬ 
tify  the  points  at  which  those  decisions  must  be  made. 
Much  of  the  research  on  decision  making  suggests  that 
many  decisions  are  made  prior  to  awareness  of  the 
need  to  make  a  decision  (Bargh  et  al.,  2001;  Wegner, 
2002).  Abreu  (1999)  found  that  practicing  psychother¬ 
apists  evaluated  fictitious  case  studies  more  negatively 
when  they  were  primed  with  material  about  African- 
American  stereotypes  than  when  they  rated  the  same 
information  without  priming.  Similarly,  when  Bargh 
et  al.  (2001)  subconsciously  primed  participants  with 
goals  of  either  cooperation  or  high  performance,  the 
actions  of  the  participants  in  a  variety  of  tasks  typically 
conformed  to  the  subliminal  goal  despite  their  being 
completely  unaware  of  either  the  content  of  the  prime 
or  the  fact  that  they  held  the  goal  itself. 

In  professions,  automaticity  presents  significant 
problems  for  training  if  experts  are  relied  upon  to 
explain  the  points  at  which  decisions  must  be  made. 
In  medicine,  for  example,  studies  of  the  reliability  of 
diagnoses  by  expert  physicians  for  identical  symptoms 
presented  at  different  times  only  correlated  between 
.40  and  .50  (Einhom,  1974;  Hoffman  et  al.,  1968). 
Despite  self-reports  suggesting  that  the  participants 
considered  extended  lists  of  symptoms,  analysis  of  the 
symptoms  in  the  cases  presented  indicated  that  only 
one  to  four  symptoms  actually  influenced  diagnosis 
decisions  (Einhorn,  1974). 

Some  experts  freely  acknowledge  that  they  are 
unable  to  accurately  recall  aspects  of  their  problem¬ 
solving  strategies.  Johnson  (1983)  observed  significant 
discrepancies  between  an  expert  physician’s  actual 
diagnostic  technique  and  the  technique  that  he  articu¬ 
lated  to  medical  students.  Later,  he  discussed  with  the 
physician  why  his  practice  and  his  explanation  dif¬ 
fered.  The  physician’s  explanation  for  the  contradic¬ 
tion  was,  “Oh,  I  know  that,  but  you  see,  I  don’t  know 


how  I  do  diagnosis,  and  yet  I  need  things  to  teach 
students.  I  create  what  I  think  of  as  plausible  means 
for  doing  tasks  and  hope  students  will  be  able  to  con¬ 
vert  them  into  effective  ones”  (Johnson,  1983,  p.  81). 

Cognitive  Skills 

Correctly  identifying  and  explaining  the  sequences  of 
cognitive  and  psychomotor  actions  that  are  triggered 
by  cues  at  decision  points  are  likewise  crucial  to  effec¬ 
tive  instruction.  Although  psychomotor  aspects  of  a 
task  are  relatively  simple  for  learners  to  observe,  cog¬ 
nitive  operations  require  articulation  for  a  learner  to 
successfully  replicate  an  expert’s  performance;  how¬ 
ever,  automaticity  often  impairs  this  process.  As  an 
example,  a  team  of  engineers  and  technicians  with 
expertise  in  the  assembly  of  sophisticated  research 
equipment  attempted  unsuccessfully  to  generate  a 
complete  set  of  assembly  instructions,  despite  exten¬ 
sive  and  repeated  efforts  to  include  every  relevant 
fact,  process,  and  heuristic  (Collins  et  al.,  1985). 
When  scientists  who  purchased  the  equipment 
attempted  to  assemble  it  according  to  the  instructions, 
the  equipment  did  not  function.  After  many  discus¬ 
sions  with  the  engineers ,  the  scientists  eventually  dis¬ 
covered  that  the  expert  team  had  accidentally  omitted 
a  necessary  step  from  the  instructions.  The  step 
turned  out  to  be  a  universally  implemented  practice 
among  the  engineers  and  technicians  that  they  had 
failed  to  articulate. 

Chao  and  Salvendy  (1994)  systematically  docu¬ 
mented  the  rates  at  which  experts  omit  cognitive  skills 
from  self-reports.  Six  expert  programmers  were  asked 
to  complete  a  series  of  challenging  troubleshooting 
tasks,  and  all  of  their  actions  were  recorded.  The 
programmers  were  then  asked  to  explain  their  proce¬ 
dures  using  a  variety  of  different  knowledge  elicita¬ 
tion  methods.  No  single  expert  was  able  to  report 
more  than  41%  of  their  diagnostic  actions,  53%  of 
their  debugging  actions,  or  29%  of  their  interpreta¬ 
tions,  regardless  of  the  knowledge  elicitation  method 
used;  however,  when  the  researchers  began  compiling 
the  elicited  explanations  from  different  experts,  they 
found  that  the  percentage  of  actions  explained 
increased.  When  explanations  from  all  six  experts 
were  aggregated,  the  percentages  of  verbalization  for 
each  category  of  actions  increased  to  87%,  88%,  and 
62%,  respectively.  The  improvement  in  information 
elicited  reflects  the  experts’  individual  differences  in 
which  subgoal  productions  had  been  automated  to 
greater  and  lesser  extents.  Thus,  one  promising  prac¬ 
tice  for  instruction  based  on  expert  knowledge  is  to 
employ  CTA  methods  with  multiple  experts  prior  to 
developing  instruction. 
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Instructional  Evidence 

Several  studies  provide  direct  evidence  for  the  efficacy 
of  CTA-based  instruction.  In  a  study  of  medical  school 
surgical  instruction,  an  expert  surgeon  taught  a  procedure 
(central  venous  catheter  placement  and  insertion)  to  first- 
year  medical  interns  in  a  lecture/demonstration/practice 
sequence  (Maupin,  2003;  Velmahos  et  al.,  2004).  The 
treatment  group’s  lecture  was  generated  through  a  CTA 
of  two  experts  in  the  procedure.  The  control  group’s 
lecture  consisted  of  the  expert  instructor’s  explanation 
as  a  free  recall,  which  is  the  traditional  instructional 
practice  in  medical  schools.  Both  conditions  allotted 
equal  time  for  questions,  practice,  and  access  to  equip¬ 
ment.  The  students  in  each  condition  completed  a  writ¬ 
ten  post-test  and  performed  the  procedure  on  multiple 
human  patients  during  their  internships.  Students  in  the 
CTA  condition  showed  significantly  greater  gains  from 
pretest  to  post-test  than  those  in  the  control  condition. 
They  also  outperformed  the  control  group  when  using 
the  procedure  on  patients  in  every  measure  of  perfor¬ 
mance,  including  an  observational  checklist  of  steps  in 
the  procedure,  number  of  needle  insertion  attempts 
required  to  insert  the  catheter  into  patients’  veins,  fre¬ 
quency  of  required  assistance  from  the  attending  phy¬ 
sician,  and  time  to  completion  for  the  procedure. 

Similarly,  Schaafstal  et  al.  (2000)  compared  the 
effectiveness  of  a  preexisting  training  course  in  radar 
system  troubleshooting  with  a  new  version  generated 
from  cognitive  task  analyses.  Participants  in  both  ver¬ 
sions  of  the  course  earned  equivalent  scores  on  knowl¬ 
edge  pretests;  however,  after  instruction,  students  in  the 
CTA-based  course  solved  more  than  twice  as  many  mal¬ 
functions,  in  less  time,  as  those  in  the  traditional  instruc¬ 
tion  group.  In  all  subsequent  implementations  of  the 
CTA-based  training  design,  the  performance  of  every 
student  cohort  replicated  or  exceeded  the  performance 
advantage  over  the  scores  of  the  original  control  group. 

Merrill  (2002)  compared  CTA-based  direct 
instruction  with  a  discovery  learning  (minimal  guid¬ 
ance)  format  and  a  traditional  direct  instruction  format 
in  spreadsheet  use.  The  CTA  condition  provided  direct 
instruction  based  on  strategies  elicited  from  a  spread¬ 
sheet  expert.  The  discovery  learning  format  provided 
authentic  problems  to  be  solved  and  made  an  instructor 
available  to  answer  questions  initiated  by  the  learners. 
The  traditional  direct  instruction  format  provided 
explicit  information  on  skills  and  concepts  and  guided 
demonstrations  taken  from  a  commercially  available 
spreadsheet  training  course.  Scores  on  the  post-test 
problems  favored  the  CTA-based  instruction  group 
(89%  vs.  64%  for  guided  demonstration  vs.  34%  for 
the  discovery  condition).  Further,  the  average  times  to 
completion  also  favored  the  CTA  group.  Participants 


in  the  discovery  condition  required  more  than  the  allot¬ 
ted  60  minutes,  the  guided  demonstration  participants 
completed  the  problems  in  an  average  of  49  minutes, 
and  the  participants  in  the  CTA-based  condition 
required  an  average  of  only  29  minutes. 

Generalizability  of  CTA-Based 
Training  Benefits 

Fee  (2004)  conducted  a  meta-analysis  to  determine 
how  generalizable  CTA  methods  are  for  improving 
training  outcomes  across  a  broad  spectrum  of  disci¬ 
plines.  A  search  of  the  literature  in  ten  major  academic 
databases  (Dissertation  Abstracts  International,  Article 
First,  ERIC,  ED  Index,  APA/Psyclnfo,  Applied  Sci¬ 
ence  Technology,  INSPEC,  CTA  Resource,  IEEE, 
Elsevier/ AP/Science  Direct)  using  keywords  such  as 
“cognitive  task  analysis,”  “knowledge  elicitation,”  and 
“task  analysis”  yielded  318  studies.  Seven  studies 
qualified,  based  on  the  qualifications  of  (1)  training 
based  on  CTA  methods  with  an  analyst,  (2)  conducted 
between  1985  and  2003,  and  (3)  reported  pre-  and 
post-test  measures  of  training  performance.  A  total  of 
39  comparisons  of  mean  effect  size  for  pre-  and  post- 
test  differences  were  computed  from  the  seven  studies. 
Analysis  of  the  studies  found  effect  sizes  between  .9 1 
and  2.45,  which  are  considered  to  be  large  (Cohen, 
1992).  The  mean  effect  size  was  d  =  +1.72,  and  the 
overall  percentage  of  post-training  performance  gain 
was  75.2%.  Results  of  a  chi-square  test  of  indepen¬ 
dence  on  the  outcome  measures  of  the  pre-  and  post¬ 
tests  (x2  -  6.50,  p  <  0.01)  indicated  that  CTA  most 
likely  contributed  to  the  performance  gain. 

Cost-Benefit  Studies  of  CTA 

There  are  few  published  cost-effectiveness  or  cost-ben¬ 
efit  studies  that  compare  CTA  with  other  task  analysis 
approaches.  One  exception,  reported  by  Clark  and  Estes 
(1996),  described  a  field-based  comparison  of  traditional 
task  analysis  and  cognitive  analysis  by  a  large  (10,000+ 
employees)  European  organization  that  redesigned  a 
required  training  course  in  emergency  and  safety  proce¬ 
dures  for  approximately  500  managers.  The  old  and  new 
versions  of  the  course  continued  to  be  offered  after  the 
new  version  of  the  course  was  designed  so  the  relative 
efficacy  of  the  two  approaches  could  be  compared.  All 
objectives  and  test  items  were  similar  in  both  the  old  and 
new  versions.  As  Table  43.1  indicates,  the  use  of  CTA 
required  a  greater  front-end  investment  of  time  (the  orga¬ 
nization  refused  to  release  salary  data)  both  for  the  CTA 
itself  and  the  training  of  instructors  for  the  course  (data 
on  the  time  required  to  train  instructors  for  the  old  course 
were  not  available).  Yet,  even  with  the  approximately 


586 


Cognitive  Task  Analysis 


TABLE  43.1 

Cost  Comparison  of  Behavioral  and  Cognitive  Task  Analysis 


Comparison  Activities 

Behavioral  Task  Analysis 
and  Design  Days* 

Cognitive  Task  Analysis 
and  Design  Days 

Task  analysis  and  design 

7 

38 

Training  of  presenters 

0 

18 

Delivery  by  trainers 

80 

34 

Subtotal 

87 

90 

Total  time  for  500  trainers 

1000 

500 

Total  training  daysb 

1087 

590 

a  Day  =  person  work  day  to  design  and  present  safety  course. 
b  Total  savings  with  CTA:  1087  days — 590  days  =  497  days,  or  2.5  person  years. 

Source:  Clark,  R.E.  and  Estes,  F.,  Int.  J.  Educ.  Res.,  25, 403-417,  1996.  With  permission. 


85%  more  front-end  time  invested  in  design,  develop¬ 
ment,  and  instructor  training,  the  new  course  resulted 
in  2.5  person-years  of  time  savings,  because  it  could 
be  offered  in  1  day  (compared  with  2  days  for  the 
previous  course)  with  equal  or  greater  scores  on  the 
performance  post-test.  Although  these  data  are  only 
suggestive,  the  time  savings  reported  by  Clark  and 
Estes  (1996)  reflect  similar  time  savings  reported 
above  by  Velmahos  et  al.  (2004)  and  Merrill  (2002). 

INTEGRATING  COGNITIVE  TASK 
ANALYSIS  AND  TRAINING  DESIGN 

Optimal  Integration  of  CTA 
and  Training  Design 

For  optimal  application  to  instruction,  CTA  methods 
should  be  fully  integrated  with  a  training  design  model 
to  facilitate  the  alignment  between  learning  objectives, 
knowledge  (declarative  and  procedural)  necessary  for 
attaining  the  objectives,  and  instructional  methods 
appropriate  to  the  required  knowledge.  Currently,  three 
major  systems  take  this  approach:  the  Integrated  Task 
Analysis  Model  (ITAM)  (Redding,  1995;  Ryder  and 
Redding,  1993),  Guided  Experiential  Learning  (GEL) 
(Clark,  2004,  2007),  and  the  four-component  instruc¬ 
tional  design  (4C/ID)  system  (van  Merrienboer,  1997; 
van  Merrienboer  and  Kirschner,  2007;  van  Merrien¬ 
boer  et  al.,  2002).  Of  these,  the  4C/ID  model  is  the 
most  extensively  developed.  It  can  be  distinguished 
from  other  instructional  design  models  in  three  ways. 
First,  the  emphasis  of  the  model  is  on  the  integrated 
and  coordinated  performance  of  task-specific  constit¬ 
uent  skills  rather  than  specific  knowledge  types  or 
sequenced  performance  of  tasks.  Second,  a  distinction 
is  made  between  supportive  information,  which  helps 


learners  perform  the  nonrecurrent  aspects  of  a  complex 
skill,  and  procedural  or  just-in-time  (JIT)  information, 
which  is  presented  to  learners  during  practice  and 
helps  them  to  perform  the  recurrent  aspects  of  a  com¬ 
plex  skill.  Third,  the  4C/ID  model  is  based  on  learners 
performing  increasingly  complex  skills  as  a  whole 
task,  with  part-task  practice  only  of  the  recurrent  skills; 
in  contrast,  traditional  design  methods  emphasize  the 
deconstruction  of  a  complex  task  into  part  tasks, 
which,  once  learned  separately,  are  compiled  as  whole- 
task  practice.  The  assumption  of  the  4C/ID  model  is 
that  environments  supporting  complex  skill  learning 
can  be  described  in  terms  of  four  interrelated  compo¬ 
nents:  learning  tasks,  supportive  information,  just-in- 
time  information,  and  part-task  practice. 

Learning  Tasks 

Learning  tasks  are  concrete,  authentic,  whole-task 
experiences  that  are  organized  sequentially  from  easy 
to  difficult.  Learning  tasks  at  the  same  level  of  diffi¬ 
culty  comprise  a  task  class,  or  group  of  tasks  that  draw 
upon  the  same  body  of  knowledge.  Learning  tasks 
within  a  class  initially  employ  scaffolding  that  fades 
gradually  over  subsequent  tasks  within  the  class. 
Learning  tasks  foster  schema  development  to  support 
nonrecurrent  aspects  of  a  task.  They  also  facilitate  the 
development  of  automaticity  for  schemata  used  during 
recurrent  aspects  of  a  task. 

Supportive  Information 

Supportive  information  assists  the  learner  with  inter¬ 
preting,  reasoning,  and  problem-solving  activities  that 
comprise  the  nonrecurrent  aspects  of  learning  tasks.  It 
includes  mental  models  demonstrated  through  case 
studies,  cognitive  strategies  modeled  in  examples,  and 
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cognitive  feedback.  Through  elaboration,  supportive 
information  helps  learners  to  apply  their  prior  knowl¬ 
edge  when  learning  new  information  they  need  to  per¬ 
form  the  task. 

Just-in-Time  Information 

Just-in-time  information  consists  of  rules,  procedures, 
declarative  knowledge,  and  corrective  feedback 
required  by  learners  to  perform  recurrent  aspects  of 
the  task.  JIT  information  is  presented  in  small  units  as 
"how-to”  instruction,  with  demonstrations  of  proce¬ 
dures  and  definitions  of  concepts  illustrated  with 
examples.  As  learners  perform  the  recurrent  aspects  of 
a  task  and  acquire  automaticity,  the  amount  of  JIT 
information  provided  diminishes. 

Part-Task  Practice 

Part-task  practice  opportunities  are  provided  for  repeti¬ 
tive  performance  of  the  recurrent  aspects  of  a  task  when 
a  high  degree  of  automaticity  is  required.  Part-task  prac¬ 
tice  is  repeated  throughout  instruction  and  mixed  with 
other  types  of  practice.  Part-task  practice  includes  items 
that  vary  from  very  familiar  to  completely  novel. 

COGNITIVE  TASK  ANALYSIS 
AND  4C/ID  MODEL 

The  4C/ID  model  utilizes  CTA  to  accomplish  four 
tasks:  (1)  decomposing  complex  skills  into  skill  hierar¬ 
chies,  (2)  sequencing  the  training  program  within  task 
classes,  (3)  analyzing  nonrecurrent  aspects  of  complex 
skills  to  identify  cognitive  strategies  and  mental  models, 
and  (4)  analyzing  recurrent  aspects  of  the  complex  skill 
to  identify  rules  or  procedures  and  their  prerequisite 
knowledge  that  generate  effective  performance.  In  gen¬ 
eral,  these  activities  occur  within  the  framework  of  the 
five-stage  CTA  process:  however,  because  this  process 
is  highly  integrated  with  the  4C/ID  model,  the  instruc¬ 
tional  design  model  guides  the  CTA  activities.  This 
integration  with  the  instructional  design  process  tends 
to  highlight  the  reiterative  nature  of  the  CTA  process. 

Decomposition  of  the  Complex  Skill 

In  the  first  group  of  task  analysis  activities,  complex 
skills  are  broken  down  into  constituent  skills,  and  their 
interrelationships  are  identified.*  Performance  objec- 


*  The  three  categories  of  interrelationships  are  coordinate  (performed 
in  temporal  order),  simultaneous  (performed  concurrently),  and  trans- 
posable  (performed  in  any  order). 


fives  are  specified**  for  all  constituent  skills,  and  the 
objectives  are  classified  as  recurrent  or  nonrecurrent. 
Objectives  are  classified  as  nonrecurrent  if  the  desired 
behavior  varies  from  problem  to  problem  and  is  guided 
by  the  use  of  cognitive  strategies  or  mental  models. 
Objectives  are  recurrent  if  the  desired  behavior  is 
highly  similar  from  problem  to  problem  and  is  guided 
by  rules  or  procedures.  Sometimes  recurrent  constitu¬ 
ent  skills  require  a  high  degree  of  automaticity;  these 
skills  are  identified  for  additional  part-task  practice. 

Documentation  analysis,  observation,  and  unstruc¬ 
tured  interviews  with  SMEs  provide  the  information 
for  building  a  preliminary  skills  hierarchy  to  guide 
further  knowledge  elicitation  efforts.  Data  collection, 
verification,  and  validation  of  the  skills  hierarchy 
require  multiple  iterations  of  knowledge  elicitation 
using  multiple  SMEs.  The  verified  skills  hierarchy  then 
serves  as  a  guide  for  deeper  CTA  techniques,  such  as 
Clark’s  concepts,  principles,  and  processes  (Clark, 
2004,  2007).  The  CPP  data  identify  constituent  skills 
and  their  interrelationships,  performance  objectives  for 
each  constituent  skill,  and  the  classification  of  the  skill 
as  recurrent  or  nonrecurrent.  The  CPP  method  also 
identifies  problems  ranging  from  easy  to  difficult  to 
assist  in  sequencing  task  classes. 

Sequencing  Task  Classes 

The  second  group  of  task  analysis  activities  involves 
categorizing  learning  tasks  into  task  classes.  The 
skills  hierarchy  and  classified  performance  objectives 
determine  the  sequence  of  training  for  individual  con¬ 
stituent  skills.  The  4C/ID-model  employs  a  whole- 
task  approach,  in  which  trainees  learn  all  constituent 
skills  at  the  same  time.  In  the  first  task  class,  learners 
perform  the  simplest  version  of  the  whole  task.  As 
the  conditions  under  which  the  task  is  performed 
become  increasingly  complex,  the  whole  task  scenar¬ 
ios  become  more  authentic  and  reflective  of  those 
encountered  by  experts  in  the  real  world.  CTA  pro¬ 
cesses  are  used  both  to  verify  the  skills  hierarchy  and 
to  confirm  the  sequencing  of  task  classes  from  simple 
to  complex. 

Analyze  the  Nonrecurrent 
Aspects  of  the  Complex  Skill 

The  third  set  of  analytic  activities  identifies  the  sup¬ 
portive  information  necessary  for  each  task  class  in  the 
form  of  mental  models  (how  is  the  problem  domain 


**  Performance  objectives  reflect  the  performance  as  a  result  of  learning 
and  include  an  action  verb,  a  description  of  tools  used,  conditions,  and 
standards  for  performance. 


588 


Cognitive  Task  Analysis 


organized?)  and  cognitive  strategies  (how  to  approach 
problems  in  the  domain?).  Knowledge  elicitation 
methods  commonly  used  with  SMEs  to  capture  data 
for  nonrecurrent  aspects  of  a  complex  skill  include 
interviews  and  think- aloud  protocols.  The  CTA  meth¬ 
ods  are  repeated  for  both  simple  versions  and  complex 
versions  of  the  task  to  capture  the  knowledge  required 
for  performing  the  nonrecurrent  aspects  of  the  task. 

Analyze  the  Recurrent 
Aspects  of  the  Complex  Skill 

The  final  set  of  task  analysis  activities  in  the  4C/ID 
model  is  an  in-depth  analysis  of  the  recurrent  constit¬ 
uent  skills.  These  are  identified  during  the  skill  decom¬ 
position  process  to  identify  the  JIT  information  required 
for  the  recurrent  aspects  of  the  learning  tasks.  Each 
constituent  skill  that  enables  the  performance  of  another 
constituent  skill  is  identified  in  a  reiterative  process, 
until  the  prerequisite  knowledge  already  mastered  by 
learners  at  the  lowest  level  of  ability  is  identified. 

Analysts  employ  CTA  techniques  to  identify  task 
rules  and  generate  highly  specific ,  algorithmic  descrip¬ 
tions  of  task  performance.  Next,  the  prerequisite 
knowledge  required  to  apply  the  procedure  is  identi¬ 
fied.  The  analysis  of  concepts  occurs  through  the  cre¬ 
ation  of  feature  lists  that  identify  the  characteristics  of 
all  instances  of  a  concept.  At  a  lower  level,  facts  (which 
have  no  prerequisites)  are  identified.  At  a  higher  level, 
processes  and  principles  are  identified.  When  com¬ 
pleted,  the  analyst  incorporates  these  prerequisite 
knowledge  components  into  the  rules  or  procedures 
for  performing  the  task. 

In  sum,  the  results  of  the  four  sets  of  CTA  activities 
in  the  4C/ID  model  provide  detailed  and  in-depth  infor¬ 
mation  about  the  skills,  sequence,  cognitive  strategies, 
mental  models,  rules,  and  prerequisite  knowledge 
required  for  complex  skill  learning  through  the  instruc¬ 
tional  design  of  its  four  interrelated  components:  (1) 
learning  tasks,  (2)  supportive  information,  (3)  JIT 
information,  and  (4)  part-task  practice.  Combined,  they 
form  a  fully  integrated  system  for  problem-based  learn¬ 
ing  in  complex  domains.  A  complete  description  and 
procedure  for  implementing  the  4C/ID  model  can  be 
found  in  van  Merrienboer  and  Kirschner  (2007). 

THE  NEXT  GENERATION 
OF  RESEARCH  ON 
COGNITIVE  TASK  ANALYSIS 

Although  CTA  appears  to  have  significant  potential  to 
improve  various  kinds  of  performance,  it  shares  many 
of  the  challenges  reported  in  studies  of  instructional 


design  theories  and  models  (Glaser,  1976;  Salas  and 
Cannon-Bowers,  2001).  We  need  many  more  well- 
designed  studies  that  systematically  compare  the 
impact  of  different  forms  of  CTA  on  similar  outcome 
goals  and  measures.  We  also  need  to  understand  the 
efficacy  of  different  CTA  methods  when  used  with 
different  training  design  models  and  theories. 

So  many  types  of  CTA  have  been  used  and 
reported,  and  variation  in  the  application  of  methods 
is  so  overwhelming  that  it  is  doubtful  that  any  gener¬ 
alization  about  CTA  will  satisfy  basic  standards  of 
construct  validity.  Researchers  are  cautioned  to  look 
carefully  at  the  description  of  the  methods  used  to 
implement  CTA  to  classify  the  family  origin  of  the 
technique  being  replicated.  We  attempted  to  describe 
five  common  elements  of  most  CTA  methods  in  the 
first  part  of  this  chapter,  but  the  specific  strategies  used 
to  implement  each  of  these  elements  varies  across  stud¬ 
ies.  The  elements  we  described  are  focused  on  the 
common  steps  used  to  implement  CTA.  This  is  a 
sequence  model  and  is  similar  to  the  analysis,  design, 
development,  implementation,  and  evaluation 
(ADDIE)  model  for  instructional  design.  Schraagen  et 
al.  (2000)  and  Cooke  (1994)  have  discussed  this  prob¬ 
lem  in  detail  and  have  attempted  to  organize  the  vari¬ 
ous  methods  into  families  based  on  the  type  of  out¬ 
come  being  pursued  (e.g.,  training,  job  design,  and 
assessment).  Wei  and  Salvendy  (2004)  have  suggested 
1 1  very  useful  guidelines  for  selecting  the  best  CTA 
method  to  achieve  a  goal  (see  Table  43.2). 

First  Principles  of  Cognitive  Task  Analysis 

A  different  and  equally  valuable  strategy  for  tackling 
the  multiplicity  of  CTA  methods  would  be  to  apply 
Merrill’s  (2002)  first  principles  approach  to  a  similar 
problem  with  instructional  design  models.  Merrill 
classified  what  appeared  to  be  the  most  psychologi¬ 
cally  active  instructional  methods  in  a  group  of  popu¬ 
lar,  evidence-based  instructional  design  models.  One 
of  the  principles  he  suggested  is  that  designs  that  help 
learners  connect  with  prior  knowledge  are  more  suc¬ 
cessful.  An  attempt  to  identify  first  principles  of  CTA 
would  be  a  benefit  to  researchers  and  practitioners  by 
identifying  the  active  ingredients  in  key  CTA  methods; 
for  example,  nearly  all  CTA  methods  seem  to  place  a 
heavy  premium  on  the  identification  of  the  environ¬ 
mental  or  contextual  cues  that  indicate  the  need  to 
implement  a  skill.  The  study  of  neonatal  nurses  by 
Crandall  and  Gretchell-Reiter  (1993)  involved  gener¬ 
ating  more  accurate  diagnostic  symptoms  (cues) 
expressed  by  very  sick  babies.  Because  the  recognition 
of  conditional  cues  may  be  automated  and  unconscious 
for  many  SMEs,  the  need  for  accurate  and  exhaustive 
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TABLE  43.2 

Guidelines  for  Selecting  CTA  Methods 


When  to  Use  Various  CTA  Methods 

Tasks  and  domain  are  not  well  defined  in  the  initial  stages. 
Procedures  to  perform  a  task  are  not  well  defined. 

Tasks  are  representative,  and  the  process  is  clear. 

Task  process  and  performance  require  tracking. 

Verbal  data  are  easily  captured  without  compromising  performance. 
Domain  knowledge  and  structures  require  defining. 

Multiple  task  analyzers  are  used,  and  task  requires  less  verbalization. 
Task  requires  quantitative  predication,  and  task  models  change  little 
when  the  scenario  changes. 

Task  performance  is  affected  or  distracted  by  interference. 

Task  analyzers  lack  significant  knowledge  and  techniques. 

Tasks  are: 

Skill-based 

Rule-based 

Knowledge-based 


Families  of  CTA  Methods 

Observations 

and  Interviews 

Process  Tracing 

Conceptual 

Techniques 

Formal  Models 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Source:  Adapted  from  Wei,  J.  and  Salvendy,  G.,  Behav.  Inform.  Technol.,  23(4),  273-299,  2004. 


identification  of  important  cues  may  be  one  of  the  most 
important  principles  of  CTA.  In  the  case  of  the  neo¬ 
natal  nursing  studies,  the  cues  captured  during  CTA 
have  changed  the  textbook  instructions  for  future  neo¬ 
natal  nurses.  Other  principles  may  be  associated  with 
the  identification  of  the  sequence  in  which  productions 
must  be  performed  and  the  decisions  that  must  be  made 
(including  the  alternatives  that  must  be  considered  and 
the  criteria  for  selecting  alternatives).  Principles  may 
also  be  related  to  the  protocols  that  are  used  to  observe 
and  interview  experts  to  capture  the  most  accurate  and 
exhaustive  description  of  their  task-based  knowledge. 
It  is  also  likely  that  a  separate  set  of  principles  would 
be  needed  to  characterize  team  or  organizational  CTAs 
(Schraagen  et  al.,  2000). 

Research  on  Automated,  Unconscious 
Expert  Knowledge 

Concerns  about  experts’  awareness  of  their  own  exper¬ 
tise  and  the  strategies  used  to  capture  unconscious 
knowledge  are  arguably  the  most  important  research 
issues  associated  with  CTA.  The  body  of  research  on 
unconscious,  automated  knowledge  has  yet  to  be  widely 
integrated  into  instructional  design  or  the  practice  of 
educational  psychologists.  Most  of  the  research  in  this 
area  has  been  conducted  by  those  interested  in  psycho¬ 
therapy  and  the  dynamics  of  stereotypes  and  bias  in 
decision  making  (Abreu,  1999;  Bargh  and  Ferguson, 
2000;  Wheatley  and  Wegner,  2001)  and  motivation 


(Clark  et  al.,  2007).  Yet,  we  have  ample  evidence  of  the 
importance  of  this  issue  in  CTA  and  training  from  the 
results  of  current  research  by,  for  example,  Velmahos 
et  al.  (2004)  and  Chao  and  Salvendy  (1994).  We  need 
to  know  much  more  about  how  unconscious  expertise 
influences  the  accuracy  of  task  analysis.  We  also  need 
to  know  much  more  about  how  to  modify  automated, 
unconscious  knowledge  when  people  must  learn  to 
modify  skills.  Clark  and  Elen  (2006)  have  reviewed  past 
research  and  made  suggestions  for  further  study. 

Cost  Effectiveness  and  Cost-Benefit  Research 

Another  promising  area  of  future  research  is  cost- 
effectiveness  and  cost-benefit  analysis  (Fevin  and 
McEwan,  2000).  Existing  studies  have  not  explored 
this  issue  systematically,  but  very  promising  prelimi¬ 
nary  analyses  indicate  significant  learner  time  savings 
and  decreases  in  significant  performance  errors  (Clark 
and  Estes,  1996;  Merrill,  2002;  Schaafstal  et  al.,  2000; 
Velmahos  et  al.,  2004).  These  data  are  important  in 
part  because  many  key  decision  makers  have  the 
impression  that  CTA  is  an  overly  complex  process  that 
requires  a  great  deal  of  time  to  conduct  and  should  be 
avoided  due  to  its  cost  (Cooke,  1994, 1999).  It  is  accu¬ 
rate  to  state  that  CTA  increases  the  time  and  effort 
required  for  front-end  design— particularly  when  a 
number  of  experts  who  share  the  same  skill  must  be 
observed  and  interviewed;  yet,  it  is  also  possible  that 
these  costs  may  be  offset  by  delivery-end  savings  due 
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to  increased  learner  accuracy  and  decreased  learning 
time.  People  in  formal  school  settings  seldom  consider 
decreased  learning  time  as  a  benefit,  but  in  business 
and  government  settings  time  is  a  valuable  commodity. 
The  conditions  under  which  savings  are,  and  are  not, 
available  would  be  a  valuable  adjunct  to  continued 
development  of  CTA.  Many  other  suggestions  are  pos¬ 
sible  but  are  beyond  the  scope  of  this  chapter. 


CONCLUSION 

Cognitive  task  analysis  is  one  of  the  major  contribu¬ 
tions  to  instructional  technology  that  have  resulted 
from  the  cognitive  revolution  in  psychology  and  edu¬ 
cation  beginning  in  the  1970s.  CTA  does  not  seek  to 
replace  behavioral  task  analysis  (or  the  analysis  of  doc¬ 
uments  and  research  to  support  training)  but  instead 
adds  to  existing  methods  that  help  capture  the  covert 
mental  processes  that  experts  use  to  accomplish  com¬ 
plex  skills.  The  importance  of  CTA  is  based  on  com¬ 
pelling  evidence  that  experts  are  not  fully  aware  of 
about  70%  of  their  own  decisions  and  mental  analysis 
of  tasks  (Clark  and  Elen,  2006;  Feldon  and  Clark,  2006) 
and  so  are  unable  to  explain  them  fully  even  when  they 
intend  to  support  the  design  of  training,  assessment, 
job  aids,  or  work.  CTA  methods  attempt  to  overcome 
this  problem  by  specifying  observational  and  interview 
strategies  that  permit  designers  to  capture  more  accu¬ 
rate  and  complete  descriptions  of  how  experts  succeed 
at  complex  tasks.  Research  evidence  described  in  this 
chapter  strongly  suggests  huge  potential  benefits  for 
designers  and  learners  when  CTA-based  performance 
descriptions  are  used  in  training  and  job  aids. 

Many  designers  are  apparently  not  aware  of  or  are 
not  using  CTA.  In  January  2007,  we  searched  Google 
Scholar  for  the  terms  “task  analysis”  or  “task  analysis 
models”  and  then  “cognitive  task  analysis”  or  “cogni¬ 
tive  task  analysis  models.”  The  former  terms  returned 
about  nine  to  ten  times  more  hits  than  the  cognitive 
task  analysis  terms.  We  looked  at  a  number  of  the  texts 
used  to  teach  instructional  design  and  could  not  find 
any  references  to  CTA. 

Cognitive  task  analysis  has  been  the  subject  of 
research  more  often  than  it  has  been  applied  in  prac¬ 
tice,  so  we  suspect  that  few  designers  have  been  trained 
to  conduct  effective  cognitive  task  analyses.  It  is  also 
possible  that  the  assumptions  underlying  CTA  conflict 
with  the  assumptions  that  underlie  some  of  the  cur¬ 
rently  popular  design  theories  such  as  constructivism 
and  problem-based  learning  (Kirschner  et  al.,  2006). 
Educators  who  avoid  direct  instruction  in  favor  of 
expert-supported  group  problem  solving  or  communi¬ 
ties  of  practice  would  not  be  inclined  to  conduct  CTA 


to  support  a  constructivist  context  for  learning  new 
skills  or  to  teach  CTA  in  graduate  programs.  Our 
review  of  the  research  evidence  for  CTA  strongly  indi¬ 
cates  that  if  it  is  adopted  it  might  make  a  huge  contri¬ 
bution  to  learning  and  performance.  It  is  also  clear  that 
many  questions  about  CTA  remain  to  be  answered. 
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ABSTRACT 

This  chapter  describes  approaches  to  the  design  and 
technical  verification  of  assessments  of  performance. 
The  chapter  begins  with  a  brief  definition  of  perfor¬ 
mance  measurement  and  describes  a  set  of  validity 
criteria  that  should  be  in  place  to  guide  the  design  of 
performance  assessments  that  may  serve  multiple  pur¬ 
poses.  Design  criteria  include  descriptions  of  families 


of  cognitive  demands,  approaches  to  ensure  content 
representation,  and  templates  or  objects  to  promote 
economic  design  and  renewal.  We  then  illustrate  with 
multiple  examples  from  K-12  education  and  from  the 
training  community.  The  context  of  most  of  the  chap¬ 
ter  is  the  use  of  technology  tools  for  design,  admin¬ 
istration,  and  automated  scoring.  We  conclude  with 
future  directions  in  performance  measurement  and  its 
implications. 
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KEYWORDS 

Assessment:  The  systematic  observation  of  achieve¬ 
ment  using  any  of  a  number  of  formats,  including 
paper,  technology,  or  live  judgment. 

Performance:  Ability  of  student  or  trainee  to  demon¬ 
strate  accomplishment  in  a  very  specific  domain 
such  as  a  formal  examination,  across  domains  and 
subject  matters,  or  in  realistic  contexts. 

Validation  (validity)'.  The  process  and  outcome  of  col¬ 
lecting  and  interpreting  results  of  assessments  or 
measurement  so  that  inferences  from  findings  are 
warranted  by  evidence. 


INTRODUCTION 

The  design  of  outcome  measures  for  instructional  or 
learning  interventions  has  become  transformed  as  the 
promise  of  computer  technology  has  begun  to  be  real¬ 
ized.  In  this  chapter,  the  characteristics  of  outcome 
measurement  are  described,  validity  and  design  criteria 
are  offered  and  justified  by  research  findings,  and 
examples  of  applications  and  of  the  design  and  valida¬ 
tion  process  are  detailed.  The  chapter  concludes  with 
the  likely  and  more  speculative  future  of  outcome  mea¬ 
surement  and  the  implications  that  this  future  will  have 
for  testing  assessment  practice  and  policy  writ  large. 

Performance  Measurement 

Although  there  are  different  interpretations  by  appli¬ 
cation  context,  such  as  military  training,  K-12  school¬ 
ing,  and  the  business  community,  we  use  performance 
measurement  to  mean  the  systematic  sampling  of 
domains  of  learning,  skills,  propensities,  and  accom¬ 
plishments  in  constrained  or  standardized  form.  We 
use  the  term  performance  to  mean  a  wide  range  of 
stimulus  and  response  modes,  including  selected  and 
constructed  short  answers,  examples  of  multiple 
projects,  and  extended  task  performance  under  simu¬ 
lated  or  actual  conditions.  This  use  of  performance  can 
address  individual  or  team,  unit,  or  organizational  per¬ 
formance.  Its  appropriate  design  requires  the  articula¬ 
tion  of  the  set  of  cognitive  demands,  skills,  and  content 
domains  to  be  assessed  and  the  conditions  and  criteria 
used  to  judge  the  characteristics  of  performance  to 
create  a  score  or  a  category  system.  A  further  require¬ 
ment  is  that  the  criteria  or  other  schemes  used  to  rate 
performance  are  replicable  and,  if  human  raters  are 
used,  the  raters  can  be  trained  to  reliably  evaluate 
performance.  The  purpose  of  performance  measure¬ 
ment  is  to  develop  results  that  can  be  used  to  draw  an 
inference  about  ability  or  competence  presumably 


developed  or  to  be  developed  by  an  intervention.  The 
inferences  about  students  may  involve  ordering 
respondents  by  their  scores,  predicting  future  perfor¬ 
mance  in  other  settings,  or  connecting  outcomes  to 
instructional  attributes.  A  general  use  of  performance 
measurement  is  in  the  evaluation  process  to  determine 
the  degree  to  which  particular  goals  have  been  attained, 
retained,  and  generalized  to  different  situations. 


VALIDITY 

Just  as  outcome  measures  may  be  used  for  different 
purposes  (e.g.,  certifying  a  trainee,  certifying  the 
instruction,  or  comparatively  evaluating  alternatives), 
so  will  the  validity  strategies  be  used  for  different 
purposes  and  situations.  Of  most  importance  is  the 
notion  that  validity  does  not  inhere  in  a  measure  or 
procedure,  once  and  for  all,  but  is  moderated  by  the 
context  and  inferences  of  use.  Rather  than  provide  a 
single  procedure,  such  as  always  correlating  an  exist¬ 
ing  set  of  scores  with  those  of  another  measure,  a 
validity  argument  is  developed  that  marshals  a  wide 
range  of  data  to  make  the  case  (AERA  et  al.,  1999). 
Unfortunately,  when  a  technical  procedure  such  as 
validity  seems  to  move  away  from  a  routinized  or 
recipe  base,  both  its  application  and  its  interpretation 
require  greater  expertise  by  the  assessment  designer 
and  user.  The  former  makes  an  argument  based  on 
theory  and  empirical  results,  and  the  latter  must  use 
his  resources  to  determine  if  the  argument  is  of  suffi¬ 
cient  relevance  and  merit  for  the  purposes  intended  for 
the  measure.  Such  an  approach  mitigates  the  checklist 
approach  to  ensuring  assessment  quality  and  may  deter 
those  charged  with  validity  case  making  and  interpre¬ 
tation.  In  current  practice,  a  crusty  subset  of  practitio¬ 
ners  uses  old  models  of  validity,  such  as  content  (is 
what  is  being  measured  related  to  goals  or  domains?), 
face  validity  (does  the  test  performance  look  like  what 
is  supposed  to  be  measured?),  predictive  (do  people 
with  higher  scores  do  better  on  a  distal  criterion  mea¬ 
sure?),  and  criterion  (does  performance  on  the  new 
measure  relate,  or  correlate,  in  predictable  ways  with 
an  existing  measure  of  known  quality?). 

Although  all  of  these  aspects  may  come  into  play 
in  making  a  validity  argument,  there  is  no  longer  the 
notion  that  one  sorts  validation  procedures  into  a 
four-way  table  and  chooses  one  to  use.  It  is  also  the 
case,  unfortunately,  that  the  validity  of  assessment  is 
often  seen  as  being  separate  from  the  learning  and 
instructional  goals  of  the  measure.  It  may  appear  to 
be  an  add-on  and  as  such  simply  encourages  the 
designer  to  look  for  an  existing,  good-enough  mea¬ 
sure  to  use.  This  approach  has  the  benefit  of  time  and 
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resource  conservation.  It  also  has  the  risk  that  the 
measure  selected  will  not  have  validity  evidence  for 
the  tasks,  learners,  situations,  or  purposes  intended  by 
the  designer.  As  worrisome  is  that  the  measure  may 
not  be  sufficiently  related  to  the  goals  of  the  instruc¬ 
tion-goal  alignment— or  be  sensitive  to  the  instruc¬ 
tional  intervention  as  delivered.  Thus,  the  performance 
or  scores  obtained  on  the  measure  may  be  either  irrel¬ 
evant  to  the  planned  use  of  the  measure  or  insensitive 
to  the  instructional  or  learning  experiences  of  interest. 

To  understand  the  validity  perspective  presented 
below,  it  is  important  for  the  designer  to  adopt  the 
notion  that  the  outcome  measure  and  its  characteristics 
are  an  integral  part  of  the  instructional  or  learning 
vision  that  motivates  the  program  design.  Thus, 
attributes  of  validity  should  map  to  the  range  of  exper¬ 
tise  intended  for  the  learner.  The  list  below  has  been 
adapted  from  other  statements  about  validity  that  are 
intended  to  evaluate  the  effects  of  instruction  (Baker 
et  al.,  1993;  Linn  et  al.,  1991;  Pellegrino  et  al.,  2001). 

Validity  Criteria 

1 .  Cognitive  demands  in  the  measure  represent 
the  range,  complexity,  and  subtasks  required 
for  success. 

Comment.  Specifications  used  for  instruc¬ 
tional  design  should  in  essential  respects 
correspond  to  the  range  of  cognitive 
demands  on  the  performance  measure. 

2.  Content  domains  from  which  examples  of 
situations  and  topics  will  be  drawn  explicate 
and  represent  the  domain  of  interest  rather 
than  any  arbitrary  subset.  A  systematic  pro¬ 
cedure  for  sampling  subject  matter  should 
be  described  and  used. 

Comment.  Various  knowledge  representa¬ 
tions  can  be  used  to  make  explicit  the  domain 
representation  and  its  boundaries,  provide  a 
common  framework  for  task  instantiation, 
and  facilitate  communication  among  devel¬ 
opers  and  subject-matter  experts. 

3.  Evidence  should  be  provided  about  the  sta¬ 
bility  of  scores  (over  time)  and  consistency 
of  scoring  or  marking  to  ensure  dependable 
(reliable)  results. 

Comment.  Score  reliability  depends  on  inter¬ 
actions  among  the  task,  student,  scoring 
scheme,  and  raters,  if  any.  Computer-based 
algorithms  used  for  scoring  student 
responses  should  be  described. 

4.  Evidence  is  used  to  set  criterion  performance 
standards  or  to  create  a  set  of  performance 
categories,  typically  inferred  from  contrasts 


among  performers  with  differing,  estab¬ 
lished  levels  of  expertise  (i.e.,  expert-novice 
comparisons). 

Comment.  Systematic  experiments  with  cri¬ 
terion  groups  present  one  approach  to  this 
criterion.  An  alternative  is  one  that  docu¬ 
ments  the  dependency  relationships  (both 
logically  and  empirically)  of  the  tasks  in  dif¬ 
ferent  categories.  A  third  method  involves 
monitoring  subsequent  criterion  perfor¬ 
mance  and  adjusting  classifications  to  con¬ 
form  to  empirical  findings. 

5 .  Evidence  of  the  performer’s  ability  to  trans¬ 
fer  and  generalize  to  likely  but  directly  unin¬ 
structed  situations  should  be  provided. 
Comment.  Most  instruction  is  directed  to 
objectives,  goals,  or  standards  rather  than  to 
a  particular  subset  of  test  items  or  tasks. 
Even  in  cases  where  procedures  are  the  focus 
of  instruction  (e.g.,  troubleshooting  or  a  par¬ 
ticular  surgical  procedure),  there  are  still 
numerous  ways  in  which  situations,  time,  or 
constraints  can  vary.  Assessments  should 
address  both  the  class  of  illustrations  given 
in  instruction  as  well  as  some  probable 
requirements  to  apply  the  knowledge  or  skill 
to  a  different  setting  or  task. 

6.  It  is  essential  to  obtain  routine  evidence  that 
the  practice  results  in  fair  results— that  is, 
outcomes  that  are  indifferent  to  factors  other 
than  those  intended  to  be  measured. 

Comment.  Such  factors  include  background 
characteristics  of  learner’s  gender,  language 
(for  nonverbal  tasks) ,  the  manner  of  commu¬ 
nication  (syntax,  speed,  usage,  vocabulary) 
of  the  directions  and  the  assessment,  behav¬ 
ior  by  the  test  administrator,  and  interactions 
by  the  respondents  if  not  in  an  individual 
setting.  In  simulation  and  other  technology 
settings,  familiarity  with  equipment,  tasks, 
and  contexts  should  not  interact  with  group 
differences. 

These  criteria  represent  a  minimum  subset  of  validity 
concerns.  Each  requires  analysis  and  may  be 
strengthened  by  more  than  one  empirical  trial.  In  the 
process  of  system  development  of  the  task,  it  is  desir¬ 
able  for  these  validity  components  to  be  assessed, 
even  if  only  partially.  The  balance  between  the  exi¬ 
gencies  of  time  and  schedule,  on  the  one  hand,  and 
quality  of  the  inferences  presents  a  challenge  that  too 
often  ends  up  with  the  quality  of  the  measures  lacking 
the  basic  warrants  needed.  Given  that  outcome  mea¬ 
sures  are  used  for  purposes  of  certifying  competence. 
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accountability,  predicting  readiness,  evaluating  train¬ 
ing,  and  driving  the  revision  of  systems  of  learning 
and  practice,  the  quality  of  the  performance  measures 
ought  to  be  the  last  element  compromised. 

DESIGN  OF  TECHNOLOGY-BASED 
PERFORMANCE  ASSESSMENTS 

In  this  section,  we  describe  the  design  characteristics 
that  flow  from  the  validity  criteria.  Particularly 
important  is  the  specification  of  the  knowledge  and 
cognitive  skills  required  of  a  task— its  cognitive 
demands  — and  how  these  demands  influence  the 
specification  of  the  domain  of  interest,  the  task  rep¬ 
resentation,  and  scoring. 

Cognitive  Demands 

The  cognitive  demand  of  a  task  refers  to  the  domain- 
independent  and  domain-dependent  set  of  knowl¬ 
edge  and  skills  that  an  assessment  should  target 
(AERA  et  al.,  1999;  Baker,  1997;  Baker  and  Mayer, 
1999;  Mislevy  and  Riconscente,  2006).  When 
designing  assessments  of  performance,  it  is  impor¬ 
tant  to  be  clear  and  explicit  early  in  the  design  pro¬ 
cess  about  what  the  assessment  is  intended  to  mea¬ 
sure.  Baker’s  (1997)  Model-Based  Assessment 
(MBA)  explicitly  emphasizes  this  idea.  MBA  posits 
five  areas  that  broadly  capture  the  major  learning 
outcomes  of  education  and  training:  content  under¬ 
standing,  problem  solving,  teamwork/collaboration, 
communication,  and  self-regulation.  MBA  begins 
with  the  specification  of  the  type  of  learning  out¬ 
comes  to  be  measured,  followed  by  attendant  spec¬ 
ifications  of  the  content,  task  and  materials,  response 
modes,  and  scoring  approaches.  Only  after  designers 
have  addressed  the  learning  outcome  their  assess¬ 
ment  is  intended  to  measure  should  they  make  deci¬ 
sions  about,  for  example,  format,  number  of  items, 
and  the  amount  of  testing  time.  The  advantage  of  the 
MBA  approach  is  that  designers  must  be  precise 
about  the  specific  kind  of  cognitive  outcome  they 
are  interested  in  (e.g.,  facts,  declarative  or  procedural 
knowledge,  conceptual  knowledge,  problem  solving) 
and  how  other  aspects  of  the  assessment  design  can 
support  the  measurement  of  the  cognitive  outcome. 
For  example,  if  the  decision  is  made  to  measure 
factual  knowledge,  then  a  test  format  that  is  often 
suited  for  this  purpose  is  multiple  choice.  Similarly, 
if  the  decision  is  made  to  measure  students’  concep¬ 
tual  understanding,  then  an  essay  or  knowledge  map 
may  be  useful  for  this  purpose  (Baker  et  al.,  1991; 
Herl  et  al.,  1996). 


Domain  Representation 

A  domain  representation  is  an  explicit  description  of 
the  content,  knowledge,  skills,  abilities,  interests,  atti¬ 
tudes,  and  other  properties  of  the  construct  that  is 
intended  to  be  assessed  (AERA  et  al.,  1999;  Baker, 
1997;  Baker  and  Mayer,  1999;  Baker  and  O’Neil, 
1987).  A  domain  representation  is  the  basis  for  sam¬ 
pling  test  items,  the  referent  against  which  to  evaluate 
the  relevance  and  representativeness  of  the  test  items, 
and  reflects  the  universe  to  which  performance  on  the 
test  is  being  generalized. 

An  important  function  of  a  domain  representation 
is  to  help  assessment  developers  sample  tasks  over  a 
domain  so  the  tasks  represent  the  important  skills  and 
knowledge  of  the  domain.  The  tasks  also  should  result 
in  performance  that  differentiates  between  novices  and 
experts.  The  general  idea  is  that  students’  performance 
can  be  compared  against  a  criterion  performance, 
where  the  criterion  performance  represents  a  point  on 
a  continuum  ranging  from  novice  to  expert  perfor¬ 
mance  (Baker,  1974;  Glaser,  1963;  Hively,  1974; 
Popham  and  Husek,  1969).  A  secondary  function  of 
these  absolute  (vs.  normative)  comparisons  is  to  pro¬ 
vide  diagnostic  information  on  what  individuals  can 
and  cannot  do;  for  example,  if  a  domain  is  composed 
of  different  subdomains  (e.g.,  addition  and  subtraction 
skills  belonging  to  the  larger  domain  of  arithmetic), 
then  items  can  be  sampled  from  each  subdomain  and 
scales  formed  in  a  principled  manner.  Student  perfor¬ 
mance  on  the  scales  can  be  used  to  infer  the  degree  to 
which  they  have  mastered  those  skills.  The  signifi¬ 
cance  of  such  domain-referenced  measurement  is  that 
performance  is  indexed  to  an  explicit  and  precisely 
described  definition  of  knowledge  and  skills.  Because 
the  items  are  conformant  to  the  domain  description, 
they  are  by  definition  relevant. 

Several  approaches  have  been  used  to  develop 
domain  representations.  Hively  et  al.  (1968),  for  exam¬ 
ple,  described  a  technique  in  the  area  of  arithmetic  that 
precisely  specified  the  math  operations  that  were  to  be 
demonstrated  by  students.  Canonical  forms  of  prob¬ 
lems  were  used  to  generate  instances  of  math  prob¬ 
lems,  thereby  allowing  a  large  number  of  items  to  be 
generated  (or  sampled)  traceable  to  the  domain  defi¬ 
nition.  This  general  technique  has  been  criticized  as 
being  applicable  only  to  well-specified  and  constrained 
domains  such  as  mathematics  (for  an  in-depth  discus¬ 
sion  of  domain-referenced  testing  issues,  see  Baker 
and  O’Neil,  1987). 

Variations  of  this  basic  approach  have  been  devel¬ 
oped  subsequently.  Tatsuoka’s  rule-space  methodol¬ 
ogy  (Birenbaum  et  al.,  1993;  Katz  et  al.,  1998;  Tat- 
suoka  and  Tatsuoka,  1983,  1997),  like  the  method  of 
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Hively  et  al.  (1968),  mapped  items  to  well-defined 
skill  attributes.  Tatsuoka  extended  the  work  of  Hively 
et  al.  by  using  a  matrix  to  represent  the  item  x  skill 
mappings ,  from  which  a  hierarchy  of  skills  could  be 
derived  a  priori.  A  third  method  to  describing  a 
domain  is  with  the  use  of  (external)  knowledge  rep¬ 
resentations.  Bayesian  networks  can  be  used  to  model 
the  dependencies  among  student  skills,  knowledge 
states,  and  other  variables  of  interest  in  relation  to 
observed  student  performance.  These  techniques 
underscore  the  idea  that  a  domain  representation 
should  be  explicit,  precise,  and  externalized  and 
should  capture  the  essential  elements  of  what  is  to 
be  tested  with  respect  to  the  target  environment.  Just 
as  important,  a  good  domain  representation  can  sup¬ 
port  identification  of  student  deficiencies,  which 
could  be  used  to  guide  remediation  efforts. 

Interestingly,  artificial-intelligence-based  knowl¬ 
edge  representation  techniques  are  gaining  increased 
interest  as  a  means  for  domain  representation.  Ironi¬ 
cally,  the  strong  representation  techniques  demon¬ 
strated  by  Hively  et  al.  (1968),  once  criticized  as  being 
overly  reductionist,  are  being  revisited.  An  important 
difference  today  is  that  there  are  readily  available  tools 
that  make  practical  the  development  of  domain  ontol¬ 
ogies,  which  can  be  far  more  expressive  than  previous 
representations  and  include  class  hierarchies  and  non¬ 
linear  network  associations  (Gruber,  1995). 

Examples  of  how  knowledge  representations  are 
being  used  for  assessment  purposes  can  be  seen  in  the 
use  of  ontologies  for  domain  representation  (Chung  et 
al.,  2003a, b)  and  constraint  networks  for  assessment 
authoring  support  (Chung  et  al.,  in  press;  Niemi  et  al., 
in  press;  van  der  Linden,  2005).  As  these  technologies, 
methods,  and  tools  mature,  rich  multivariate  descrip¬ 
tions  of  the  domain  become  possible  .  As  the  domain 
representations  increase  in  fidelity  with  respect  to  the 
target  environment,  students’  performance  on  the 
assessment  should  be  more  representative  of  their 
future  performance  in  the  target  environment. 

Task  Representation 

A  task  representation  comprises  the  critical  compo¬ 
nents  of  the  assessment  activity  learners  are  expected 
to  engage  in,  and  it  defines  the  testbed  within  which 
to  observe  and  gather  evidence  about  their  perfor¬ 
mance,  which  is  the  basis  for  drawing  inferences  about 
their  competence  (AERA  et  al.,  1999;  Baker,  2002; 
Baker  and  Herman,  1983;  Baker  et  al.,  1991,  2005; 
Messick,  1995).  The  task  representation  includes  the 
domain  of  interest  (e.g.,  the  set  of  knowledge,  skills, 
behavior,  attitudes,  and  other  properties)  that  perfor¬ 
mance  is  being  generalized  to,  the  stimulus  materials 


(e.g.,  the  prompt  and  materials  learners  are  expected 
to  interact  with),  the  format  of  administration  (e.g., 
paper-  or  computer-based),  the  set  of  measures  and  the 
format  of  students’  responses  (e.g.,  selected  response, 
constructed  response,  clickstream),  the  administration 
details  (e.g.,  setting,  time  limit),  and  the  scoring 
method  associated  with  the  measures.  Because  the  task 
representation  defines  the  conditions  under  which  stu¬ 
dent  performance  is  interpreted,  the  higher  the  fidelity 
of  the  task  to  the  target  environment,  the  more  likely 
the  performance  on  the  assessment  task  will  generalize 
to  the  target  environment  (Kane  et  al.,  1999;  Messick, 
1995).  As  alluded  to  in  the  discussion  on  domain  rep¬ 
resentation,  the  substrate  underlying  task  design  is  the 
domain  representation.  If  the  tasks  are  misaligned  with 
the  domain  representation  or  the  domain  representa¬ 
tion  is  not  representative  of  the  target  environment, 
then  inferences  based  on  students’  performance  on 
these  tasks  will  be  suspect.  When  students  are  tested 
on  content  they  have  not  been  exposed  to  or  are  not 
tested  on  content  they  have  been  exposed  to,  the  assess¬ 
ment  results  will  not  accurately  reflect  students’ 
achievement  (Baker  et  al.,  1991). 

Development  in  computer-based  assessments  has 
focused  on  creating  tasks  that  allow  observation  of 
features  of  student  behavior  that  could  not  be  done 
feasibly  in  other  ways  (Baker  and  Mayer,  1999;  Ben¬ 
nett,  1999;  Chung  and  Baker,  2003a;  Clauser,  2000; 
Huff  and  Sireci,  2001;  Pellegrino  et  al.,  2001;  Scalise 
and  Gifford,  2006;  Yang  et  al.,  2002).  In  addition  to 
the  examples  described  in  this  chapter,  computer-based 
assessments  have  been  developed  for  Web-searching 
skills  (Schacter  et  al.,  1999),  math  reasoning  (Bennett 
et  al.,  2000),  teamwork  skills  (Chung  et  al.,  1999), 
design  skills  (Chung  and  Baker,  2003a;  Katz  and 
James,  1998),  architecture  design  skills  (Bejar,  1991), 
and  writing  skills  (Attali  and  Burstein,  2006;  Burstein, 
2003;  Landauer  et  al.,  2003;  Page  and  Petersen,  1995). 

A  common  objective  of  computer-based  assess¬ 
ments  of  performance  is  that  they  deliver  tasks  that 
require  students  to  demonstrate  the  use  of  domain 
knowledge  and  problem  solving  strategies.  A  key  issue 
underlying  computer-based  techniques  is  operational¬ 
izing  how  an  activity  would  elicit  the  desired  behaviors 
and  cognitive  demands  from  students  and,  assuming 
that  is  achieved,  how  to  synthesize  and  interpret  the 
observations.  We  discuss  scoring  in  the  Computer- 
Based  Scoring  Approaches  section. 

Design  Templates  and  Objects 

The  search  for  efficient  and  cost-effective  means  to 
simultaneously  increase  the  throughput  and  quality 
of  assessment  development  has  been  the  driver 
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behind  efforts  to  create  automated  supports  for  task 
design  (Baker,  2002;  Chung  et  al.,  2002,  2004,  in 
press;  Katz,  1998).  The  Assessment  Design  and 
Delivery  System  (ADDS)  has  been  developed  to  pro¬ 
vide  teachers  tools  with  which  to  create  assessments. 
With  ADDS,  teachers  have  the  capability  to  create 
assessment  using  assessment  objects  (e.g.,  new  or 
preexisting  prompts,  information  sources).  Com¬ 
pared  to  non- ADDS  teachers ,  ADDS  users  have  been 
found  to  focus  more  on  measuring  conceptual  knowl¬ 
edge  and  to  create  more  appropriate  rubrics  and 
coherent  prompts  that  address  critical  ideas  (Niemi 
et  al.,  in  press;  Vendlinski  et  al.,  2005).  A  similar 
effort  is  attempting  to  operationalize  Mislevy’s  evi¬ 
dence-centered  design  (ECD)  approach  in  the  Prin¬ 
cipled  Assessment  Designs  for  Inquiry  (PADI)  sys¬ 
tem  (Mislevy  and  Riconscente,  2005;  Mislevy  et  al., 
2003;  Riconscente  et  al.,  2005).  Although  the  PADI 
system  is  still  under  development,  existing  tasks  of 
known  structure  and  quality  have  been  successfully 
reengineered  using  the  PADI  framework.  This  suc¬ 
cess  suggests  that  a  mature  system  will  be  able  to 
impart  the  ECD  approach  into  the  design  of  assess¬ 
ments  from  scratch. 


COMPUTER-BASED 
SCORING  APPROACHES 

A  scoring  model  refers  to  the  method  of  translating 
observations  of  student  performance  into  scores  that 
represent  a  meaningful  quantity  about  their  perfor¬ 
mance.  The  scoring  model  includes  an  information 
measurement  scale,  scoring  criteria,  performance 
descriptions  of  each  criteria  at  each  point  on  the 
scale,  and  sample  responses  that  illustrate  the  various 
levels  of  performance  (AERA  et  al.,  1999;  Baker  et 
al.,  2005).  Scoring  issues  in  computer-based  perfor¬ 
mance  assessments  are  complex,  as  they  offer  the 
potential  to  yield  a  rich  set  of  observations,  particu¬ 
larly  when  the  observations  are  fine  grained,  interre¬ 
lated,  and  process  oriented  (Baker  and  Mayer,  1999; 
Bennett,  1999;  Chung  and  Baker,  2003b;  Clauser, 
2000;  Huff  and  Sireci,  2001;  Pellegrino  et  al.,  2001; 
Scalise  and  Gifford,  2006;  Yang  et  al.,  2002).  In  this 
case,  it  is  important  to  define  how  the  observations 
are  combined  and  how  they  are  scored  and  scaled. 
Evidence  should  be  collected  on  how  the  measures 
relate  to  other  measures  of  the  construct  and  how  the 
measures  discriminate  between  high  and  low  per¬ 
formers.  Several  classes  of  scoring  techniques  have 
been  used  for  scoring:  (1)  expert-based  scoring,  (2) 
data-driven  methods,  and  (3)  domain-modeling 
methods . 


Expert  Performance  as  the  Criterion 

The  first  approach  is  consistent  with  the  idea  that  cri¬ 
terion  performance  is  a  point  along  a  performance 
continuum.  Expert  performance  is  considered  the  ref¬ 
erent  or  gold  standard  against  which  to  compare  stu¬ 
dent  performance  (Baker,  1997;  Baker  et  al.,  1991;Chi 
et  al.,  1988),  not  what  experts  say  should  be  competent 
performance  or  experts’  rating  of  student  performance. 
This  scoring  approach  has  been  used  successfully  to 
develop  tasks  for  content  understanding— for  example, 
essays  (Baker  et  al.,  1991 , 2005)  and  knowledge  maps 
(Herl  et  al.,  1996,  1999). 

One  example  of  how  an  expert  referent  has  been 
used  is  in  the  area  of  scorable  concept  or  knowledge 
maps  (Herl  et  al.,  1996,  1999).  A  knowledge  map  is  a 
graphical  way  to  show  how  concepts  relate.  Nodes 
represent  concepts,  and  labeled  links  represent  rela¬ 
tions  among  concepts  (Lambiotte  et  al.,  1989;  Novak 
and  Gowin,  1984).  To  support  automated  scoring, 
domain  experts  define  the  set  of  terms  and  links  for 
the  mapping  task.  The  task  is  then  administered  to  one 
or  more  experts,  and  the  set  of  expert  maps  is  used  as 
the  scoring  criteria  for  student  maps.  In  numerous  stud¬ 
ies  across  age,  content,  and  setting,  scoring  student 
knowledge  maps  using  expert-based  referents  has  been 
found  to  be  sensitive  to  instruction,  to  discriminate 
between  experts  and  novices,  to  discriminate  between 
different  levels  of  student  performance,  to  relate  mod¬ 
erately  to  external  measures  of  the  same  construct,  and 
to  be  sensitive  to  language  proficiency,  as  well  as 
showing  reasonable  technical  properties  (Chung  et  al., 
2001 , 2003a;  Herl  et  al.,  1996,  1999;  Kim  et  al.,  2004; 
Klein  et  al.,  2002;  Lee,  2000;  Osmundson  et  al.,  1999; 
Ruiz-Primo  et  al.,  2001;  Schacter  et  al.,  1999;  Yin  and 
Shavelson,  2004). 

Modeling  Expert  Judgments 

A  second  related  approach  is  to  model  experts’  rating  of 
students’  performance  on  various  task  variables.  In  this 
case,  expert  judgment  is  considered  the  gold  standard 
against  which  to  compare  student  performance.  This 
scoring  approach  has  been  used  successfully  to  model 
expert  and  rater  judgments  in  a  variety  of  applications 
(see,  for  example,  Burstein,  2003;  Clauser  et  al.,  1999; 
Margolis  and  Clauser,  2006;  Williamson  et  al.,  1999). 

Clauser  and  colleagues  have  developed  a  com¬ 
puter-based  assessment  of  patient-management  skills 
(Clauser  et  al.,  1995,  1997,  1999,  2000;  Margolis  and 
Clauser,  2006).  Medical  students  are  presented  with  a 
description  of  the  patient’s  appearance  and  location  in 
the  hospital,  initial  vital  signs,  and  history.  Students 
then  choose  from  four  categories  of  actions  such  as 
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requesting  more  comprehensive  information  about  the 
patient’s  history  or  advancing  the  case  through  simu¬ 
lated  time.  Students’  choices  are  recorded  for  later 
scoring. 

Scoring  of  students’  performance  was  done  by 
using  a  regression  model  that  used  students’  actions 
to  predict  the  students’  score  or  by  scoring  the  stu¬ 
dents’  actions  against  a  set  of  rules  extracted  from  a 
policy  capture  procedure  from  a  panel  of  experts.  Both 
types  of  automated  scoring  procedures  corresponded 
highly  with  human  ratings  of  the  same  performance. 
Subsequent  generalizability  analyses  demonstrated 
that  these  procedures  were  as  generalizable  across 
tasks  as  those  produced  by  the  expert  raters  (Clauser 
et  al„  1995,  1997,  1999,  2000). 

Issues  with  Expert-Based  Scoring  Approaches 

An  issue  with  the  use  of  experts  is  the  selection  of  the 
expert  (Baker  and  Schacter,  1996;  Linn  et  al.,  1991). 
When  experts  are  used  as  the  performance  criterion, 
experts’  biases  may  show  up  on  their  performance  and 
the  subsequent  scoring  model  (Bennett,  2006;  Bennett 
and  Bejar,  1998).  Similarly,  experts’  judgments  may 
be  affected  by  their  content  and  world  knowledge, 
linguistic  competency,  expectations  of  student  compe¬ 
tency,  and  instructional  beliefs  (Baker  and  O’Neil, 
1996).  Baker  et  al.  (1995)  found  that  low  domain 
knowledge  contributed  to  low  reliability  between  rat¬ 
ers’  scoring  written  responses,  and  Burstein  and 
Chodorow  (1999)  found  a  significant  language  by 
scoring  method  interaction  in  the  scoring  of  essays 
using  e-rater,  which  suggested  encoding  of  biases 
(Powers  et  al.,  2001). 

Data-Driven  Techniques 

A  second  class  of  scoring  approaches  is  based  on  data- 
driven  techniques.  Student  performance  data  are  sub¬ 
jected  to  statistical  analyses  or  machine  learning  (e.g., 
artificial  neural  network)  analyses.  Student  perfor¬ 
mances  are  classified  into  different  bins,  where  the 
within-bin  similarity  is  maximized  and  the  between- 
bin  similarity  is  minimized.  Raters  examine  the  char¬ 
acteristics  of  student  performances  in  each  bin  and  an 
interpretation  of  what  those  performances  represent  is 
assigned  to  that  bin.  The  performance  interpretation  is 
the  score.  Subsequent  student  performances  catego¬ 
rized  into  a  bin  inherit  that  bin’s  performance  descrip¬ 
tion.  Studies  have  demonstrated  that  this  approach  can 
generalize  across  tasks  within  a  simulation  and  across 
simulations,  differentiating  between  groups  of  students 
with  presumed  knowledge  differences  (Stevens  and 
Casillas,  2006;  Vendlinski  and  Stevens,  2002). 


As  with  any  data-driven  approach,  initial  validation 
of  the  method  is  complicated  because  there  is  not  an 
a  priori  expectation  of  what  scores  mean  or  an  inherent 
meaning  of  the  classification  scheme.  This  situation 
requires  post  hoc  interpretation  (and  the  potential 
introduction  of  biases)  of  the  categories  after  the  cat¬ 
egories  have  been  formed.  A  second  criticism  is  that 
machine  learning  techniques  can  be  highly  sample 
dependent,  and  the  scoring  process  is  driven  by  statis¬ 
tical  rather  than  theoretical  issues  (Bennett,  2006). 
Thus,  validity  evidence  is  particularly  important  when 
using  data-driven  techniques  to  score  student 
responses  (Powers  et  al.,  2000). 

Domain  Modeling 

A  third  class  attempts  to  model  the  cognitive  demands 
of  the  domain  itself.  The  model  attempts  to  specify 
how  knowledge  and  skills  influence  each  other  and  the 
task  variables  on  which  observations  are  being  made. 
These  approaches  rely  on  a  priori  linking  of  student 
performance  variables  to  hypothesized  knowledge  and 
skill  states.  Student  knowledge  and  skills  are  then 
interpreted  in  light  of  the  observed  student  perfor¬ 
mance.  This  approach  has  been  used  successfully  in  a 
variety  of  domains  and  modeling  types,  from  the  use 
by  Hively  et  al.  (1968)  of  canonical  item  types,  to 
Tatsuoka’s  rule-space  methodology  (Birenbaum  et  al., 
1993;  Katz  et  al.,  1998;  Tatsuoka  and  Tatsuoka,  1997), 
to  the  use  of  Bayesian  networks  to  model  student 
understanding  in  numerous  domains  such  as  dental 
hygiene  skills,  hydraulic  troubleshooting,  network 
troubleshooting,  Web  searching,  circuit  analyses,  and 
rifle  marksmanship  (Bennett  et  al.,  2003;  Chung  et  al., 
2003b,  2006;  Mislevy  and  Gitomer,  1995;  Mislevy  et 
al.,  2002;  Williamson  et  al.,  2006).  The  most  important 
issue  in  domain  modeling  is  identifying  the  essential 
concepts  and  their  interrelationships.  This  challenge  is 
mitigated  via  thorough  knowledge  acquisition  activi¬ 
ties  such  as  cognitive  task  analyses  and  direct  obser¬ 
vation  of  performance.  Because  of  the  importance  of 
the  domain  model,  it  is  critical  to  gather  validity  evi¬ 
dence,  which  is  tantamount  to  validating  the  reasoning 
structure  and  inferences  drawn  by  the  Bayesian  net¬ 
work.  For  examples  of  empirical  validation  techniques, 
see  Chung  (2003b,  2006)  and  Williamson  et  al.  (2000). 


FUTURE  DIRECTIONS 

An  obvious  design  need  for  assessment  of  outcomes  is 
the  automation  or  partial  automation  of  the  task  design 
and  performance  measurement  components.  Various 
efforts  have  been  undertaken  to  do  some  automated 
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assessment  design,  some  involving  very  specific  topics 
(Koedinger  and  Nathan,  2004),  some  involving  sys¬ 
tems  of  training  (Mislevy  and  Riconscente,  2005;  Mis- 
levy  et  al.,  2003;  Riconscente  et  al.,  2005),  and  others 
focused  on  the  cognition  and  content  templates  and 
objects  to  be  used.  In  the  future,  it  will  be  desirable  to 
use  complex  modeling  during  instruction,  post-instruc¬ 
tion,  retention  trials,  and  generalization  and  transfer 
measurement  to  understand  and  locate  the  specific 
areas  within  the  system  where  various  learners  lose 
their  footing,  as  a  combination  of  lack  of  knowledge, 
attention,  motivation,  or  integration  of  content  and 
skill.  Using  clickstream  methods  (Chung  and  Baker, 
2003a;  Schacter  et  al.,  1999;  Stevens  and  Casillas, 
2006)  some  of  these  areas  can  be  pinpointed  now. 
Because  of  the  growing  sophistication  of  computation¬ 
ally  supported  data  collection,  and  the  importance  of 
formative  information  about  the  trainee’s  process  dur¬ 
ing  learning,  the  future  of  outcome  assessment  will 
undoubtedly  merge  with  process  information  to  create 
learner  profiles  (rather  than  scores  or  classifications). 
We  anticipate  that  these  will  have  domain-independent 
components  that  may  predict  from  learned  responses 
(rather  than  from  aptitude  measures)  learners’  likely 
success  in  a  range  of  other  tasks.  We  further  expect 
greater  automation  and  verification  of  ontological 
approaches  and  see  the  study  of  experts  as  providing 
a  continuing  key  to  knowledge  of  performance  mea¬ 
surement  and  its  validity. 
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ABSTRACT 

This  chapter  situates  the  evaluation  of  technology- 
based  programs  in  the  context  of  the  field  of  general 
educational  program  evaluation.  It  begins  with  an 
overview  of  the  main  evaluation  approaches  devel¬ 
oped  for  general  educational  programs,  including 
Tyler’s  early  conception  of  assessing  attainment  of 
program  objectives,  decision-making  approaches ,  nat¬ 
uralistic  evaluation,  and  Kirkpatrick’s  four  levels  for 
evaluating  program  effectiveness .  Following  this  is  an 
overview  of  commonly  used  technology-specific  pro¬ 


gram  evaluation  criteria  and  frameworks.  Strategies 
distilled  from  these  two  fields  are  then  suggested  for 
evaluating  technology-based  learning  programs. 
These  strategies  emphasize  clarifying  the  goal  or  pur¬ 
pose  of  the  evaluation  and  determining  the  informa¬ 
tion  needs  of  the  intended  audiences  of  the  evaluation 
at  the  beginning  of  the  project.  This,  in  turn,  suggests 
the  most  appropriate  evaluation  methodology  to  be 
used.  The  chapter  concludes  with  a  description  of  tools 
that  can  be  used  for  analysis  of  evaluative  data,  fol¬ 
lowed  by  a  brief  discussion  of  the  dissemination  of 
evaluation  results. 
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KEYWORDS 

Effect  size:  A  statistical  measure  of  the  difference 
between  the  mean  of  the  control  group  and  the 
mean  of  the  experimental  group  in  a  quantitative 
research  study. 

Evaluation:  The  process  of  gathering  information 
about  the  merit  or  worth  of  a  program  for  the  pur¬ 
pose  of  making  decisions  about  its  effectiveness  or 
for  program  improvement. 

Naturalistic  evaluation:  An  evaluation  approach  that 
relies  on  qualitative  methodology  but  gives  evalu¬ 
ators  freedom  to  choose  the  precise  method  used 
to  collect,  analyze,  and  interpret  their  data. 

Web  log  file:  A  data  file  residing  on  a  Web  server  that 
contains  a  record  of  all  visitors  to  the  site  hosted 
by  the  server,  where  they  came  from,  what  links 
they  clicked  on,  as  well  as  other  information. 


INTRODUCTION 

New  technologies  that  have  potential  implications  for 
learning  are  being  developed  almost  daily:  blogs,  wikis, 
podcasting,  response  clickers,  interactive  pads  and 
whiteboards,  advanced  educational  games  and  simula¬ 
tions,  and  social  websites,  to  name  a  few.  Although 
individual  teachers  are  always  willing  to  pioneer  the 
use  of  these  technologies  in  their  classrooms,  system 
administrators  often  face  the  challenge  of  having  to 
make  informed  decisions  on  whether  these  technolo¬ 
gies  should  be  adopted  on  a  wider  scale  or  integrated 
into  curricula.  The  main  criterion  for  their  adoption 
frequently  is  how  effective  they  are  at  improving  learn¬ 
ing.  Because  of  the  newness  of  the  technologies,  sel¬ 
dom  do  we  have  any  compelling  evidence  of  their  effec¬ 
tiveness  apart  from  anecdotal  accounts  of  early 
adopters.  This  inevitably  leads  to  a  call  for  a  formal 
evaluation  of  programs  that  employ  the  technology. 

The  goal  of  this  chapter  is  to  provide  guidance  to 
those  charged  with  the  evaluation  of  technology-based 
programs  on  how  to  approach  the  task.  What  is  very 
apparent  from  an  examination  of  the  literature  on  tech¬ 
nology  program  evaluation  is  the  large  gap  between  it 
and  the  literature  on  the  general  field  of  program  eval¬ 
uation.  As  will  be  seen  from  the  discussion  that  fol¬ 
lows,  program  evaluation  has  become  a  mature  field 
of  study  that  offers  a  variety  of  approaches  and  per¬ 
spectives  from  which  the  evaluator  can  draw.  Those 
writing  about  technology  evaluation  tend  either  to 
ignore  the  field  or  to  give  it  only  cursory  attention  on 
the  way  to  developing  their  own  approaches,  so  another 
goal  of  this  chapter  is  to  bridge  the  gap  between  these 
two  fields.  I  take  the  position  that  technology -based 


program  evaluation  is  a  particular  case  of  general  pro¬ 
gram  evaluation;  therefore,  the  methods  and  tools  in 
the  program  evaluation  literature  are  equally  applicable 
to  technology  evaluation.  At  the  same  time,  the  criteria 
that  technology  program  evaluators  offer  can  inform 
the  more  general  evaluation  approaches. 

This  chapter  begins  with  a  discussion  of  the  field 
of  general  program  evaluation  and  outlines  some  of 
the  more  influential  evaluation  approaches  that  have 
emerged.  Following  this  is  an  overview  of  common 
technology  program  evaluation  criteria  and  frame¬ 
works.  Drawing  from  these  two  areas,  I  then  suggest 
strategies  that  can  be  used  to  evaluate  technology- 
based  learning  programs  and  describe  several  new  data 
collection  and  analysis  software  tools  that  can  help 
evaluators. 


GENERAL  PROGRAM 
EVALUATION  MODELS 

Evolution  of  Program  Evaluation 

Prior  to  the  1970s,  educational  program  evaluators 
tended  to  concentrate  on  determining  the  extent  to 
which  a  program  met  its  stated  objectives,  a  model  first 
advocated  by  Tyler  (1942)  in  a  longitudinal  study  of 
schools  in  the  1930s.  That  model  seemed  sensible 
enough  and  served  a  generation  or  two  of  educators 
well, but  during  the  1960s  and  1970s  researchers  began 
developing  new  evaluation  models  that  went  far 
beyond  Tyler’s  original  conception  of  evaluation. 

The  models  that  emerged  were  developed  in 
response  to  the  need  to  provide  accountability  for  large 
U.S.  government  program  expenditures  in  health,  edu¬ 
cation,  and  welfare  during  this  period.  Scriven  (1972) 
argued  that  evaluators  must  not  be  blinded  by  exam¬ 
ining  only  the  stated  goals  of  a  project  as  other  pro¬ 
gram  outcomes  may  be  equally  important.  By  impli¬ 
cation,  Scriven  urged  evaluators  to  cast  a  wide  net  in 
evaluating  the  results  of  a  program  by  looking  at  both 
the  intended  and  unintended  outcomes.  In  fact,  he  went 
as  far  as  advising  evaluators  to  avoid  the  rhetoric 
around  the  program  by  not  reading  program  brochures, 
proposals,  or  descriptions  and  to  focus  only  on  the 
actual  outcomes.  Scriven  also  popularized  the  terms 
formative  and  summative  evaluations  as  a  way  of  dis¬ 
tinguishing  two  kinds  of  roles  evaluators  play:  They 
can  assess  the  merits  of  a  program  while  it  is  still  under 
development,  or  they  can  assess  the  outcomes  of  an 
already  completed  program.  In  practice,  these  two 
roles  are  not  always  as  clearly  demarcated  as  Scriven 
suggests;  nonetheless,  this  distinction  between  the  two 
purposes  of  evaluation  is  still  widely  drawn  on  today. 
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Suchman  (1967)  argued  that  evaluating  the  attain¬ 
ment  of  a  program’s  goals  is  still  essential,  but  more 
critical  is  to  understand  the  intervening  processes  that 
led  to  those  outcomes.  He  suggested  that  an  evaluation 
should  test  a  hypothesis  such  as:  “Activity  A  will  attain 
objective  B  because  it  is  able  to  influence  process  C, 
which  affects  the  occurrence  of  this  objective”  (p.  177). 
Following  this  reasoning,  Weiss  (1972)  showed  how  a 
model  could  be  developed  and  tested  to  explain  how  a 
chain  of  events  in  a  teacher  home  visit  program  could 
lead  to  the  ultimate  objective  of  improving  children’s 
reading  achievement.  This  early  work  led  to  the  devel¬ 
opment  of  an  approach  known  today  as  theory-based 
evaluation,  theory-driven  evaluation,  or  program  theory 
evaluation  (PTE).  PTE  consists  of  two  basic  elements: 
an  explicit  theory  or  model  of  how  the  program  causes 
the  intended  or  observed  outcomes  and  an  actual  eval¬ 
uation  that  is  at  least  guided  by  the  model  (Rogers  et 
al.,  2000).  The  theory  component  is  not  a  grand  theory 
in  the  traditional  social  science  sense,  but  rather  it  is  a 
theory  of  change  or  plausible  model  of  how  a  program 
is  supposed  to  work  (Bickman,  1987).  The  program 
model,  often  called  a  logic  model ,  is  typically  developed 
by  the  evaluator  in  collaboration  with  the  program 
developers,  either  before  the  evaluation  takes  place  or 
afterwards.  Evaluators  then  collect  evidence  to  test  the 
validity  of  the  model.  PTE  does  not  suggest  a  method¬ 
ology  for  testing  the  model,  although  it  is  often  associ¬ 
ated  with  qualitative  methodology.  Cook  (2000)  argues 
that  program  theory  evaluators  who  use  qualitative 
methods  cannot  establish  that  the  observed  program 
outcomes  were  caused  by  the  program  itself,  as  causal¬ 
ity  can  only  be  established  through  experimental  design. 
Generally  speaking,  the  contribution  of  PTE  is  that  it 
forces  evaluators  to  move  beyond  treating  the  program 
as  a  black  box  and  leads  them  to  examining  why 
observed  changes  arising  from  a  program  occurred. 

Decision-Making  Evaluation  Approaches 

During  the  same  period,  other  evaluators  focused  on 
how  they  could  help  educational  decision  makers.  Best 
known  is  Stufflebeam  (1973),  who  viewed  evaluation 
as  a  process  of  providing  meaningful  and  useful  infor¬ 
mation  for  decision  alternatives.  Stufflebeam  proposed 
his  context,  input, process,  and  product  (CIPP)  model, 
which  describes  four  kinds  of  evaluative  activities. 
Context  evaluation  assesses  the  problems,  needs,  and 
opportunities  present  in  the  educational  program’s  set¬ 
ting.  Input  evaluation  assesses  competing  strategies 
and  the  work  plans  and  budgets.  Process  evaluation 
monitors,  documents,  and  assesses  program  activities. 
Product  evaluation  examines  the  impact  of  the  program 
on  the  target  audience,  the  quality  and  significance  of 


outcomes,  and  the  extent  to  which  the  program  is  sus¬ 
tainable  and  transferable.  In  essence,  the  CIPP  model 
asks  of  a  program:  What  needs  to  be  done?  How  should 
it  be  done?  Is  it  being  done?  Did  it  succeed?  Stuffle¬ 
beam  also  reconciled  his  model  with  Scriven’s  forma¬ 
tive  and  summative  evaluation  by  stating  that  formative 
evaluation  focuses  on  decision  making  and  summative 
evaluation  on  accountability. 

Another  popular  approach  that  emerged  was  Pat¬ 
ton’s  (1978)  utilization-focused  evaluation.  Patton 
addressed  the  concern  that  evaluation  findings  are  often 
ignored  by  decision  makers.  He  probed  evaluation  pro¬ 
gram  sponsors  to  attempt  to  understand  why  this  is  so 
and  how  the  situation  could  be  improved.  From  this 
study,  he  developed  not  so  much  an  evaluation  model 
as  a  general  approach  to  evaluation  that  has  only  two 
fundamental  requirements.  First,  he  stated  that  relevant 
decision  makers  and  evaluation  report  audiences  must 
be  clearly  identified.  Second,  he  maintained  that  eval¬ 
uators  must  work  actively  with  the  decision  makers  to 
decide  upon  all  other  aspects  of  the  evaluation,  includ¬ 
ing  such  matters  as  the  evaluation  questions,  research 
design,  data  analysis,  interpretation,  and  dissemina¬ 
tion.  Patton  admitted  that  the  challenge  of  producing 
evaluation  studies  that  are  actually  used  is  enormous 
but  remained  optimistic  that  it  is  possible  and  worth 
attempting. 

Cronbach  (1980),  a  student  of  Tyler,  also  focused 
on  the  decision-making  process.  His  contribution  was 
to  emphasize  the  political  context  of  decision  making, 
saying  that  it  is  seldom  a  lone  person  who  makes 
decisions  about  a  program;  rather,  decisions  are  more 
likely  to  be  made  in  a  lively  political  setting  by  a 
policy-shaping  community.  Cronbach  advocated  that 
the  evaluator  should  be  a  teacher,  educating  the  client 
group  throughout  the  evaluation  process  by  helping 
them  refine  their  evaluation  questions  and  determine 
what  technical  and  political  actions  are  best  for  them. 
During  this  educative  process,  the  evaluator  is  con¬ 
stantly  giving  feedback  to  the  clients,  and  the  final 
evaluation  report  is  only  one  more  vehicle  for  commu¬ 
nicating  with  them.  Unlike  the  other  evaluation  theo¬ 
rists  mentioned  above,  Cronbach  did  not  believe  that 
the  evaluator  should  determine  the  worthiness  of  a 
program  nor  provide  recommended  courses  of  action. 

Naturalistic  Evaluation  Approaches 

At  the  same  time  these  researchers  were  developing 
approaches  that  focused  on  how  evaluation  results  are 
used,  others  concentrated  their  efforts  on  developing 
methods  that  place  few,  if  any,  constraints  on  the  eval¬ 
uator.  Known  as  naturalistic  or  qualitative ,  these 
approaches  give  the  evaluator  freedom  to  choose  the 
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methods  used  to  collect,  analyze,  and  interpret  their 
data.  Stake’s  (1975)  responsive  evaluation  is  one  such 
model.  Stake  was  concerned  that  conventional 
approaches  were  not  sufficiently  receptive  to  the  needs 
of  the  evaluation  client.  He  advocated  that  evaluators 
must  attend  to  actual  program  activities  rather  than 
intents,  respond  to  the  audience’s  needs  for  informa¬ 
tion,  and  present  different  value  perspectives  when 
reporting  on  the  success  and  failure  of  a  program. 
Stake  believed  that  evaluators  should  use  whatever 
data-gathering  schemes  seem  appropriate;  however,  he 
did  emphasize  that  they  will  likely  rely  heavily  on 
human  observers  and  judges.  Rather  than  relying  on 
methodologies  of  experimental  psychology,  as  is  often 
done  in  conventional  evaluations.  Stake  saw  evaluators 
drawing  more  from  the  traditions  of  anthropology  and 
journalism  in  carrying  out  their  studies. 

Two  other  approaches  are  of  interest  in  this  dis¬ 
cussion  of  naturalistic  methods.  First,  is  Eisner’s 
(1979)  connoisseur  ship  model ,  which  is  rooted  in  the 
field  of  art  criticism.  His  model  relies  on  the  evalua¬ 
tor’s  judgment  to  assess  the  quality  of  an  educational 
program,  just  as  the  art  critic  appraises  the  complexity 
of  a  work  of  art.  Two  concepts  are  key  to  Eisner’s 
model:  educational  connoisseurship  and  educational 
criticism.  Educational  connoisseurship  involves  the 
appreciation  of  the  finer  points  of  an  educational  pro¬ 
gram,  a  talent  that  derives  from  the  evaluator’s  expe¬ 
rience  and  background  in  the  domain.  Educational  crit¬ 
icism  relies  on  the  evaluator’s  ability  to  verbalize  the 
features  of  the  program,  so  those  who  do  not  have  the 
level  of  appreciation  that  the  connoisseur  has  can  fully 
understand  the  program’s  features. 

The  second  approach  is  ethnographic  evaluation , 
whose  proponents  believe  can  yield  a  more  meaningful 
picture  of  an  educational  program  than  would  be  pos¬ 
sible  using  traditional  scientific  methods  (Guba,  1978). 
Ethnographic  evaluators  immerse  themselves  in  the 
program  they  are  studying  by  taking  part  in  the  day- 
to-day  activities  of  the  individuals  being  studied.  Their 
data-gathering  tools  include  field  notes,  key  informant 
interviews,  case  histories,  and  surveys.  Their  goal  is 
to  produce  a  rich  description  of  the  program  and  to 
convey  their  appraisal  of  the  program  to  the  program 
stakeholders. 

Kirkpatrick’s  Four  Levels 

Although  it  is  well  established  in  the  human  resource 
development  community,  Kirkpatrick’s  (2001)  four- 
level  model  is  less  known  in  educational  evaluation 
circles  because  it  focuses  on  the  evaluation  of  corpo¬ 
rate  training  programs.  I  have  placed  it  in  a  category 
by  itself  because  it  has  little  in  common  with  the  other 


models  discussed,  as  Kirkpatrick  does  not  emphasize 
negotiation  with  the  decision  makers  nor  does  he  favor 
a  naturalistic  approach.  Kirkpatrick’s  first  writing  on 
the  model  dates  back  to  over  40  years  ago,  but  it  was 
not  until  more  recently  that  he  provided  a  detailed 
elaboration  of  its  features.  Even  though  it  focuses  on 
training  program  evaluation,  the  model  is  still  relevant 
to  general  educational  settings;  for  example,  Guskey 
(2000)  adapted  it  for  the  evaluation  of  teacher  profes¬ 
sional  development  programs. 

Kirkpatrick  proposed  four  levels  that  the  evaluator 
must  attend  to:  reaction,  learning,  behavior,  and 
results.  Reaction  refers  to  the  program  participants’ 
satisfaction  with  the  program;  the  typical  course  eval¬ 
uation  survey  measures  reaction.  Learning  is  the  extent 
to  which  participants  change  attitudes,  improve  their 
knowledge,  or  increase  their  skills  as  a  result  of  attend¬ 
ing  the  program;  course  exams,  tests,  or  surveys  mea¬ 
sure  this  kind  of  change.  The  next  two  levels  are  new 
to  most  educational  evaluators  and  are  increasingly 
more  difficult  to  assess.  Behavior  refers  to  the  extent 
to  which  participants’  behavior  changes  as  a  result  of 
attending  the  course;  to  assess  this  level,  the  evaluator 
must  determine  whether  participants’  new  knowledge, 
skills,  or  attitudes  transfer  to  the  job  or  another  situa¬ 
tion  such  as  a  subsequent  course.  The  fourth  evaluation 
level,  results,  focuses  on  the  lasting  changes  to  the 
organization  that  occurred  as  a  consequence  of  the 
course,  such  as  increased  productivity,  improved  man¬ 
agement,  or  improved  quality.  In  a  formal  educational 
setting,  the  fourth  evaluation  level  could  refer  to 
assessing  how  students  perform  on  the  job  after  grad¬ 
uation.  Kirkpatrick  has  recommended  the  use  of  con¬ 
trol  group  comparisons  to  assess  a  program’s  effec¬ 
tiveness  at  these  two  higher  levels,  if  at  all  possible. 

TECHNOLOGY  EVALUATION 
APPROACHES 

So  far  I  have  concentrated  on  models  that  are  applicable 
to  a  wide  range  of  educational  programs,  whether  or 
not  they  might  involve  technology.  Several  frameworks 
have  been  proposed  specifically  to  assess  technology- 
based  learning,  although  none  has  been  employed  much 
by  researchers  other  than  their  developers.  These 
frameworks  tend  to  recommend  areas  in  which  evalu¬ 
ators  should  focus  their  data  collection,  provide  criteria 
against  which  technology-based  learning  could  be 
judged,  or  provide  questions  for  the  evaluator  to  ask. 
For  example.  Riel  and  Harasim  (1994)  proposed  three 
areas  on  which  data  collection  might  focus  for  the 
evaluation  of  online  discussion  groups:  the  structure  of 
network  environment,  social  interaction  that  occurs 


608 


Models  and  Methods  for  Evaluation 


TABLE  45.1 
CIAO!  Framework 


Context 

Interactions 

Outcomes 

Rationale 

To  evaluate  technology,  we  need  to 
know  about  its  aims  and  the  context 

of  its  use. 

Observing  students  and  obtaining 
process  data  help  us  to  understand 
why  and  how  some  element  works 

in  addition  to  whether  or  not  it 

works. 

Being  able  to  attribute  learning  outcomes  to 
technology  when  it  is  one  part  of  a  multifaceted 
course  is  very  difficult.  It  is  important  to  try  to  assess 
both  cognitive  and  affective  learning  outcomes  (e.g., 
changes  in  perceptions  and  attitudes). 

Data 

Designers’  and  course  teams’  aims 
Policy  documents  and  meeting 
records 

Records  of  student  interactions 

Student  diaries 

Online  logs 

Measures  of  learning 

Changes  in  students’  attitudes  and  perceptions 

Methods 

Interviews  with  technology  program 
designers  and  course  team  members 
Analysis  of  policy  documents 

Observation 

Diaries 

Video/audio  and  computer  recording 

Interviews 

Questionnaires 

Tests 

Source:  Adapted  from  Scanlon,  E.  et  al.,  Educ.  Technol.  Soc.,  3(4),  101-107,  2000. 


during  the  course  or  project,  and  the  effects  of  the 
experience  on  individuals.  Bates  and  Poole’s  (2003) 
SECTION  model  calls  for  the  comparison  of  two  or 
more  online  instructional  delivery  modes  on  the  basis 
of  the  appropriateness  of  the  technology  for  the  targeted 
students,  its  ease  of  use  and  reliability,  costs,  teaching 
and  learning  factors,  interactivity  fostered  by  the  tech¬ 
nology,  organizational  issues,  novelty  of  the  technol¬ 
ogy,  and  how  quickly  courses  can  be  mounted  and 
updated.  Ravitz  (1998)  suggested  a  framework  that 
encourages  the  assessment  of  a  project’s  evolution 
through  interactive  discussion,  continual  recordkeep¬ 
ing,  and  documentation.  Mandinach  (2005)  has  given 
evaluators  a  set  of  key  questions  to  ask  about  an  e-learn- 
ing  program  in  three  general  areas:  student  learning, 
pedagogical  and  intuitional  issues,  and  broader  policy 
issues.  Finally,  Baker  and  Herman  (2003)  have  pro¬ 
posed  an  approach,  which  they  call  distributed  evalu¬ 
ation,  to  deal  with  large-scale,  longitudinal  evaluation 
of  technology.  They  emphasize  clarifying  evaluation 
goals  across  all  stakeholders,  using  a  variety  of  quan¬ 
titative  and  qualitative  measures  ranging  from  question¬ 
naires  and  informal  classroom  tests  to  standardized 
tests,  designing  lengthier  studies  so  changes  can  be 
assessed  over  time,  collecting  data  at  the  local  level  and 
entering  them  into  a  systemwide  repository,  and  pro¬ 
viding  feedback  targeted  at  various  audiences. 

Of  particular  note  because  of  its  origins  and  com¬ 
prehensiveness  is  the  context,  interactions,  attitudes, 
and  outcomes  (CIAO!)  framework  developed  by  Scan¬ 
lon  et  al.  (2000).  The  CIAO!  framework  represents  a 
culmination  of  some  25  years  of  technology  evaluation 
experience  of  the  authors  at  the  Open  University  in  the 
United  Kingdom.  As  shown  in  Table  45.1 ,  the  columns 
in  the  framework  represent  three  dimensions  of  the 
technology-based  learning  program  that  must  be  eval¬ 


uated:  the  context  dimension  concerns  how  the  tech¬ 
nology  fits  within  the  course  and  where  and  how  it  is 
used;  interactions  refers  to  how  students  interact  with 
the  technology  and  with  each  other;  and  outcomes  deals 
with  how  students  change  as  a  result  of  using  the  tech¬ 
nology.  The  first  row  of  the  framework  provides  a  brief 
rationale  for  the  need  to  evaluate  each  of  the  three 
dimensions.  The  second  and  third  rows,  respectively, 
highlight  the  kinds  of  data  that  should  be  collected  for 
each  dimension  and  the  methods  that  should  be 
employed  for  each.  The  authors  point  out  that,  while 
the  framework  has  proven  to  be  very  valuable  in  high¬ 
lighting  areas  in  which  evaluative  data  should  be  col¬ 
lected,  caution  should  be  exercised  in  not  applying  the 
framework  in  an  overly  prescriptive  manner. 

Perhaps  the  most  widely  used  criteria  for  evaluat¬ 
ing  teaching  with  technology  in  higher  education  are 
the  Seven  Principles  for  Good  Practice  in  Undergrad¬ 
uate  Education,  described  in  a  seminal  article  by 
Chickering  and  Gamson  (1987).  Almost  10  years  after 
this  article  was  published,  Chickering  and  Ehrmann 
(1996)  illustrated  how  the  criteria,  which  were  distilled 
from  decades  of  research  on  the  undergraduate  educa¬ 
tion  experience,  could  be  adapted  for  information  and 
communication  technologies.  Briefly,  the  criteria  sug¬ 
gest  that  faculty  should: 

•  Encourage  contact  between  students  and  the 
faculty. 

•  Develop  reciprocity  and  cooperation  among 
students. 

•  Encourage  active  learning. 

•  Give  prompt  feedback. 

•  Emphasize  time  on  task. 

•  Communicate  high  expectations. 

•  Respect  diverse  talents  and  ways  of  learning. 
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Graham  and  colleagues  applied  the  criteria  to 
the  evaluation  of  four  online  courses  in  a  profes¬ 
sional  school  of  a  large  midwestern  American  uni¬ 
versity  (Graham  et  al.,  2000).  The  evaluation  team 
developed  a  list  of  “lessons  learned”  for  online 
instruction,  aimed  at  improving  the  courses  and 
which  correspond  to  the  seven  principles.  Simi¬ 
larly,  Cook  et  al.  (2003a)  applied  the  criteria  to  the 
evaluation  of  a  technology-enhanced  undergradu¬ 
ate  economics  course.  They  used  the  principles  as 
the  basis  of  codes  for  the  qualitative  analysis  of 
open-ended  student  survey  responses  and  assessed 
the  extent  to  which  the  criteria  were  exemplified  in 
the  course. 

Although  the  Seven  Principles  describe  effective 
teaching  from  the  faculty  member’s  perspective,  the 
American  Psychological  Association  has  produced  an 
often-cited  list  of  14  principles  that  pertain  to  the 
learner  and  the  learning  process  (see  http://www.apa. 
org/ed/lcp2/lcp  14.html).  The  learner-centered  princi¬ 
ples  are  intended  to  deal  holistically  with  learners  in 
the  context  of  real-world  learning  situations;  thus, 
they  are  best  understood  as  an  organized  set  of  prin¬ 
ciples  that  influence  the  learner  and  learning  with  no 
principle  viewed  in  isolation.  The  14  principles, 
which  are  grouped  into  four  main  categories,  are  as 
follows: 

•  Cognitive  and  metacognitive  (six  princi¬ 
ples):  Nature  of  the  learning  process;  goals 
of  the  learning  process;  construction  of 
knowledge;  strategic  thinking;  thinking 
about  thinking;  context  of  learning 

•  Motivational  and  affective  (three  princi¬ 
ples):  Motivational  and  emotional  influ¬ 
ences  on  learning;  intrinsic  motivation  to 
learn;  effects  of  motivation  on  effort 

•  Developmental  and  social  (two  principles): 
Developmental  influences  on  learning; 
social  influences  on  learning 

•  Individual  difference  factors  (three  princi¬ 
ples):  Individual  differences  in  learning; 
learning  and  diversity;  standards  and 
assessment 

Bonk  and  Cummings  (1998)  discussed  how  these 
principles  are  relevant  for  the  design  of  online  courses 
from  a  learner-centered  perspective  and  for  providing 
a  framework  for  the  benefits,  implications,  problems, 
and  solutions  of  online  instruction.  By  implication,  the 
APA  principles  could  serve  as  criteria  to  guide  the 
evaluation  of  the  effectiveness  of  technology-based 
learning  environments. 


IMPLICATIONS  FOR  THE 
EVALUATION  OF  TECHNOLOGY 

What  should  be  abundantly  clear  at  this  point  is  the 
surfeit  of  evaluation  approaches,  criteria,  and  models. 
Few  experienced  evaluators,  however,  pick  one  model 
and  adhere  to  it  for  all  of  their  work;  they  are  more 
likely  to  draw  upon  different  aspects  of  several  mod¬ 
els.  Worthen  and  Saunders  (1987,  p.  151)  expressed 
this  well: 

The  value  of  alternative  approaches  lies  in  their  capacity 
to  help  us  think,  to  present  and  provoke  new  ideas  and 
techniques,  and  to  serve  as  mental  checklists  of  things 
we  ought  to  consider,  remember,  or  worry  about.  Their 
heuristic  value  is  very  high;  their  prescriptive  value  seems 
much  less. 

Several  implications  can  be  drawn  from  this  discussion 
of  models  so  far  that  will  help  in  making  decisions 
about  the  design  of  technology-based  program  evalu¬ 
ations.  These  are  summarized  in  Figure  45.1 .  First,  we 
must  clarify  why  we  are  proposing  an  evaluation:  Is  it 
to  assess  a  blended  learning  course  developed  by  a 
faculty  member  who  was  given  a  course  development 
grant?  Is  it  to  evaluate  an  elementary  school  laptop 
computer  initiative?  Is  it  being  conducted  because  stu¬ 
dents  are  expressing  dissatisfaction  with  an  online 
course?  Is  it  to  see  how  an  online  professional  learning 
community  facilitates  pedagogical  change?  The  pur¬ 
pose  of  the  evaluation  will  lead  us  to  favor  one 
approach  over  another;  for  example,  in  the  case  of  the 
faculty  member  developing  a  course,  the  Seven  Prin¬ 
ciples  and/or  the  APA’s  learner-centered  principles 
may  be  good  criteria  to  judge  the  course.  The  Seven 
Principles  may  also  be  appropriate  to  guide  the  eval¬ 
uation  of  the  course  where  there  is  student  dissatisfac¬ 
tion.  On  the  other  hand,  in  the  case  of  the  professional 
program,  Kirkpatrick’s  model  (or  Guskey’s  extension 
of  it)  would  direct  us  not  only  to  examining  teachers’ 
perceptions  of  and  learnings  in  the  community  but  also 
to  studying  the  impact  of  the  program  on  the  classroom 
practice.  Table  45.2  provides  additional  guidance  on 
selecting  a  model  from  among  the  most  widely  used 
ones  for  six  common  program  evaluation  purposes. 
Readers  should  exercise  caution  when  interpreting  the 
table,  as  there  are  no  hard  and  fast  rules  about  what 
model  to  use  for  a  given  purpose.  Rarely  is  one  model 
the  only  appropriate  one  to  use  in  an  evaluation;  how¬ 
ever,  more  often  than  not  some  models  are  better  than 
others  for  a  particular  study. 

We  next  have  to  give  careful  thought  about  who 
the  intended  audiences  of  the  evaluation  report  are  and 
should  plan  on  providing  those  individuals  with  the 
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Figure  45.1  Decisions  for  designing  an  evaluation  study. 


TABLE  45.2 

Evaluation  Models  Best  Suited  for  Particular  Evaluation  Purposes 


Primary  Purpose  of  Evaluation 


Attainment  of 
the  Program’s 

Accreditation 

Development 
of  Theory 

Meeting 

Information 

Needs  of 

Overall 

Goals  and 

Program 

of  the 

about 

Diverse 

Impact  of 

Evaluation  Model 

Objectives 

Improvement 

Program 

Intervention 

Audiences 

Program 

Goal-based  (Tyler,  1942) 

X 

X 

Goal-free  evaluation  (Scriven,  1972) 

X 

X 

X 

Theory-based  (Weiss,  1972) 

X 

X 

X 

X 

Context,  input,  process ,  and  product 
(CIPP)  (Stufflebeam,  1973) 
Utilization-focused  (Patton,  1978) 

X 

X 

X 

X 

Responsive  (Stake,  1975) 
Connoisseurship  (Eisner,  1979) 

X 

X 

X 

X 

Ethnographic  (Guba,  1978) 

X 

X 

X 

X 

Multilevel  (Guskey,  2000; 

X 

X 

X 

Kirkpatrick,  2001) 

Cl  AO!  framework  (Scanlon  et  al.. 

X 

X 

X 

2000) 

Seven  principles  of  good  practice  in 

X 

X 

undergraduate  education 
(Chickering  and  Ehrmann,  1996) 

kinds  of  data  needed  to  take  appropriate  action.  Recall 
Stufflebeam’s  statement  that  the  purpose  of  evaluation 
is  to  present  options  to  decision  makers.  In  a  university 


setting,  the  decision  makers  or  stakeholders  might  be  a 
faculty  member  who  is  teaching  an  online  course,  a  cur¬ 
riculum  committee,  a  technology  roundtable,  a  faculty 
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council,  or  senior  academic  administrators.  The  stake¬ 
holders  in  a  school  setting  could  be  a  combination  of 
parents,  teachers,  a  school  council,  and  the  district 
superintendent.  The  challenge  to  the  evaluator,  there¬ 
fore,  is  to  identify  these  audiences  and  then  find  out 
what  their  expectations  are  for  the  evaluation  and  the 
kind  of  information  they  seek  about  the  program.  Pat¬ 
ton,  Cronback,  and  Stake  all  emphasized  the  critical 
importance  of  this  stage.  The  process  may  involve 
face-to-face  meetings  with  the  different  stakeholders, 
telephone  interviews,  or  brief  surveys.  Because  con¬ 
sensus  in  expectations  is  unlikely  to  be  found,  the 
evaluator  will  have  to  make  judgments  about  the  rel¬ 
ative  importance  of  each  stakeholder  and  whose  infor¬ 
mation  should  be  given  priority. 

With  the  expectations  and  information  needs  in 
hand,  the  study  now  must  be  planned.  We  saw  from 
Scriven’s  perspective  that  all  program  outcomes 
should  be  examined  whether  or  not  they  are  stated  as 
objectives.  My  experience  has  taught  me  not  only  to 
assess  the  accomplishment  of  program  objectives,  as 
this  is  typically  what  stakeholders  want  done,  but  also 
to  seek  data  on  unintended  outcomes,  whether  positive 
or  negative,  as  they  can  lead  to  insights  one  might 
otherwise  have  missed. 

Design  of  Study 

Next  the  evaluator  must  decide  upon  the  actual  design 
of  the  study.  A  major  decision  has  to  be  made  about 
whether  to  embark  on  an  experimental  design  involving 
a  comparison  group  or  a  non-experimental  design.  The 
information  needs  of  the  stakeholders  should  determine 
the  path  to  follow  (Patton,  1978;  Stake,  1975).  If  the 
stakeholders  seek  proof  that  a  technology-based  pro¬ 
gram  works,  then  an  experimental  design  is  what  is 
likely  required.  The  What  Works  Clearinghouse  estab¬ 
lished  by  the  U.S.  Department  of  Education’s  Institute 
of  Education  Sciences  holds  experimental  designs  as 
the  epitome  of  “scientific  evidence”  for  determining  the 
effectiveness  of  educational  interventions  (http://www. 
w-w-c.org).  On  the  other  hand,  if  the  stakeholders  seek 
information  on  how  to  improve  a  program,  then  non- 
experimental  or  qualitative  approaches  may  be  appro¬ 
priate.  Some  even  argue  that  defining  a  placebo  and 
treatment  does  not  make  sense  given  the  nature  of  edu¬ 
cation;  hence,  accumulation  of  evidence  over  time  and 
qualitative  studies  are  a  more  meaningful  means  of 
determining  what  works  (Olson,  2004). 

If  a  decision  is  made  to  conduct  a  randomized 
experimental  study.  Cook  et  al.  (2003b)  offer  some 
helpful  advice.  They  suggest  that,  rather  than  asking  a 
broad  question  such  as,  “Do  computers  enhance  learn¬ 
ing?”  (p.  18),  the  evaluator  should  formulate  a  more 


precise  question  that  will  address  the  incremental 
impact  of  technology  within  a  more  global  experience 
of  technology  use.  The  authors  illustrate,  for  example, 
how  a  study  could  be  designed  around  a  narrower 
question:  “What  effect  does  Internet  research  have  on 
student  learning?”  (p.  19).  Rather  than  simply  com¬ 
paring  students  who  do  research  on  the  Internet  with 
those  who  do  not,  they  created  a  factorial  design  in 
which  the  presence  or  absence  of  Internet  research  is 
linked  to  whether  teachers  do  or  do  not  instruct  stu¬ 
dents  on  best  practices  for  Internet  research.  The  result 
is  four  experimental  conditions:  best  practice  with 
Internet,  best  practice  without  Internet,  typical  Internet 
practice,  and  a  control  group  whose  teacher  neither 
encourages  nor  discourages  students  from  doing  Inter¬ 
net  research.  The  authors’  recommendation  echoes  that 
offered  by  Carol  Weiss  some  time  ago  when  she  made 
the  point  that  the  control  group  does  not  necessarily 
have  to  receive  no  treatment  at  all;  it  can  receive  a 
lesser  version  of  the  treatment  program  (Weiss,  1972). 
This  advice  is  particularly  relevant  when  speaking  of 
technology,  as  it  is  commonly  used  by  students  today 
either  in  classrooms  or  outside  of  school,  so  to  expect 
that  the  control  group  contains  students  who  do  not 
use  technology  would  be  unrealistic. 

A  problem  that  Cook  et  al.  (2003b)  mention  only 
in  passing  is  that  of  sample  size  and  units  of  analysis  — 
key  considerations  in  an  experimental  study.  In  a  report 
commissioned  by  the  U.S.  Institute  of  Education  Sci¬ 
ences,  Agodini  et  al.  (2003)  analyzed  these  issues 
when  developing  specifications  for  a  national  study  on 
the  effectiveness  of  technology  applications  on  student 
achievement  in  mathematics  and  reading.  The  authors 
concluded  that  an  effect  size  of  0.35  would  be  a  rea¬ 
sonable  minimum  goal  for  such  a  study  because  pre¬ 
vious  studies  of  technology  have  detected  effects  of 
this  size,  and  it  was  judged  to  be  sufficiently  large  to 
close  the  achievement  gaps  between  various  segments 
of  the  student  population.  An  effect  size  of  0.35  means 
that  the  effect  of  the  treatment  is  35%  larger  than  the 
standard  deviation  of  the  outcome  measure  being  con¬ 
sidered.  To  achieve  this  effect  size  would  require  the 
following  number  of  students  under  the  given  condi¬ 
tions  of  random  assignment: 

•  Students  randomly  assigned  to  treatments 
would  require  10  classrooms  with  20  stu¬ 
dents  in  each  (total  of  200  students). 

•  Classrooms  randomly  assigned  to  treat¬ 
ments  would  require  30  classrooms  with  20 
students  in  each  (total  of  600  students)  for  a 
study  of  the  effects  of  technology  on  reading 
achievement;  however,  40  classrooms  with 
20  students  (total  of  800  students)  would  be 
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required  for  mathematics  because  of  statis¬ 
tical  considerations  on  the  way  mathematics 
scores  cluster. 

•  Schools  randomly  assigned  to  treatments 
would  require  29  schools  with  20  students 
in  each  (total  of  1160  students). 

The  first  condition  of  random  assignment  of  students 
to  treatment  is  not  likely  a  very  feasible  option  in  most 
schools,  so  the  evaluator  is  left  with  the  choice  of 
random  assignment  to  classrooms  or  to  schools,  both 
of  which  would  require  many  more  students.  The  result 
is  that  an  evaluation  of  technology  using  an  experi¬ 
mental  design  would  likely  be  a  fairly  costly  under¬ 
taking  if  these  guidelines  are  followed. 

Unfortunately,  even  random  assignment  to  class¬ 
rooms  or  schools  may  be  problematic;  therefore,  the 
evaluator  is  left  with  having  to  compare  intact  classes, 
a  design  that  is  weak  (Campbell  et  al.,  1966).  Finding 
teachers  or  students  from  an  intact  class  to  act  as  a 
comparison  group  is  difficult.  Even  if  their  cooperation 
is  obtained,  so  many  possible  competing  hypotheses 
could  explain  any  differences  found  between  experi¬ 
mental  and  comparison  groups  (e.g.,  the  comparison 
group  may  have  an  exceptional  teacher  or  the  students 
in  the  experimental  group  may  be  more  motivated)  that 
they  undermine  the  validity  of  the  findings. 

When  the  goal  of  the  study  is  program  improve¬ 
ment  rather  than  proving  the  program  works,  qualita¬ 
tive  approaches  such  as  those  of  Stake  and  of  Guba 
described  earlier  in  this  chapter  are  particularly  appro¬ 
priate.  Owston  (2000)  argued  that  the  mixing  of  both 
qualitative  and  quantitative  methods  shows  stronger 
potential  for  capturing  and  understanding  the  richness 
and  complexity  of  e-learning  environments  than  if 
either  approach  is  used  solely.  Although  some  meth¬ 
odologists  may  argue  against  mixing  research  para¬ 
digms  ,  I  take  a  more  pragmatic  stance  that  stresses  the 
importance  and  predominance  of  the  research  ques¬ 
tions  over  the  paradigm.  This  approach  frees  the  eval¬ 
uator  to  choose  whatever  methods  are  most  appropriate 
to  answer  the  questions  once  they  are  articulated.  Ulti¬ 
mately,  as  Feuer  et  al.  (2002)  pointed  out,  “No  method 
is  good,  bad,  scientific,  or  unscientific  in  itself;  rather, 
it  is  the  appropriate  application  of  method  to  a  partic¬ 
ular  problem  that  enables  judgments  about  scientific 
quality.” 

Data  Sources  and  Analysis 

When  the  basic  design  of  the  study  is  developed,  the 
next  decision  will  be  to  determine  the  evaluation  data 
sources.  Generally,  the  best  strategy  is  to  use  as  many 
different  sources  as  practical,  such  as  test  scores  or 


scores  on  other  dependent  measures,  individual  and 
focus  group  interviews  of  students  and  teachers,  Web- 
based  survey  data,  relevant  program  documents,  and 
classroom  observation.  The  use  of  multiple  data 
sources  is  standard  practice  in  qualitative  evaluation, 
as  the  need  to  triangulate  observations  is  essential  (Pat¬ 
ton,  2002).  In  experimental  studies,  other  qualitative 
and  quantitative  data  sources  may  be  used  to  help 
explain  and  interpret  observed  differences  on  depen¬ 
dent  measures. 

Log  files  generated  by  Web  servers  are  a  relatively 
new  source  of  data  that  can  be  used  to  triangulate 
findings  from  surveys  and  interviews  when  the  tech¬ 
nology  being  evaluated  is  Web  based.  These  files  con¬ 
tain  a  record  of  communication  between  a  Web 
browser  and  a  Web  server  in  text-based  form.  The  files 
vary  slightly  depending  on  the  type  of  server,  but  most 
Web  servers  record  the  following  information: 

•  Address  of  the  computer  requesting  a  file 

•  Date  and  time  of  the  request 

•  Web  address  of  the  file  requested 

•  Method  used  for  the  requested  file 

•  Return  code  from  the  Web  server  that  spec¬ 
ifies  if  the  request  was  successful  or  failed 
and  why 

•  Size  of  the  file  requested 

Web  server  log  files  do  not  reveal  or  record  the  content 
of  a  Web  browser  request— only  the  fact  that  a  request 
was  made.  Because  each  Web  page  has  a  distinct 
address,  it  is  possible  to  determine  that  a  user  viewed 
a  particular  page.  Log  files  grow  to  be  exceedingly 
large  and  are  often  discarded  by  system  administrators; 
however,  evaluators  can  analyze  the  files  using  com¬ 
mercial  tools  such  as  WebTrends  Log  Analyzer 
(http://www.webtrends.com)  or  freeware  tools  such  as 
AWStats  (http://awstats.sourceforge.net).  Output  from 
the  tools  can  be  in  tabular  or  graphical  format  (see 
Figure  45 .2  for  sample  output) .  The  tools  can  be  used 
by  the  evaluator  to  answer  questions  such  as  what  time 
of  day  or  week  users  were  accessing  the  system,  how 
long  they  were  logged  into  the  system,  what  pages  they 
viewed,  and  what  paths  they  followed  through  the  web¬ 
site.  Figure  45.2  is  typical  of  the  graphical  output  that 
may  be  obtained  on  the  average  number  of  users  vis¬ 
iting  a  website  per  day  of  the  week. 

The  author  and  his  colleagues  have  used  log  file 
analysis  successfully  in  several  technology  evalua¬ 
tion  studies.  In  one  study,  Wideman  et  al.  (1998) 
found  that  students  in  a  focus  group  said  they  made 
frequent  use  of  a  simulation  routine  in  an  online 
course,  but  the  log  files  revealed  that  the  routine  was 
seldom  used.  In  another  study.  Cook  et  al.  (2003a) 
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Figure  45.2  Sample  output  from  log  file  analysis. 

were  able  to  correlate  student  access  to  a  university 
course  website  to  final  course  grades  to  obtain  an  indi¬ 
cator  of  how  helpful  the  site  was  to  students.  The 
researchers  were  able  to  obtain  these  data  because  the 
website  required  students  to  log  in,  and  a  record  of 
each  log-in  appeared  in  the  log  file  which  could  be 
matched  to  the  student  grades.  Log-file  analysis  has 
some  limitations  (Haigh  and  Megarity,  1998),  but  we 
found  that  it  provided  more  and  better  quality  data  than 
are  generated  by,  for  example  ,  the  course  management 
system  WebCT  (http://www.webct.com). 

Another  tool  developed  by  the  author  and  his  col¬ 
leagues  to  aid  in  the  evaluation  of  technology-based 
learning  is  the  Virtual  Usability  Lab  (VULab)  (Owston 
et  al.,  2005).  VULab  was  originally  developed  for  edu¬ 
cational  game  research,  but  it  is  applicable  to  any  Web- 
based  learning  research  where  the  learner’s  computer 
is  connected  to  the  Internet.  The  tool  allows  for  the 
automated  integration  of  a  wide  range  of  sources  of 


data,  ranging  from  user  activity  logs,  online  demo¬ 
graphic  questionnaire  responses,  and  data  from  auto¬ 
matically  triggered  pop-up  questions  (see  example  in 
Figure  45.3)  to  the  results  of  queries  designed  to  auto¬ 
matically  appear  at  key  points  when  users  interact  with 
the  application.  Another  feature  of  VULab  is  its  capa¬ 
bility  to  record  the  screens  and  voice  conversations  of 
remote  users  and  store  the  files  on  the  VULab  server 
without  the  need  to  install  special  software  on  the 
users’  computers.  The  data  that  are  collected  are  stored 
in  an  integrated  database  system,  allowing  for  subse¬ 
quent  data  mining  and  ad  hoc  querying  of  the  data  by 
researchers.  VULab  also  allows  for  ease  of  use  for 
researchers  in  setting  up  the  parameters  for  studies  and 
automatically  monitoring  users  whether  they  are  inter¬ 
acting  with  computers  locally  or  are  scattered  across 
the  Internet.  Owston  et  al.  (2005)  reported  on  how 
VULab  was  used  to  record  student  discussions  when 
they  were  filling  out  an  online  questionnaire  after  play- 
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Figure  45.3  Screen  shot  of  VULab. 
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Figure  45.4  Essential  (E)  and  contributing  (C)  factors  to  the  sustainability  of  innovative  use  of  technology  in  the  classroom.  (Adapted 
from  Owston,  R.D.,  J.  Educ.  Change,  8(1),  61-77,  2007.) 


ing  an  online  game.  The  students  were  asked  on  the 
questionnaire  whether  or  not  they  enjoyed  playing  the 
game,  and  a  rich  discussion  of  several  minutes’  dura¬ 
tion  ensued  among  a  small  group  of  students  playing 
the  game  at  one  computer.  When  it  came  time  to  enter 
their  responses  into  the  questionnaire  form,  they  sim¬ 
ply  entered  “yes”;  thus,  valuable  user  feedback  would 
have  been  lost  if  it  had  not  been  for  the  VULab  record¬ 
ing.  The  tool  also  proved  useful  for  identifying  role 
playing  among  the  groups  of  students  playing  the 
game,  intra- group  competition  and  collaboration,  and 
pinpointing  technical  problems  within  the  game  itself. 

Frequently,  evaluations  involve  collecting  large 
quantities  of  qualitative  data,  such  as  interview  tran¬ 
scripts,  open-ended  responses  to  questionnaires,  dia¬ 
ries,  field  notes,  program  documents,  and  minutes  of 
meetings.  Managing  and  analyzing  these  files  can  be 
simplified  using  qualitative  data  analysis  (QDA)  soft¬ 
ware  tools.  Two  of  the  most  popular  QDA  tools  are 
Atlas.ti  (http://atlasti.com/)  and  NVivo  (http://www. 
qsrinternational.com/).  These  tools  do  not  perform  the 
analysis,  but  they  help  in  the  coding  and  interpretation 
of  the  data.  Both  of  these  tools  also  have  a  feature  that 
allows  researchers  to  visually  map  relationships 
between  codes  that  may  lead  to  theory  development; 
for  example,  Owston  (2007)  studied  factors  that  con¬ 
tribute  to  the  sustainability  of  innovative  classroom  use 
of  technology.  Using  Atlas.ti,  he  mapped  the  relation¬ 
ships  among  codes  and  developed  a  model  (see  Figure 
45.4)  that  helps  explain  why  teachers  are  likely  to 
sustain  innovative  pedagogical  practices  using  technol¬ 
ogy.  Atlas.ti  allows  the  importing  of  audio  and  video 
files  as  well  as  textual  files,  whereas  NVivo  does  not. 


In  Atlas  .ti,  these  files  are  coded  the  same  way  as  textual 
files;  in  NVivo,  the  files  cannot  be  directly  imported 
but  coding  of  external  video  and  audio  files  can  be 
done.  If  a  project  involves  only  audio  or  video,  the  best 
strategy  may  be  to  use  Transana  (http://transana.org) 
which  is  a  free,  open-source  tool  designed  for  the 
analysis  of  these  kinds  of  files.  A  helpful  feature  of 
Transana  is  that  while  audio  or  video  files  are  being 
played  a  typist  can  transcribe  the  voices  directly  into 
a  separate  window  within  the  application. 

An  excellent  website  maintained  by  the  Computer- 
Assisted  Qualitative  Data  Analysis  (CAQDAS)  Net¬ 
working  Project  (see  http://caqdas.soc.surrey.ac.uk/)  in 
the  United  Kingdom  provides  independent  academic 
comparisons  of  popular  qualitative  data  analysis  tools 
and  as  well  as  other  helpful  resources  and  announce¬ 
ments.  Those  new  to  computerized  analysis  of  quali¬ 
tative  data  are  well  advised  to  visit  this  website  for 
guidance  in  selecting  the  most  appropriate  tool  to  use 
in  an  evaluation. 

Dissemination 

A  final  issue  that  needs  addressing  is  the  dissemina¬ 
tion  of  evaluation  findings .  The  American  Evaluation 
Association’s  Guiding  Principles  for  Evaluators  (see 
http://www.eval  .org/Publications/GuidingPrinciples. 
asp)  provides  valuable  advice  to  evaluators  who  are 
disseminating  their  results.  Evaluators  should  com¬ 
municate  their  methods  and  approaches  accurately 
and  in  sufficient  detail  to  allow  others  to  understand, 
interpret,  and  critique  their  work.  They  should  make 
clear  the  limitations  of  an  evaluation  and  its  results. 
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Evaluators  should  discuss  in  a  contextually  appropriate 
way  those  values,  assumptions,  theories,  methods, 
results,  and  analyses  significantly  affecting  the  inter¬ 
pretation  of  the  evaluative  findings.  These  statements 
apply  to  all  aspects  of  the  evaluation,  from  its  initial 
conceptualization  to  the  eventual  use  of  findings. 

Beyond  this,  the  final  report  should  contain  no 
surprises  for  the  stakeholders  if  evaluators  are  doing 
their  job  properly.  That  means  that  there  should  be  an 
ongoing  dialog  between  the  evaluators  and  stakehold¬ 
ers,  including  formal  and  informal  progress  reports. 
This  allows  for  the  stakeholders  to  make  adjustments 
to  the  program  while  it  is  in  progress.  At  the  same 
time,  it  is  a  way  of  gradually  breaking  news  to  the 
stakeholders  if  it  looks  as  though  serious  problems  are 
occurring  with  the  program.  Surprising  stakeholders 
at  the  end  of  a  project  with  bad  news  is  one  way  to 
ensure  that  the  evaluation  report  will  be  buried  and 
never  seen  again!  All  the  evaluation  models  reviewed 
in  this  chapter  encourage,  to  varying  degrees,  contin¬ 
uous  dialog  between  evaluators  and  stakeholders  for 
these  reasons.  The  end  result  should  be  that  the  eval¬ 
uation  report  is  used  and  its  recommendations  or  impli¬ 
cations  are  given  due  consideration. 


CONCLUSIONS 

The  challenge  facing  evaluators  of  technology-based 
programs  is  to  design  studies  that  can  provide  the 
feedback  needed  to  enhance  their  design  or  to  provide 
evidence  on  their  effectiveness.  Evaluators  need  to 
look  broadly  across  the  field  of  program  evaluation 
theory  to  help  discern  the  critical  elements  required 
for  a  successful  evaluation  undertaking.  These  include 
attention  to  aspects  such  as  the  audience  of  the  report 
and  their  information  needs,  deciding  to  what  extent 
the  study  will  be  influenced  by  stated  objectives, 
whether  a  comparative  design  will  be  used,  and  if 
quantitative,  qualitative,  or  a  combination  of  methods 
will  be  brought  into  play.  The  study  should  also  be 
guided  by  the  criteria  and  approaches  developed  for 
or  applicable  to  the  evaluation  of  e-learning.  When 
these  steps  are  taken,  evaluators  will  be  well  on  their 
way  to  devising  studies  that  will  be  able  to  answer 
some  of  the  pressing  issues  facing  teaching  and  learn¬ 
ing  with  technology. 
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ABSTRACT 

This  chapter  presents  theoretical  foundations  and 
empirical  findings  of  the  construct  of  change  agentry 
in  order  to  aid  change  agents  and  scholars  in  effecting 
substantive  organizational  or  educational  change. 
Familiarity  with  this  literature  gives  credibility  to  those 
of  us  in  the  educational  communications  and  technol¬ 


ogy  field  seeking  to  effect  significant  change  through 
collaborative  efforts.  We  begin  by  defining  change 
agentry  and  focusing  on  the  theoretical  foundations  of 
change  theories,  agency  theories,  and  diffusion  theories. 
After  this  consideration  of  theoretical  constructs,  we 
turn  to  an  examination  of  the  empirical  findings  in 
research  studies  associated  with  K-12  school  contexts, 
government  organizations,  corporations,  and  the  health- 
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care  industry.  Our  findings  indicate  three  core  ideas 
central  to  change  agentry:  (1)  connecting  an  organiza¬ 
tion  to  its  environment  is  important,  (2)  flexibly  adap¬ 
tive  change  may  be  a  desired  goal,  and  (3)  local  leaders 
as  well  as  external  supports  are  needed  to  support  any 
successful  change  effort. 


KEYWORDS 

Change  agentry:  The  activity  of  facilitating  change. 

Lethal  adaptation:  An  alteration  to  an  innovation  that 
undermines  the  expected  benefits  of  the  innovation. 

Loose  coupling:  An  organizational  arrangement  char¬ 
acterized  by  significant  freedom  between  levels  in 
the  hierarchy. 

Mutual  adaptation:  The  coevolution  of  both  an  inno¬ 
vation  and  the  environment  in  which  it  is  imple¬ 
mented. 

Perturbances:  Events  that  cause  a  small  disruption  in 
organizational  function;  useful  for  encouraging 
reflection  on  an  organization’s  purposes. 

Punctuated  equilibrium:  A  view  of  organizational 
change  characterized  by  long  periods  of  stasis  and 
short  periods  of  change. 

Reculturing:  Changes  that  involve  examining  the 
assumptions  and  purposes  of  an  organization. 

Restructuring:  Changes  that  involve  organizational 
structure,  new  work  patterns,  or  new  functions. 


DEFINING  CHANGE  AGENTRY 

Suchan  (2005,  p.  17)  defines  change  as  “a  planned 
activity  whose  goal  is  to  realign  the  organization  with 
its  environment.”  Within  instructional  technology  we 
keep  hoping  that  whatever  new  or  improved  technol¬ 
ogies  we  have  to  offer  will  revolutionize  our  class¬ 
rooms  and  our  organizations;  yet,  this  has  rarely  been 
the  case  (Cuban,  1986).  Instead,  we  have  seen  one 
wave  after  another  of  relatively  incremental  but 
minor  changes  with  very  little  substantive  impact  of 
any  technologies  on  the  system  of  the  school  or  orga¬ 
nization.  This  has  led  the  field  of  instructional  tech¬ 
nology  to  take  very  seriously  the  importance  of 
understanding  and  facilitating  change  (Ellsworth, 
2000;  Ely,  1976).  We  have  come  to  understand  that 
to  best  create  the  effective  use  of  our  excellently 
designed  human  learning  environments  we  must  first 
figure  out  how  to  get  people  to  use  our  designs  rather 
than  resisting,  battling,  refusing,  or  sabotaging  our 
efforts.  Although  a  good  design  is  only  as  good  as 
its  actual  implementation,  we  must  recognize  that  our 
innovations  exist  in  immensely  complex  webs  of  rela¬ 


tionships  and  that  understanding,  not  manipulation, 
must  feature  prominently  in  any  discussion  of  change 
agentry.  Indeed,  we  define  change  agentry  as  those 
activities  of  education  and  facilitation  through  which 
organizational  stakeholders  inhabit  a  new  state  of 
their  own  design.  Now,  this  last,  as  we  will  see  in 
this  chapter,  is  not  part  of  the  traditional  understand¬ 
ing  of  change  agentry  but  more  a  part  of  user-design 
(Banathy,  1991;  Carr-Chellman,  2007);  however,  we 
feel  that  true  change  agentry  is,  in  fact,  centered  on 
facilitation  of  communities,  organizations,  and  stake¬ 
holders  . 


THEORETICAL  FOUNDATIONS 
OF  CHANGE  AGENTRY 

This  facilitation  process  is  grounded  in  a  number  of 
theoretical  approaches  to  change.  Van  de  Ven  and 
Poole  (1995)  offered  a  typology  of  four  change  the¬ 
ories  including  life-cycle  theories,  teleological  theo¬ 
ries,  dialectical  theories,  and  evolutionary  theories. 
Although  we  find  this  typology  useful,  a  more  pro¬ 
cess-oriented  distinction  is  used  here  as  we  describe 
individual  and  organizational  change  theories ,  devel¬ 
opmental  change  theories,  and  diffusion  theories. 
Within  this  we  include  what  little  there  is  on  agency 
theories  that  is  relevant. 

Individual/Organizational  Change  Theories 

Individual  and  organizational  change  theories  are  per¬ 
haps  the  most  complicated  set  of  theories  associated 
with  change  agentry  and  surely  the  most  relevant. 
Among  these,  one  of  the  earliest  is  Lewin’s  (1951) 
planned  change  theories,  which  have  given  us  the  typ¬ 
ically  understood  unfreeze-change-refreeze  cycle. 
Lewin’s  work  has  been  debated  and  criticized,  in  part 
due  to  the  recent  phenomenon  of  increasingly  rapid 
cycles  of  freezing  and  unfreezing  which  leave  organi¬ 
zations  in  a  state  of  constant  change. 

A  less  cyclical  theory  of  change  was  posited  by 
Burke  and  Schmidt  (1971),  who  argued  that  the 
group  is  the  proper  level  of  analysis  for  organiza¬ 
tional  change.  Within  such  organizational  change 
theories  there  are  distinctions  between  episodic 
change  (Abernathy  and  Utterback,  1978)  and  contin¬ 
uous  change  (Weick  and  Quinn,  1999).  There  is  a 
traditional  view  of  change  in  which  longer  periods 
of  small,  incremental  changes  are  interrupted  by  brief 
periods  of  discontinuous  and  radical  change  (Brown 
and  Eisenhardt,  1997).  This  episodic  view  of  change 
has  been  termed  punctuated  equilibrium  (Abernathy 
and  Utterback,  1978)  and  produces  the  image  of  a 
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rational  organization  that  works  according  to  plan 
unless  dramatic  external  events  (economic  shifts, 
social  conditions)  force  the  organization  to  change. 
Change  agents  working  in  this  episodic  paradigm  are 
primary  actors  in  the  overall  change  process  as  they 
generate  responses  to  external  events ,  build  manage¬ 
ment  support,  and  manage  implementation  (Weick 
and  Quinn,  1999). 

This  view  is  contrasted  with  continuous  change 
in  which  organizations  act  in  unpredictable  ways  that 
are  based  on  the  actions  of  individual  group  members 
seeking  to  satisfy  their  social  and  professional  needs. 
Weick  and  Quinn  (1999,  p.  375)  described  continu¬ 
ous  change  as  that  in  which  “small  continuous  adjust¬ 
ments,  made  continuously  across  units,  can  cumulate 
and  create  substantial  change.”  This  much  more 
decentralized  view  of  the  change  process,  influenced 
by  chaos  and  complexity  theories  and  elements  of 
complex  system  dynamics,  requires  a  different  role 
for  change  agents.  Change  agentry  becomes  more 
about  responsiveness  and  facilitation  as  opposed  to 
goal  setting  and  motivating  others.  The  differences 
between  these  two  views  of  change,  episodic  and 
continuous,  will  have  great  implications  on  the  role 
of  change  agents. 

French  and  Raven  (1959)  described  change  with 
their  social  influence  theory.  According  to  the  Evi- 
denced-Based  Intervention  Work  Group  (EBIWG)  at 
the  University  of  Wisconsin-Madison,  this  theory 
focuses  on  “the  power  of  one  individual  to  change  the 
beliefs,  attitudes,  or  behaviors  of  another”  (EBIWG, 
2005,  p.  476).  Models  of  social  influence  have  been 
developed  that  contain  such  components  as  physical 
attractiveness,  position  in  the  organizational  hierarchy, 
interpersonal  social  history,  and  the  perceived  ability 
to  distribute  rewards  or  punishments  (Cialdini,  2001; 
French  and  Raven,  1959;  Yukl,  1994).  Individuals  who 
possess  certain  characteristics  are  more  likely  to  be 
change  agents  than  individuals  without  them.  Social 
influence  theory  informs  our  discussion  of  change 
agency  by  emphasizing  the  importance  of  the  people 
who  introduce  and  implement  reforms.  This  theoretical 
framework  shows  that  if  an  unpopular  leader  presents 
a  proposed  change  to  an  organization,  the  reform  may 
be  instantly  undermined  by  the  leader’s  lack  of  social 
influence.  An  approach  that  might  minimize  the  neg¬ 
ative  effect  of  unpopular  individuals  is  to  create  a 
social  space  in  which  groups  of  individuals  can  exam¬ 
ine  data  about  current  organizational  performance  and 
participate  in  dialog  about  improving  performance. 
Such  a  process  would  likely  be  more  time  consuming 
than  management-driven  change,  but  it  would  also  be 
likely  to  increase  stakeholder  commitment  and 
decrease  resistance. 


Developmental  Change  Theories 

Developmental  change  takes  the  traditional  notions  of 
change  beyond  the  focus  on  individual  pieces  to  a 
wider,  broader  notion  of  how  change  occurs  across  an 
entire  system.  Systemic  change  theories  bring  to  the 
foreground  the  ripple  effects  that  cause  disruptions  far 
from  the  site  of  the  initial  change,  much  like  occurs 
in  a  pond  when  a  stone  is  thrown  into  it.  When  change 
happens  in  one  part  of  a  system,  interdependencies 
cause  the  rest  of  the  system  to  react  (Hutchins,  1996). 
The  application  of  these  ideas  to  K-12  schooling  has 
been  undertaken  by  Banathy  (1991),  Squire  and  Reige- 
luth  (2000),  and  Jenlink  et  al.  (1998). 

Another  theory  that  views  change  as  a  develop¬ 
mental  process  is  the  Concems-Based  Adoption  Model 
(CBAM),  for  which  Hall  and  Hord  (1987)  identified 
seven  stages  of  concern  that  characterize  the  attitudes 
of  potential  adopters  toward  a  given  innovation.  These 
stages  range  from  no  knowledge  of  the  innovation  to 
concerns  about  how  the  innovation  might  fit  into  exist¬ 
ing  structures  to  adapting  the  innovation  to  serve  func¬ 
tions  beyond  its  original  intention.  The  focus  here  is 
on  understanding  the  attitudes  of  stakeholders  who,  in 
their  roles  as  street-level  bureaucrats  (Weatherly  and 
Lipsky,  1977)  ultimately  decide  the  fate  of  a  change 
effort. 

The  focus  on  the  use  of  the  innovation  makes 
CBAM  more  appropriate  with  first-order  ( morphosta - 
sis)  as  opposed  to  second-order  change  ( morphogene¬ 
sis )  (Mink  et  al.,  1993).  First-order  changes  that  do 
not  alter  the  core  process  of  an  organization  (such  as 
utilizing  computers  instead  of  textbooks  for  reading 
instruction)  are  easily  observed  with  the  “levels  of  use” 
checklist;  however,  second-order  changes  that  more 
fundamentally  alter  practice  (establishing  a  commu¬ 
nity  relations  office)  are  not  quickly  or  easily  measured 
by  an  outside  observer.  The  need  for  dialog  and  overall 
mindset  change  required  for  second-order  changes  is 
not  accommodated  in  CBAM;  however,  its  utility  for 
the  implementation  of  first-order  changes  has  been 
well  established  empirically  (Ellsworth,  2000).  Work 
in  this  area  of  special  interest  to  educational  technol¬ 
ogists  is  that  of  Caffarella  (2000). 

Diffusion  Theory 

Diffusion  theory  focuses  on  the  rather  unpredictable 
path  that  an  innovation  will  follow  from  its  conception 
to  the  divergent  endpoints  of  implementation  or  rejec¬ 
tion.  Although  theorists  have  sought  to  describe  the 
many  paths  a  given  innovation  might  take  along  the 
way,  it  must  be  said  that  the  number  of  forces  at  play 
in  any  social  organization  make  prediction  a  nearly 
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impossible  task.  In  relation  to  the  change  theories  dis¬ 
cussed  earlier,  these  diffusion  theories  keep  the  inno¬ 
vation  at  the  center  of  the  frame  and  deal  with  cultural, 
environmental,  and  individual  forces  as  bumps  along 
the  road  to  adoption.  Any  serious  discussion  of  diffu¬ 
sion  begins  with  the  seminal  work  of  Rogers  (2003) 
and  his  studies  of  the  diffusion  of  hybrid  com  among 
farmers  in  the  United  States.  Based  on  his  theories  and 
research,  he  identified  five  perceived  attributes  of  an 
innovation  that  influence  its  rate  of  adoption: 

•  Relative  advantage— How  well  the  innova¬ 
tion  outperforms  the  current  methods.  The 
more  potential  adopters  view  the  innovation 
as  better  than  their  current  method,  the  more 
quickly  diffusion  will  take  place. 

•  Compatibility— How  well  the  innovation  can 
peacefully  coexist  with  the  work  environ¬ 
ment  in  which  it  is  to  be  adopted.  If  the 
environment  cannot  accept  the  innovation,  it 
will  not  diffuse  widely. 

•  Complexity— Adopters  are  more  likely  to 
utilize  an  innovation  that  they  understand 
and  that  does  not  take  excessive  effort  to 
leam. 

•  Trialability — The  extent  to  which  an  innova¬ 
tion  can  be  test  driven  before  a  wholesale 
adoption  takes  place.  Users  are  more  likely 
to  try  out,  and  thus  adopt,  an  innovation  if 
it  does  not  cost  them  anything  to  do  so. 

•  Observability— An  innovation  is  more  likely 
to  diffuse  rapidly  if  the  benefits  to  early 
adopters  can  be  seen  by  those  still  hesitant 
about  the  innovation. 

It  has  been  long  expressed  that  diffusion  research  is 
often  biased  toward  successful  innovation  (Ellsworth, 
2000;  Rogers,  2003),  and  this  point  is  not  argued  here 
nor  is  the  point  that  Rogers  original  works  were,  rel¬ 
atively  speaking,  colonialist  in  nature.  But,  the  frame¬ 
works  for  thinking  about  change  and  the  vocabulary 
given  to  change  research  by  Rogers  are  invaluable 
gifts. 

EMPIRICAL  STUDIES 
OF  CHANGE  AGENTRY 

K-12  Change  Agentry 

Perhaps  the  largest  body  of  empirical  work  is  that  of 
change  agentry  in  K-12  (kindergarten  to  twelfth  grade) 
schools.  Empirical  work  focused  on  change  agents  in 
K-12  environments  is  fairly  widespread  and  shows  the 


difficulty  of  changing  the  deeply  ingrained  social  insti¬ 
tution  of  school.  With  regard  to  school  change,  Fullan 
(2000,  p.  581)  wrote  that  "there  has  been  strong  adop¬ 
tion  and  implementation,  but  not  strong  institutional¬ 
ization.”  In  other  words,  although  school  reforms  have 
been  dreamed  up  and  tried  out  regularly,  few  of  them 
seem  to  stick,  and  the  result  is  little  fundamental 
change  in  schooling  practices.  With  several  decades  of 
mostly  failed  school  reforms  behind  us  (Fullan,  2001; 
Sarason,  1990),  attention  is  being  focused  on  what 
processes,  actions,  and  mindsets  are  most  helpful  in 
getting  meaningful  changes  to  sustain  themselves 
within  schools.  In  this  research,  many  things  or  people 
may  serve  as  change  agents:  governmental  policies 
(Borko  et  al.,  2003),  district  leaders  (Spillane,  2002), 
school  principals  (Avissar  et  al.,  2003),  classroom 
teachers  (Olsen  and  Kirtman,  2002),  student  teachers 
(Lane  et  al.,  2003) ,  community  groups  (Arriaza,  2004), 
K-12  and  university  partnerships  (Fishman  et  al., 
2004;  Rust  and  Freidus,  2001),  K-12  and  business 
partnerships  (Corcoran  and  Lawrence,  2003),  and  even 
students  themselves  (Fielding,  2001).  Despite  this  vast 
array  of  “stuff’  that  can  fit  under  the  title  of  change 
agent,  there  are  some  common  characteristics  that 
appear  in  the  empirical  research. 

Change  Agents  Connect  Organizations 
to  Their  Environment 

The  first  clear  message  from  the  research  on  K-12 
change  is  that  change  agents  must  connect  organiza¬ 
tions  to  their  environment.  The  people,  resources,  new 
ideas,  feedback,  and  political  support  that  school  sys¬ 
tems  can  receive  from  this  reaching  out  process  are 
vital  to  any  successful  change  effort.  Despite  the  many 
potential  benefits  to  forging  strong  connections  with 
the  environment,  however,  there  are  risks  as  well.  Ful¬ 
lan  (2000)  termed  this  phase  of  school  change  the 
inside-out  story,  in  which,  paradoxically,  “most  outside 
forces  that  have  moved  inside  threaten  schools  in  some 
way,  but  they  are  also  necessary  for  success”  (p.  582). 
In  other  words,  these  forces  demand  that  schools  take 
those  uncomfortable  glances  in  the  mirror  that  often 
lead  to  positive  change  within  the  institution. 

In  their  survey  study  of  110  Israeli  elementary 
school  principals,  Avissar  et  al.  (2003)  examined  the 
inclusion  of  Israeli  special  education  students  in  regular 
education  classes  and  found  that  the  passage  of  a 
national  law  in  1998  had  a  significant  impact  on  the 
number  of  principals  who  implemented  the  reform.  This 
instance  of  policy  as  change  agent  demonstrates  the 
influence  a  system’s  environment  can  have  on  how 
changes  are  implemented.  While  policies  do  not  trans¬ 
late  directly  into  predictable  cause-and-effect  responses 
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in  schools,  they  do  exert  considerable  influence  on 
the  change  process .  Similarly,  in  an  exploratory  study 
of  a  local  parent  group  organizing  to  put  pressure  on 
the  school  system  to  better  accommodate  the  growing 
number  of  Spanish-speaking  families,  Arriaza  (2004, 
p.  10)  noted  “that  school  reform  initiatives  have 
higher  chances  of  becoming  institutionalized  when 
the  community  actively  participates  as  an  empowered 
change  agent.”  Through  analysis  of  historical  docu¬ 
ments,  interviews  with  participants  in  the  movement, 
and  interviews  with  current  educators,  Arriaza  traced 
one  example  of  a  community  exerting  tremendous 
force  on  schooling  practices.  Although  the  commu¬ 
nity  as  change  agent  featured  in  his  study  was  not 
invited  by  the  school  to  help  facilitate  the  change 
process,  they  were  nonetheless  able  to  exert  enough 
environmental  pressure  to  create  lasting  changes  in 
their  school  system. 

Two-way  lines  of  communication  between  a  com¬ 
munity  and  its  schools  undoubtedly  lead  to  a  healthier 
system  in  terms  of  getting  community  support  and 
meeting  community  needs,  but  there  is  also  a  height¬ 
ened  state  of  uncertainty  because  changes  in  the  eco¬ 
nomic,  social,  or  political  landscape  can  put  unwanted 
pressure  on  schools.  Reaching  out  into  the  environ¬ 
ment  presents  some  danger  in  that  schools  will  be 
working  with  groups  (parents,  businesses,  universities, 
funding  agencies,  etc.)  who  might  have  very  different 
views  of  what  education  can,  and  should,  be.  Change 
agents  will  face  difficulty  in  reconciling  these  diverse 
views  in  pursuit  of  a  successful  reform  effort  (EBIWG, 
2005).  Fishman  et  al.  (2004,  p.  67)  described  a  project 
in  Tennessee  where  “local  politics  created  unforeseen 
challenges  to  a  project  that  was  otherwise  successful.” 
Change  efforts  are  always  subject  to  environmental 
pressures,  so  a  wise  reformer  keeps  a  close  eye  on 
environmental  trends  that  might  impact  an  ongoing 
change.  As  Fullan  (2000,  p.  583)  noted: 

Schools  need  the  outside  to  get  the  job  done.  These 
external  forces,  however,  do  not  come  in  helpful  pack¬ 
ages;  they  are  an  amalgam  of  complex  and  uncoordinated 
phenomena.  The  work  of  the  school  is  to  figure  out  how 
to  make  its  relationship  with  them  a  productive  one. 

Schools  that  actively  seek  energy  for  change  from 
their  environment  can  turn  a  contentious  relationship 
between  a  school  and  its  community  into  a  productive 
one.  In  her  interpretive  case  study  of  a  school  considered 
a  high  implementor  of  information  technology,  Tearle 
(2003)  concluded  that  schools  that  proactively  search 
their  surroundings  for  potential  changes  may  be  more 
likely  to  implement  information  technology  reform  than 
those  schools  who  do  not.  She  found  that  a  “recognised 


model  commonly  adopted  in  the  school  for  implemen¬ 
tation  of  a  ‘project’  or  curriculum  development”  (Tearle, 
2003,  p.  574)  was  a  sign  that  a  school  was  regularly 
interfacing  with  its  environment  and  poised  to  take 
advantage  of  opportunities  that  came  along. 

The  strength  of  the  school-community  relationship 
has  led  some  to  support  the  use  of  intermediary  orga¬ 
nizations  to  buffer  specific  change  efforts  from  the 
constantly  changing  environment  in  which  it  exists. 
Corcoran  and  Fawrence  (2003)  described  a  K-12  and 
corporate  partnership  that  worked  with  four  school 
districts  to  improve  science  teaching.  Their  longitudi¬ 
nal  evaluation  study  combined  survey,  interview,  and 
observational  data  with  student  test  scores  in  science. 
The  authors  were  positive  about  the  role  of  the  external 
nonprofit  organization  that  sponsored  the  program, 
saying  that  (Corcoran  and  Fawrence,  2003,  p.  34): 

Reform  support  organizations  can  help  school  districts 
stay  focused.  They  can  legitimate  strategies  and  policies, 
build  public  support,  and  buy  the  time  to  make  them 
work...  Intermediaries  often  are  able  to  shape  the  stake¬ 
holders’  definition  of  the  “problem”  and  build  a  more 
stable  reform  agenda.  Unlike  schools  and  districts,  they 
are  not  subject  to  direct  political  authority  and  are  more 
focused  in  their  aims. 

Although  change  agentry  should  include  connect¬ 
ing  an  organization  with  its  environment  as  a  top  pri¬ 
ority,  there  must  also  be  an  awareness  that  some  pro¬ 
tections,  such  as  partnering  with  supportive 
organizations,  must  be  created  to  prevent  the  reform 
effort  from  collapsing  under  the  turbulence  of  daily 
life  (Bodilly  et  al.,  2004). 

Additional  research  has  also  supported  the  use  of 
external  change  facilitators  to  increase  the  success  of 
a  change  effort  (Goldenberg,  2003;  Jenlink  et  al.,  1998; 
Fane  et  al.,  2003).  In  a  unique  case.  Fane  and  col¬ 
leagues  (2003)  describe  the  influences  that  change- 
oriented  student  teachers  had  on  their  mentor  teachers 
while  student  teaching  in  the  Fos  Angeles  Unified 
School  District.  Their  study  analyzed  written  student 
teacher  reflections  as  well  as  mentor  teacher  interviews 
to  examine  the  impact  that  preservice  teachers  could 
have  when  trained  to  be  critical  practitioners.  This  case 
of  student  teacher  as  change-agent  emphasizes  the 
understanding  that,  whereas  we  generally  view  insti¬ 
tutional  power  and  the  ability  to  cause  change  as  being 
highly  correlated,  those  with  little  official  power  are 
also  able  to  create  positive  change.  The  external  sup¬ 
ports  these  student  teachers  received  from  the  univer¬ 
sity  (emotional  support,  critical  dialog  in  courses,  etc.) 
are  shown  to  have  been  important  parts  of  their  ability 
to  act  as  change  agents  in  their  placement  schools. 
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An  organization’s  environment  is  also  an  important 
source  for  well-designed  theories  of  change.  These 
theories  can  guide  a  facilitator  along  the  process 
(Reigeluth,  2004).  These  theories  can  connect  change 
facilitators  with  each  other  and  lead  to  refinements  in 
the  reform  process.  Just  a  sampling  of  school  change 
theories  includes  the  Guidance  System  for  Transform¬ 
ing  Education  (GSTE)  (Jenlink  et  al.,  1998),  Step-Up- 
to-Excellence  (Duffy,  2006),  the  professional  develop¬ 
ment  approach  (Caine,  2006),  user-design  (Carr-Chell- 
man  and  Almeida,  2006),  and  chaos  theory  (Reigeluth, 
2004).  When  schools  and  change  agents  are  cognizant 
of  the  theories  that  seek  to  describe  and  predict  change, 
they  have  a  scaffold  that  can  support  them  toward 
seeing  the  process  through  to  completion. 

K-12  Change  Is  Wholly  Dependent 
on  Teacher  Change 

Traditional  planned  change  often  seeks  to  take  the  sci¬ 
entifically  validated  “right”  way  to  do  schooling  and 
transplant  it  uniformly  into  a  diverse  collection  of 
schooling  environments;  however,  current  views  of 
change  tend  to  identify  the  source  of  change  in  the  day- 
to-day  teaching  choices  made  by  teachers  in  our  class¬ 
rooms  ,  rather  than  the  policy  decisions  made  by  reform¬ 
ers  at  a  distance.  Fullan  (2001 ,  p.  115)  noted  that  “edu¬ 
cational  change  depends  on  what  teachers  think  and 
do— it's  as  simple  and  as  complex  as  that.”  Similarly, 
it  has  been  said  that  in  effecting  change,  “challenging 
teachers’  current  thinking  and  guiding  them  toward 
new  understandings  was  central”  (Spillane,  2002,  p. 
396) .  If  teachers  and  their  thoughts  and  actions  are  the 
center  of  school  change,  then  how  can  change  agents 
work  toward  engaging  these  overworked  and  practice- 
oriented  professionals  in  reform  efforts?  Strategies  for 
centering  change  on  teachers  will  guide  this  section. 

It  is  now  taken  as  fact  that  the  “specificity  of  con¬ 
text  (within  which  an  innovation  is  tried  out)  plays  a 
fundamental  role  for  change  to  be  embraced  first  by 
individuals,  then  groups  and,  eventually,  large  numbers 
of  people”  (Arriaza,  2004,  p.  14).  A  study  by  Olsen 
and  Kirtman  (2002)  involving  interviews  with  a  variety 
of  stakeholders  from  36  California  schools  showed  that 
the  importance  of  a  teacher’s  context  is  ignored  at  the 
peril  of  any  proposed  change  effort.  Once  classroom 
doors  are  closed,  “regardless  of  a  school’s  efforts, 
teachers  will  mediate  school  reforms  in  various— and 
identifiable— ways;  schools  may  wish  to  appropriate 
this  force  for  constructive  means  rather  than  observe 
its  occurrence  passively  from  the  sidelines”  (Olsen  and 
Kirtman,  2002,  p.  318).  So,  change  agentry  in  this 
implementation-centered  mindset  must  be  informed  by 
the  classroom  conditions  in  which  a  reform  is  to  be 


implemented.  If  the  change  effort  is  the  seed  that  will 
yield  a  better  school,  then  we  must  pay  as  much  atten¬ 
tion  to  the  soil  as  we  do  to  the  type  of  seed. 

In  the  RAND  change  agent  studies  of  the  1970s, 
the  concept  of  mutual  adaptation  was  developed  to 
explain  the  process  in  which  a  school  would  alter  an 
external  reform  idea  to  meet  its  own  local  needs  (Ber¬ 
man  and  McLaughlin,  1975).  In  their  examination  of 
observational  and  survey  data  from  293  reform 
projects  in  U.S.  schools,  they  concluded  that  both  the 
reform  and  the  school  itself  would  change  during  the 
process  of  implementation.  Given  this  idea,  high  fidel¬ 
ity  with  the  original  design  did  not  necessarily  con¬ 
tribute  to  a  successful  change  effort.  This  notion  “rec¬ 
ognizes  the  importance  of  local  re-invention  of 
innovations  in  order  to  better  match  the  norms  (and 
capacity)  of  the  adopting  organization”  (Fishman  et 
al.,  2004,  p.  66).  Given  the  rationalist  roots  of  the  study 
of  change,  the  term  lethal  adaptation  (Bron  and  Cam- 
pione,  1994)  entered  our  language  to  describe  innova¬ 
tions  that  are  altered  locally  in  a  way  that  their  original 
purposes  are  not  met.  Researchers  in  the  field  have 
said  that  “we  faced  an  inherent  challenge  of  this 
[mutual  adaptation]  approach— variations  that  cause 
innovations  to  become  very  different  than  originally 
envisioned,  potentially  weakening  the  impact  of  the 
innovation”  (Fishman  et  al.,  2004,  p.  66).  As  an  exam¬ 
ple,  Fishman  and  his  colleagues  (2004)  found  that  expe¬ 
rienced  teachers  were  so  comfortable  in  deviating  from 
the  planned  technology  innovation  that  they  left  out  the 
technology  component  altogether,  which,  from  the 
researcher’s  perspective,  sabotaged  the  entire  project. 

Change  agents  must  also  recognize  that  teachers  are 
socialized  to  take  instructional  ideas  from  others  and 
adapt  them  to  the  needs  of  their  students.  In  this  sense, 
teachers  have  always  been  experts  in  mutual  adaptation. 
If  certain  elements  of  an  innovation  are  crucial  to  its 
success,  teachers  must  be  given  a  rationale,  space  to 
discuss  it  with  colleagues,  and  time  to  experiment  for 
themselves.  Fostering  the  view  of  teachers  “as  active 
agents  in  their  own  learning”  (Spillane,  2002,  p.  391)  is 
a  central  task  of  modem  change  agents.  The  example 
of  student  teacher  as  change  agent  given  by  Lane  and 
colleagues  (2003)  is  a  good  example  of  this  mindset. 
Listening  to  training  delivered  by  “experts”  and  reading 
memos  drafted  from  administrators  will  rarely  be  suffi¬ 
cient  to  significantly  change  teacher  practice  (Cuban, 
2001).  For  teachers  to  change  their  practice  in  the  class¬ 
room,  they  must  be  given  both  autonomy  and  support. 

This  shift  in  focus  from  developing  good  programs 
to  enacting  good  implementations  has  led  to  increased 
attention  on  the  process  of  implementing  reforms  in 
particular  schools.  As  the  RAND  change  agent  study 
concluded  (Berman  and  McLaughlin,  1975,  p.  83): 
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. . .  many  project  evaluations  focus  on  educational  treat¬ 
ments  in  an  attempt  to  relate  them  to  student  outcomes. 
These  efforts  may  be  misguided.  Educational  treatments, 
defined  solely  in  terms  of  their  technology  or  method, 
were  only  weakly  related  to  implementation  outcomes, 
because  other  elements  of  project  design  had  stronger 
effects.  The  analysis  showed  that  a  project’s  implemen¬ 
tation  strategy  affected  implementation. 

Change  agents  can  influence  mutual  adaptation  (and 
project  success)  by  their  choice  of  implementation 
strategy.  One  approach  is  through  the  involvement  of 
teachers  in  a  dialog  with  each  other  about  a  planned 
change  effort.  Professional  development  that  allows 
for  regular  teacher-consultant  contact  and  delves  into 
the  specific  issues  of  classroom  implementation  were 
found  to  increase  a  project’s  success  (Berman  and 
McLaughlin,  1975). 

Communicating  reforms  in  ways  that  are  meaning¬ 
ful  to  teachers’  daily  practice  is  also  a  relevant  strategy. 
Borko  et  al.  (2003)  described  this  strategy  in  their  case 
study  of  six  schools  considered  exemplary  models  of 
standards-based  reform  in  Washington  state.  Building 
on  initial  successes,  focusing  on  implementable  pieces 
of  a  larger  reform,  and  “grafting  of  reform  ideas  onto 
familiar  practices”  (Knapp,  1997,  as  cited  in  Borko  et 
al.,  2003,  p.  195)  are  suggested  methods  of  implement¬ 
ing  reforms  in  teacher- friendly  ways.  Another  strategy 
to  assist  reform-weary  teachers  is  to  ensure  that  all 
reforms  undertaken  are  “sustained,  innovative  efforts” 
(Borko  et  al.,  2003,  p.  199).  This  long-term  commit¬ 
ment  to  a  change  effort  can  be  reassuring  to  teachers 
who  are  unfortunately  accustomed  to  losing  support 
when  funding  cycles  or  leadership  tenures  are  cut  short 
(Fishman  et  al.,  2004). 

Change  Agents  Are  Vision  Builders, 

Not  Technicians 

Organizational  climate  was  one  of  three  factors  that 
heavily  influenced  change  in  the  RAND  change  agent 
studies  of  the  1970s.  As  McLaughlin  (1 990,  p.  12)  later 
stated:  “The  local  expertise,  organizational  routines, 
and  resources  available  to  support  planned  change 
efforts  generate  fundamental  differences  in  the  ability 
of  practitioners  to  plan,  execute,  or  sustain  an  innova¬ 
tive  effort.”  Expertise  and  routines  are  deep  structures 
that  often  implicitly  guide  the  work  of  people  in  com¬ 
plex  organizations  such  as  schools.  To  address  these 
deep  structures,  change  agents  must  guide  teachers  to 
ask  some  fundamental  questions  of  themselves:  What 
is  the  purpose  of  our  school?  What  are  some  unmet 
needs  of  our  students?  Is  the  community  benefiting 
from  our  efforts?  Change  agents  undertaking  these 
types  of  questions  are  not  merely  working  as  techni¬ 


cians,  but  are  acting  as  facilitators  in  guiding  teachers 
toward  a  dialog  about  where  their  organization  needs 
to  go. 

Researchers  have  distinguished  the  differences 
between  restructuring  (Schlechty,  1990)  and  recultur¬ 
ing  (Fullan,  2000).  Restructuring  involves  creating 
new  procedures  or  practices  based  on  someone’s  new 
understanding  of  how  schools  should  work,  whereas 
reculturing  involves  digging  slightly  deeper  to  explic¬ 
itly  address  the  underlying  assumptions  that  guide 
everyday  teaching  practices.  Our  conception  of  change 
agentry  aligns  more  closely  with  the  notion  of  recul¬ 
turing.  We  find  that  “being  a  change  agent  includes 
both  vision  (driven  by  perceptions)  and  actions  taken” 
(Avissar  et  al.,  2003,  p.  362).  A  “receptive  school  cul¬ 
ture”  (Newmann  and  Welhage,  1995,  p.  57)  is  not  one 
in  which  passive  teachers  do  as  they  are  told,  but  one 
in  which  professional  learning  is  a  value  held  in 
esteem.  This  is  a  value  that  is  clearly  not  yet  present 
in  all  schools,  but  Cochran-Smith  (1991)  found  such 
a  professional  community  among  preservice  teachers 
and  certainly  many  school  staffs  exhibit  similar  char¬ 
acteristics.  See  the  section  on  individual  differences 
below  for  more  of  this  discussion. 

Change  agents  are  also  responsible  for  the  cultiva¬ 
tion  of  a  common  vision  of  reform  amongst  multiple 
stakeholders.  Establishing  this  vision  takes  invest¬ 
ments  in  time  and  capacity  building  (Jenlink  et  al., 
1998),  but  change  agents  who  do  this  will  find  that 
teachers  will  "persevere  despite  the  vicissitudes  of  the 
change  process”  (Corcoran  and  Lawrence,  2003,  p. 
34).  Vision  development  need  not  be  the  sole  respon¬ 
sibility  of  administrators  or  outside  consultants,  how¬ 
ever.  In  their  study  of  school  technology  coordinators 
in  New  Zealand,  Lai  and  Pratt  (2004)  found  that  impor¬ 
tant  aspects  of  their  job  included  visioning  and  plan¬ 
ning  to  meet  that  vision.  The  most  important  aspect  of 
creating  a  vision  is  not  where  it  comes  from  but  that 
it  is  communicated  and  accepted  by  a  wide  variety  of 
stakeholders. 

Although  the  creation  of  a  vision  is  essential,  one 
of  the  defining  characteristics  of  a  vision  is  that  an 
individual  or  school  can  take  multiple  pathways  to 
meet  this  vision.  As  an  example  of  this  loose  coupling, 
Newmann  and  Welhage  (1995,  p.  37)  studied  schools 
in  which  "the  mission  for  learning  was  powerful 
enough  to  guide  instruction,  but  also  flexible  enough 
to  encourage  debate,  discussion,  and  experimentation 
within  the  framework.”  Their  meta-study  of  school 
reform  research  involving  four  separate  studies  and 
over  1000  U.S.  schools  concluded  that  reforms  were 
likely  to  be  successful  when  teachers  were  encouraged 
to  find  their  own  pathways  while  still  adhering  to  an 
overarching  vision. 
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Get  Leadership  Approval  and  Participation 

Another  nearly  universal  characteristic  of  effective 
change  is  getting  the  support  of  leadership.  Borko  and 
colleagues  (2003,  p.  191)  noted  in  their  study  of  school 
change  in  Washington  state  that  "the  primary  grade 
facilitator  for  literacy  characterized  her  [the  princi¬ 
pal's]  leadership  as  ‘very  strong’  and  noted  that  it  was 
an  ‘extraordinarily  important’  factor  in  the  school’s 
success  in  attaining  its  reform  goals.”  Although  prin¬ 
cipal  leadership  and  participation  are  important  for  the 
success  of  a  reform  effort,  their  study  showed  that 
teacher  leadership  is  important  as  well  (Borko  et  al., 
2003,  p. 196): 

Leadership — both  principal  leadership  and  distributed 
leadership — was  a  key  factor  in  the  success  of  both 
schools’  reform  efforts.  In  fact,  it  was  perhaps  the  most 
important  factor  because  of  its  impact  on  the  other  five 
dimensions  of  school  capacity,  [italics  in  original] 

Other  researchers  have  described  the  benefits  of  the 
participation  of  teacher  leaders  in  change  (Fishman  et 
al.,  2004;  Lai  and  Pratt,  2004;  Olsen  and  Kirtman, 
2002;  Tearle,  2003),  of  principal  participation  in 
change  (Goldenberg,  2003),  and  of  central  office  par¬ 
ticipation  in  change  (Corcoran  and  Lawrence,  2003). 
Strong  leadership  and  the  allocation  of  resources  and 
time  are  likely  signals  to  teachers  that  a  particular 
reform  effort  is  a  legitimate  one.  Such  leadership  is 
clearly  not  yet  present  in  all  schools  (Newmann  and 
Welhage,  1995). 

Creation  of  Community  and  Individual 
Professional  Development 

It  has  been  stated  that  classrooms  of  the  21st  century 
look  strikingly  similar  to  classrooms  of  the  late  19th 
century  in  terms  of  curriculum,  pedagogy,  and  assess¬ 
ment  and  that  this  industrial-age  paradigm  of  schooling 
based  on  sorting  and  ranking  students  does  not  meet 
the  needs  of  our  current  information-age  society 
(Reigeluth,  1999).  If  schools  are  to  change  to  meet  the 
demands  of  current  society,  then  teachers  must  learn 
new  ways  of  doing  their  work.  Honing  these  new  meth¬ 
ods  will  require  dialog  and  experimentation  to  find 
instructional  practices  that  work  for  particular  students 
in  particular  classrooms;  therefore,  the  creation  of  a 
professional  community  that  serves  to  build  both  indi¬ 
vidual  and  collective  capacity  is  a  key  element  of  K-12 
change  agentry  (Corcoran  and  Lawrence,  2003;  Ful- 
lan,  2001).  The  importance  of  addressing  local  capac¬ 
ity  has  been  identified  as  especially  important  in  young 
reform  efforts  (Borko  et  al.,  2003).  Understandably, 
even  extremely  high  levels  of  internal  or  external  pres¬ 


sure  for  change  will  not  be  successful  if  the  teachers 
do  not  know  how  to  do  what  is  being  asked  of  them. 

Delivering  new  teaching  methods  under  the  train¬ 
ing  paradigm  of  professional  development  has  had  a 
suspect  history,  but  “what  does  make  a  difference  is 
reculturing:  the  process  of  developing  professional 
learning  communities  in  the  school.  ...  Structures  can 
block  or  facilitate  this  process,  but  the  development  of 
a  professional  community  must  become  the  key  driver 
of  improvement”  (Fullan,  2000,  p.  582).  Professional 
learning  communities  are  groups  of  teachers  and 
school  leaders  who  critically  examine  their  teaching 
practice  and  seek  new  ways  to  address  unmet  student 
needs.  McLaughlin  (1990)  identified  the  Bay  Area 
Writing  Project,  the  Puget  Sound  Consortium,  and  the 
Urban  Math  Collaborative  as  groups  that  capitalized 
on  existing  teacher-led  communities  to  foster  change 
in  teaching  practice.  The  reculturing  that  fosters  the 
growth  of  professional  communities  allows  teachers, 
often  dismissed  as  mere  technicians  in  the  change  pro¬ 
cess,  to  have  considerable  influence  on  the  design  and 
implementation  of  classroom  reforms.  In  their  study 
of  student  teachers  in  Los  Angeles  area  schools,  Lane 
et  al.  (2003)  found  that  the  expertise  that  mentor  teach¬ 
ers  were  accorded  by  their  student  teachers  actually 
fostered  the  critical  dialog  that  led  to  reflective  practice 
and  professional  growth.  Because  the  mentors  were 
not  threatened  by  their  student  teachers,  they  were 
perhaps  more  willing  to  enter  into  this  important  pro¬ 
cess.  Corcoran  and  Lawrence  (2003,  p.  26)  noted  in 
their  study  of  a  science  reform  program  in  New  Jersey 
and  Pennsylvania  schools: 

The  increased  respect  shown  for  the  clinical  expertise  of 
teachers  and  the  expectation  that  they  should  be  con¬ 
sulted  in  the  design  of  policies  and  programs  and  changes 
in  district  leaders'  perceptions  of  how  teachers  learn  have 
contributed  to  these  cultural  changes.  So  too,  have  the 
collegial  cultures  of  the  summer  workshops  and  the  study 
groups  fostered  by  MISE. 

Many  reforms  have  been  upended  by  the  fact  that 
teachers  are  not  supported  (structurally  or  socially)  in 
improving  their  practice  with  others.  Changing  this 
situation  and  creating  the  possibility  for  teachers  to 
develop  a  professional  community  may  have  a  positive 
effect  on  both  the  adoption  rate  of  reforms  and  the 
quality  of  implementation. 

Ignoring  professional  development  can  sidetrack  a 
change  effort  (Lai  and  Pratt,  2004),  as  can  doggedly 
pursuing  the  appearance  of  a  professional  community 
without  allocating  adequate  time,  physical  space,  and 
material  resources  (Fishman  et  al.,  2004).  Schools  that 
wish  to  change  should  see  the  professional  community 
as  a  key  component  in  their  preparation  for  change. 
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for  it  can  both  weed  out  reforms  that  are  unlikely  to 
succeed  as  well  as  build  support  for  those  that  have 
potential  for  improving  teaching  and  learning  in  the 
local  context. 

In  their  study  of  restructuring  in  36  California 
schools,  Olsen  and  Kirtman  (2002)  found  that  the 
school-wide  climate  was  a  primary  influence  on 
whether  or  not  individual  teachers  took  reform  ideas 
and  used  them  to  change  their  classroom  practice.  Sup¬ 
porting  this  view,  Tearle  (2003),  in  her  case  study  of 
a  school  with  high  implementation  of  information 
technology,  found  that  the  existence  of  a  professional 
community  aided  the  adoption  of  the  innovation.  The 
school’s  professional  community  provided  a  frame¬ 
work  for  trying  out  experimental  practices  as  well  as 
a  cadre  of  leaders  who  would  take  the  innovation  back 
to  their  departments  and  aid  in  diffusion. 

Individual  Differences  Matter 

It  is  fitting  that  this  discussion  of  K-12  change  agentry 
ends  with  the  admission  that  all  of  the  general  topics 
mentioned  above,  although  supported  by  empirical 
evidence,  are  subject  to  the  immense  variety  of  indi¬ 
vidual  characteristics  that  help  to  define  a  specific 
reform  effort  at  a  specific  school.  These  unique  char¬ 
acteristics  have  led  research  away  from  the  fidelity 
perspective  of  systematic  implementation  to  an 
“implementation  perspective”  (McLaughlin,  1990) 
that  highlights  the  negotiation  involved  in  any  change 
process.  As  McLaughlin  (1990,  p.  13)  stated,  “vari¬ 
ability  ...  signals  a  healthy  system,  one  that  is  shaping 
and  integrating  policy  in  ways  best  suited  to  local 
resources,  traditions,  and  clientele”  (p.  13).  These  indi¬ 
vidual  quirks  that  define  a  local  system  are  absolutely 
critical  in  the  decisions  made  in  the  process  of  change. 
A  brief  examination  of  some  of  the  individual  differ¬ 
ences  that  were  uncovered  in  K-12  change  research 
will  illustrate  this  point. 

In  their  study  of  Israeli  school  principals’  views 
and  practices  concerning  a  state-mandated  special  edu¬ 
cation  innovation,  Avissar  et  al.  (2003)  found  that  the 
age  of  the  school  principal  had  an  important  influence 
in  how  this  particular  reform  was  interpreted  and 
implemented.  Their  survey  research  of  1 10  elementary 
school  principals  found  that  the  older  a  principal,  the 
less  likely  he  or  she  was  to  implement  the  full-inclu¬ 
sion  practices  mandated  by  the  government.  Fishman 
et  al.  (2004)  identified  comfort  with  technology  and 
comfort  level  in  deviating  from  a  lesson  plan  as  teacher 
characteristics  that  greatly  influenced  their  technol¬ 
ogy-based  change  efforts.  Also  making  reference  to 
individual  characteristics  is  the  warning  of  Jenlink  et 
al.  (1998)  that  internal  change  facilitators,  no  matter 


how  talented,  may  carry  political  baggage  that  can 
upset  the  reform  process.  Awareness  of  early  adopters 
who  generate  interest  and  resistors  who  disagree  with 
the  goals  of  reform  is  an  essential  aspect  of  a  successful 
K-12  reform. 

Change  Agentry  in  Non-K-12  Contexts 

Although  the  majority  of  the  empirical  work  presented 
in  this  chapter  is  on  K-12  change,  valuable  research 
from  other  fields  has  examined  change  agentry  in  areas 
such  as  government,  business,  and  healthcare.  A  brief 
examination  of  this  research  both  confirms  findings 
from  the  K-12  research  and  adds  additional  insights 
into  the  discussion  of  change  agentry  which  can  be 
very  enlightening  for  educational  technologists  and 
others  interested  in  changing  human  learning. 

Government  Change  Agentry 

In  their  case  study  involving  a  dozen  U.K.  civil  ser¬ 
vants  working  together  over  15  months,  Kakabadse 
and  Kakabadse  (2003)  concluded  that  cultural  change 
was  more  important  than  structural  change  when  insti¬ 
tutionalization  is  the  goal.  Their  intensive  work  on  the 
process  of  collaborative  inquiry  (Cl)  was  initiated  with 
the  development  of  shared  values  that  served  as  spring¬ 
boards  for  a  critique  of  current  practice.  They  also 
discussed  the  importance  of  perseverance  in  the  diffi¬ 
cult  task  of  uncovering  unexamined  values  and  critiqu¬ 
ing  deeply  held  personal  assumptions  (Kakabadse  and 
Kakabadse,  2003,  p.  379): 

Cl  is  a  challenging  experience  as  the  process  of  inquiry 
confronts  the  underlying  assumptions,  values,  power 
base  and  established  ways  of  working  within  any  situa¬ 
tion.  Cl  requires  time,  resources,  social  skills,  and  the 
ability  to  share  personal  experiential  knowledge  with  the 
group. 

As  in  the  K-12  context,  it  is  important  to  have  these 
conversations  and  to  allow  for  the  appropriate  amount 
of  time  and  supports  to  complete  the  process. 

In  a  large-scale  survey  study  conducted  by  Sim¬ 
mons  and  Simmons  (2004),  it  was  concluded  that  gov¬ 
ernmental  change  is  a  complicated  process  that  encom¬ 
passes  nearly  every  issue  imaginable.  The  researchers 
found  racial  and  educational  differences  to  be  at  the 
heart  of  many  of  the  differences,  but  they  state  that, 
“no  single  contextual,  political,  or  institutional  theory 
we  considered  explains  all  the  conflict  over  government 
form”  (p.  386).  The  complexities  of  change  on  the  scale 
of  city  government  are  so  great  that  no  single  theory 
can  explain  it  all.  The  importance  of  individual  differ¬ 
ences  and  supporting  open  dialog  are  echoed  here. 
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Corporate  Change  Agentry 

Since  the  early  operations  research  work  by  Ackoff 
(1974),  business  environments  have  regularly  utilized 
change  agents ,  often  referring  to  them  as  external  con¬ 
sultants  (Bennis,  1969,  as  cited  in  Kendra  and  Taplin, 
2004).  This  definition  of  a  consultant  is  narrower  than 
our  definition  of  change  agentry  described  at  the  outset 
of  this  chapter,  but  a  brief  examination  of  these  con¬ 
sultants  should  help  in  developing  a  fuller  sense  what 
a  change  agent  is. 

These  external  change  agents  have  been  valued  for 
their  “ability  to  affect  the  organization’s  power  struc¬ 
ture  in  ways  that  employees  as  change  agents  cannot 
and  because  they  are  less  subject  than  employees  to 
implicit  and  explicit  organizational  rewards  and  pun¬ 
ishments”  (Kendra  and  Taplin,  2004,  p.  21).  This  view 
is  very  much  influenced  by  the  work  of  French  and 
Raven  (1959).  External  consultants  have  the  ability  to 
openly  criticize  people  and  practices  without  sacrific¬ 
ing  their  careers,  potentially  providing  a  less  biased 
view  of  change  efforts.  Internal  change  agents, 
although  likely  to  have  better  knowledge  of  the  change 
context  and  the  key  stakeholders,  may  be  influenced 
by  their  vested  interests  in  the  organization,  leading  to 
a  corruption  of  the  change  process.  This  is  a  tradeoff 
that  must  be  made  by  organizations  when  selecting 
people  to  act  as  change  agents. 

So,  what  are  the  qualities  that  researchers  have 
found  useful  in  these  external  consultants?  There 
seems  to  be  relative  agreement  from  business  research 
on  three  characteristics  of  successful  change  agents: 
excellent  communication  skills,  the  ability  to  function 
with  loose  coupling  between  management  and 
employees,  and  the  ability  to  cause  perturbances  in 
organizational  practice. 

Communications  skills  appear  most  often  in  the 
literature  as  an  important  characteristic  of  successful 
change  consultants  (Suchan,  1995;  Weick  and  Quinn, 
1999).  Their  work  can  be  thought  of  as  that  of  a  trans¬ 
lator  who  brings  an  idea  of  change  into  fruition.  This 
requires  that  communication,  both  formal  and  infor¬ 
mal,  take  place  between  the  consultant  and  stakehold¬ 
ers.  Sumner  (1999),  in  her  comparative  case  study  of 
seven  corporations  implementing  large-scale  informa¬ 
tion  technology  systems,  found  that  in  five  of  the  seven 
cases  communication  played  a  key  role  in  the  success 
of  the  innovation.  Similarly,  a  survey  study  of  cell 
phone  adoption  by  Vishwanath  and  Goldhaber  (2003) 
of  225  non-users  of  cell  phones  showed  that,  for  late 
adopters,  contact  between  customers  and  change 
agents  (salespeople)  was  an  important  variable  in  the 
adoption  process.  The  notion  of  speaking  differently, 
not  just  arguing  better,  contributes  to  an  understanding 


of  the  complexities  of  communication  for  consultants 
(Rorty,  1989,  as  cited  in  Weick  and  Quinn,  1999).  The 
communication  between  potential  adopters  and  cell 
phone  salespeople  decreased  the  perceived  complexity 
of  the  innovation,  which,  according  to  Rogers’  diffu¬ 
sion  theory  (2003),  increases  the  likelihood  of  adop¬ 
tion. 

Business-oriented  researchers  have  also  noted  the 
importance  of  a  change  agent’s  ability  to  foster  loose 
coupling  between  management  and  employees  (Orton 
and  Weick,  1990;  Weick,  1976).  As  described  in  the 
previous  section  on  the  creation  of  vision  in  K-12 
schools,  loose  coupling  is  predicated  on  the  creation  of 
a  shared  organizational  vision  that  guides  the  actions 
and  intentions  of  all  members.  Under  this  umbrella  of 
guidance,  workers  are  given  freedom  to  seek  multiple 
pathways  to  pursue  the  organizational  vision.  In  their 
study  of  the  work  relationships  of  Hollywood  film 
actors,  Faulkner  and  Anderson  (1987)  concluded  that 
the  loosely  coupled  organization  of  directors,  producers, 
actors,  and  technicians  led  to  the  creation  of  small  niches 
in  which  previously  successful  workers  worked  together 
repeatedly,  ensuring  future  prosperity.  The  result  is  more 
movies  that  sell  a  lot  of  tickets  and  fewer  flops. 

In  a  multiple-case  study  of  six  companies  in  the 
information  technology  industry.  Brown  and  Eisen- 
hardt  (1997)  used  interviews,  observations,  and  envi¬ 
ronmental  analysis  to  determine  factors  related  to  the 
success  of  multiple-product  innovations.  They  con¬ 
cluded  that  loose  coupling  is  one  of  three  characteristics 
seen  most  often  in  companies  successful  in  implement¬ 
ing  multiple  product  innovations.  By  contrast,  Dubois 
and  Gadde  (2002)  argued,  based  on  their  previous 
research,  that  the  loosely  coupled  nature  of  the  con¬ 
struction  business,  although  beneficial  in  terms  of  opti¬ 
mizing  the  time  and  costs  of  projects,  may  hurt  inno¬ 
vation  in  the  field.  Loose  coupling  does  encourage  local 
experimentation  and  innovation,  but  without  formal 
bonds  by  which  successful  innovations  might  be  shared 
the  construction  field  as  a  whole  is  slow  to  develop. 
The  lesson  from  these  conflicting  viewpoints  may  be 
that  change  agents  should  encourage  loose  coupling 
between  management  and  workers  while  insisting  on 
regular  communication  so  cross-pollination  can  occur. 

A  final  characteristic  of  change  agents/consultants 
in  the  literature  on  corporate  change  is  the  ability  to 
cause  perturbations  in  daily  practice  that  can  help  ini¬ 
tiate  change.  Suchan  (2005,  p.  18)  gave  a  clear  ratio¬ 
nale  for  why  this  is  an  important  change  agent  quality: 

To  overcome  inertia,  at  least  initially,  a  disruption  is 
needed  that  can  open  a  space  to  infuse  new  energy  into 
the  organizational  system  (Pfeifer,  1997).  The  source 
of  that  disruption  could  be  a  new  technology,  a  major 
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organizational  opportunity  (e.g.,  a  merger  or  acquisition), 
an  organizational  crisis  or  potential  crisis,  or  even  the 
setting  of  goals  that  may  be  virtually  impossible  to  attain. 

Change  agents  are  constantly  bumping  against  the 
inertia  of  everyday  activities  within  the  organization. 
Successful  change  consultants  are  able  to  disrupt  this 
hypnotic  flow  of  events  in  ways  that  do  not  cause 
excessive  stress  on  individual  members  but  encourage 
a  critical  self-examination  of  daily  practices.  In  a 
health  study  with  implications  for  the  disruption  of 
habitual  processes,  Prochaska  and  Norcross  (2001) 
examined  smokers  and  weight-loss  candidates.  They 
found  that  individuals  went  through  six  phases  during 
the  behavior  change  process:  precontemplation,  con¬ 
templation,  preparation,  action,  maintenance,  and  ter¬ 
mination.  It  is  noteworthy  that  only  in  one  of  the  six 
phases  are  individuals  actually  engaged  in  behavioral 
change— and  the  rest  is  preparation  and  maintenance. 
Prochaska  and  Levesque  (2001)  expanded  this  line  of 
inquiry  to  organizations  by  developing  their  Decisional 
Balance  Inventory  survey  instrument.  They  reported 
over  a  dozen  studies  that  have  utilized  this  instrument, 
ranging  from  higher  education  to  food  service  to  retail 
sales  and  concluded  that  using  their  stage -based  para¬ 
digm  of  change  brings  more  people  into  the  process 
and  allows  change  agents  to  tailor  change  activities 
differently  for  different  individuals  or  work  groups 
who  may  be  at  varying  levels  of  readiness  for  change. 

The  importance  of  perturbing  everyday  practice  is 
also  highlighted  in  Brown  and  Eisenhardt’s  (1997)  find¬ 
ing  that  the  six  product  development  managers  in  their 
study  “balanced  between  the  rigidity  of  planning  and 
the  chaos  of  reacting  by  frequently  probing  the  future 
using  a  variety  of  low-cost  lenses  . . .  experimental  prod¬ 
ucts,  futurists,  strategic  partnerships,  and  frequent  meet¬ 
ings”  (p.  16).  The  results  of  these  probes  into  the  future 
might  provide  information  to  the  group  that  would 
cause  important  shifts  in  their  vision  of  the  organiza¬ 
tion.  Scholars  have  noted  the  importance  of  providing 
increased  organizational  performance  data  to  employ¬ 
ees  as  an  early  step  in  the  change  process  (Dooley, 
1997).  Providing  such  data  and  providing  for  structured 
time  in  which  people  can  talk  about  it  are  recommended 
tactics  for  jolting  an  organization  out  of  its  unques¬ 
tioned  routines.  Change  consultants  working  in  corpo¬ 
rate  settings  are  skillful  in  disrupting  the  flow  of  every¬ 
day  activities  just  enough  to  create  some  dissonance 
that  will  bring  out  the  creative  energies  of  the  group. 

Healthcare  Change  Agentry 

The  importance  of  change  agentry  in  the  medical 
community  is  well  stated  by  Redfern  and  Christian 


(2003,  p.  236),  who  proposed  that  “a  credible  change 
agent  who  works  with  practitioners  face-to-face  to 
encourage  enthusiastic  involvement”  is  the  most 
important  factor  in  successful  healthcare  change 
efforts.  Despite  the  rather  rigid  experimental  tradi¬ 
tions  in  medicine,  research  points  to  the  oftentimes 
chaotic  nature  of  change  within  complex  healthcare 
organizations.  A  deep  understanding  of  the  organi¬ 
zational  culture  of  the  context  in  which  any  change 
agent  works  is  essential. 

Berman  and  McLaughlin’s  concept  of  mutual 
adaptation  appears  to  apply  to  studies  of  change  in 
the  medical  profession  as  well.  Slater  et  al.  (2005) 
studied  a  community-based  mammography  project, 
and  their  findings  included  the  importance  of  an  atti¬ 
tude  that  “permits  the  change  agent  organization  to 
‘reinvent’  it  [the  innovation]  in  a  variety  of  settings 
and  circumstances”  (p.  465).  The  recognition  of 
competing  agendas  (researchers  vs.  change  agents 
vs .  community)  between  different  participants  in  the 
change  effort  underscores  the  need  to  allow  for  flex¬ 
ibility  of  implementation.  To  the  contrary,  a  study  of 
the  dissemination  of  a  sun  safety  program  to  zoolog¬ 
ical  parks  in  the  United  States  found  that  tailoring 
the  way  in  which  a  sun  safety  program  was  commu¬ 
nicated  to  zoos  had  little  effect  on  the  level  of  imple¬ 
mentation  (Lewis  et  al.,  2005).  This  runs  counter  to 
much  of  the  research  that  says  that  the  more  flexible 
an  innovation  is  the  more  likely  it  is  to  be  adopted 
(Berman  and  McLaughlin,  1975;  Rogers,  2003).  A 
possible  explanation  for  this  incongruence  is  that  this 
innovation  did  not  come  through  management  first 
but  was  sent  directly  to  disinterested  implementers. 
These  contradictory  findings  give  rise  to  the  notion 
that  source  of  the  innovation  may  be  equally  as 
important  as  its  flexibility. 

The  authors  in  the  zoo  study  also  note  that  they 
did  not  facilitate  communication  between  partici¬ 
pating  zoos  in  an  effort  to  avoid  contaminating  the 
results  of  their  study.  They  concede,  however,  that 
allowing  for  such  communication  would  likely  have 
improved  adoption  rates,  and  Rogers’  (2003)  notion 
of  observability  supports  this  conclusion.  Allowing 
for  communication  between  all  participants  in  the 
change  process  appears  to  be  a  characteristic  of 
successful  medical  change  efforts.  Slater  et  al. 
(2005)  found  that  maximizing  contact  between  col¬ 
laborating  agencies  over  the  course  of  their  6-year 
project  strengthened  the  change  effort.  In  complex 
organizations  such  as  these,  where  management, 
change  agents,  and  adopters  may  not  be  regular 
collaborators,  the  change  agent  is  wise  to  pay  par¬ 
ticular  attention  to  facilitating  communication 
among  all  participants. 
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CONCLUSION 

This  discussion  of  change  agentry  has  spanned  contexts 
from  schools  to  hospitals.  We  have  described  the  current 
state  of  the  theories  surrounding  change  agentry  with  a 
particular  emphasis  on  certain  themes,  including  the 
importance  of  connecting  organizations  to  their  environ¬ 
ments,  the  centrality  of  stakeholders  in  organizational 
change,  the  primacy  of  vision  and  culture,  the  necessity 
of  leadership  approval  and  buy-in,  the  usefulness  of 
professional  communities,  and  the  recognition  of  indi¬ 
vidual  differences  among  adopters.  We  have  also  looked 
at  non-school  contexts  and  found  that  culture  is  equally 
as  important  in  organizations  as  it  is  in  schools  and  that 
change  agentry  is  a  bit  more  well  defined  and  delimited 
in  the  corporate  sector.  Change  agents  in  non-school 
contexts  have  been  shown  to  require  heightened  com¬ 
munication  skills  and  a  strong  ability  to  function  with 
loosely  coupled  management  and  employees.  And,  like 
the  calls  for  school  change  that  often  result  from  sharing 
data  related  to  school  outcomes  (Peck  and  Carr,  1997), 
corporate  change  agents  must  be  able  to  show  current 
failings  in  a  way  that  generates  positive  momentum  for 
change.  The  findings  of  most  research  seem  to  indicate 
that  change  agents  are  not  always  welcomed,  supported, 
or  given  nearly  sufficient  time  or  human  resources.  In 
fact,  most  change  agents  in  all  contexts  struggle  might¬ 
ily  to  effect  and  sustain  substantive  systemic  organiza¬ 
tional  change.  Indeed,  it  is  within  this  struggle  that  those 
who  tilt  at  windmills,  as  many  instructional  technolo¬ 
gists  are  want  to  do,  find  their  personal  meaning.  It  is 
our  hope  that  the  research  presented  here  will  lighten 
your  lances  and  sharpen  your  aim. 
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ABSTRACT 

Design  languages  and  notation  systems  hold  great 
practical  and  theoretical  significance  for  instructional 
design.  Instructional  designers  use  multiple  design  lan¬ 
guages  in  the  creation  of  designs.  Notation  systems 
make  design  languages  visible  and  document  those 
solutions.  Design  languages  provide  the  building 
blocks  of  an  evolving  design.  Design  languages  are 
used  by  individual  designers;  shared  design  languages 
and  notation  systems  are  necessary  for  multi-partici¬ 
pant  design  teams  to  function,  even  at  a  basic  level. 
Design  languages  for  instructional  design  allow  us  to 
view  instructional  design,  instructional  theory,  instruc¬ 
tional  design  theory,  and  day-to-day  practice  in  a  new 
way  that  enhances  our  understanding  of  all  of  them. 


KEYWORDS 

Design  language:  A  set  of  abstractions  used  to  give 
structure,  properties,  and  texture  to  solutions  of 
design  problems;  designs  are  expressed  in  terms  of 
design  languages. 

Design  layer:  One  aspect  of  a  decomposed  design 
problem  that  can  be  approached  using  one  or  more 
design  languages;  a  subdomain  of  the  larger  prob¬ 
lem  to  which  one  or  more  design  languages  pertain. 


INTRODUCTION 

Design  languages  and  notation  systems  hold  great 
practical  and  theoretical  significance  for  instructional 
design.  In  this  chapter,  we  examine  the  evidence  of  the 
value  of  design  languages,  arguing  that  designers  in 
many  fields  already  realize  significant  benefits  from 
conscious  study  and  application  of  design  languages. 
These  benefits  include: 

•  Improved  design  team  communications 

•  Improved  designer-producer  communications 

•  Improved  designer-client  communications 


•  Promotion  of  design  innovation 

•  More  direction  from  theory  and  more  appli¬ 
cable  theory 

•  More  nuanced  theory  integration  with 
designs 

•  Improved  design  sharing  and  comparison  of 
designs 

•  Improved  designer  education 

•  Design  and  production  automation 


WHAT  ARE  DESIGN  LANGUAGES? 

As  instructional  designers,  we  all  use  multiple  design 
languages.  The  very  words  that  we  use,  such  as  dis¬ 
cussion  ,  portfolio ,  wd  formative  evaluation ,  code  spe¬ 
cial  or  professional  concepts  that  help  us  explore  an 
instructional  problem  space  and  identify,  refine,  and 
plan  a  solution  (Goodwin,  1994).  A  design  language 
is  a  set  of  abstractions  used  to  give  structure,  proper¬ 
ties,  and  texture  to  solutions  of  design  problems. 
Design  languages  provide  building  blocks  for  designs. 
Design  languages  provide  categories  for  thinking 
about  design  problems.  Design  languages  provide  an 
important  link  between  technological  theory  and 
design  practice.  Design  language  terms  signify  objects 
to  be  acted  upon,  actors,  actions,  concepts,  types  of 
relation,  composite  objects,  qualities,  and  properties. 

One  reviewer,  a  design  theorist,  commented  that 
“whenever  designers  talk  to  members  of  their  team 
they  are  using  [the  terms  of]  a  design  language”  (pri¬ 
vate  communication).  Consider  the  following  scenario 
illustrating  a  typical  design  situation:  As  an  instruc¬ 
tional  designer,  you  are  asked  to  support  a  communi¬ 
cation  specialist  in  the  design  of  a  blended  learning 
course  in  effective  communication  skills  for  a  com¬ 
pany’s  workforce.  At  a  meeting,  you  and  the  commu¬ 
nication  specialist  are  discussing  lectures ,  discussions , 
case  studies ,  exercises,  online  discussion  forums,  and 
other  possible  activities.  As  the  discussion  proceeds, 
you  realize  that  the  communication  specialist  does  not 
recognize  some  of  these  terms.  Further,  you  realize 
that  the  definitions  of  the  terms  that  you  do  both 
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recognize  differ,  sometimes  in  significant  ways.  You 
begin  to  find  it  difficult  to  discuss  design  details 
because  you  are  not  yet  working  from  a  common 
framework  of  meaning,  and  you  do  not  have  shared 
languages  for  communicating  meaning. 

The  designer  and  the  communications  specialist 
need  a  shared  set  of  terms  for  shaping  their  design  that 
will  be  understood  in  roughly  the  same  way  by  both. 
They  will  build  one,  drawing  on  personal  and  public 
design  languages  from  their  own  training  and  experi¬ 
ence,  and  new  terms  will  be  invented  within  the  scope 
of  the  project.  A  shared  local  language  will  evolve 
through  meaning  negotiation  (Winograd  and  Flores, 
1987).  Local  notation  systems  that  include  drawings, 
symbols,  and  words  will  also  evolve  to  publicly  rep¬ 
resent  the  design.  This  notational  expression  will  be 
directed  toward  a  specialized  audience  whose  mem¬ 
bers  know  the  language  and  its  notation  well  enough 
to  understand  the  implications  of  the  design  for  man¬ 
ufacture.  The  design  documentation  will  therefore  be 
rich  and  dense  and  difficult  for  non-team  members  to 
interpret. 

Schon  (1987)  described  designing  as  a  language- 
centered  social  phenomenon.  Design  languages  are 
common  to  all  fields  of  design  (Gibbons  and  Brewer, 
2005;  Rheinfrank  and  Evenson,  1996;  Waters  and  Gib¬ 
bons,  2004;  Winograd  and  Flores,  1987).  We  can  dis¬ 
cern  features  common  to  design  languages  in  many 
fields. 

Design  Layers  and  Design  Languages 

Numerous  design  languages  are  required  in  the  cre¬ 
ation  of  a  single  design,  according  to  Schon  (1987). 
He  related  design  languages  to  domains  of  the  design 
problem,  and  different  problem  domains  represent  sub¬ 
problems  to  be  solved  during  the  completion  of  one 
design  (Schon,  1987,  p.  58): 

Elements  of  the  language  of  designing  can  be  grouped 
into  clusters,  of  which  I  have  identified  twelve....  These 
design  domains  contain  the  names  of  elements,  features, 
relations,  and  actions  and  of  norms  used  to  evaluate  prob¬ 
lems,  consequences,  and  implications. 

Schon’s  domains  correspond  roughly  with  the  design 
layers  described  by  Brand  (1994)  with  respect  to  build¬ 
ing  designs.  Brand  identified  six  layers  of  a  building’s 
design:  site,  structure,  skin,  services,  spaces,  and  stuff 
(furniture).  He  proposed  that  describing  a  design  in 
layers  allows  the  designer  to  change  the  design  within 
one  layer  with  minimal  disruption  to  other  layers.  This 
results  in  design  modularity,  allowing  individual  layers 
to  age  independently.  Separate  layers  can  be  designed 


to  flow  past  each  other  without  disruption.  Baldwin  and 
Clark  (2000)  claimed  that  design  modularization  of  this 
type  governs  the  economics  of  the  modem  computer 
industry.  This  principle  seems  to  apply  to  software 
design  as  well  (Bass  et  al.,  2003;  Czamecki  et  al.,  2006; 
Rosenberg,  2007).  Gray  (2006)  described  how  this  prin¬ 
ciple  supplies  a  major  competitive  advantage  for  the 
merchandiser  Amazon.  Layers  are  clearly  defined  in 
some  Web  design  languages  such  as  W2000  (Baresi  et 
al.,  2001)  and  IDM  (Bolchini  and  Paolini,  2006).  These 
examples  suggest  that  design  languages  and  their  lay¬ 
ered  relationship  frame  a  coherent  theory  of  designing. 

Design  Languages  and  Natural  Languages 

The  term  design  language  raises  the  question  of  the 
relationship  between  design  languages  and  natural  lan¬ 
guages.  To  understand  this  relationship,  we  must  con¬ 
sider  the  nature  of  natural  languages.  Contrary  to  our 
usual  conception  of  languages  as  static  lexicons  and 
grammars,  McWhorter  (2003,  p.  12)  described  how 
“everything  about  a  language  is  eternally  and  inher¬ 
ently  changeable.”  Words  establish  legitimacy.  The 
word  language  itself  connotes  something  bounded  and 
settled.  McWhorter  said  that  instead  "there  is  not  even 
really  such  a  thing  as  ‘a  language’  at  all.  ...It’s  the 
nature  of  language  change  that  makes  the  concept  of 
‘a  language'  logically  impossible”  and  “most  ‘lan¬ 
guages’  are  actually  bundles  of  variations  on  a  general 
theme,  dialects”  (McWhorter  ,  2003,  pp.  52-53). 

McWhorter’s  view  of  a  single  natural  language  as 
a  multiplicity  of  dialects  serves  to  situate  the  concept 
of  design  languages  within  natural  languages.  Design 
language  expressions  are  made  in  a  specialized  dialect 
of  a  natural  language.  Design  language  expressions 
blend  with  natural  language  expressions  in  everyday 
use,  being  indistinguishable  to  a  non-native  listener. 
What  makes  a  design  language  distinct  from  natural 
language  is  its  unique  semantics:  one  not  shared  by 
the  general  population  of  the  language’s  users.  To  a 
mathematician  speaking  with  another  mathematician, 
for  example,  the  term  code  might  be  most  readily 
understood  as  the  key  to  an  encryption,  but  to  a  com¬ 
puter  programmer  speaking  to  another  computer  pro¬ 
grammer  the  term  code  will  more  likely  refer  to  a  work 
product.  Gibbons  and  Rogers  (2007)  characterized  the 
differences  between  natural  languages  and  design  lan¬ 
guages  (see  Table  47.1). 

Design  languages  are  a  special  case  of  natural  lan¬ 
guages  in  which  a  subpopulation  of  language  users 
share  terms,  expression  syntax,  and  semantics.  Design 
language  expressions  mix  specialized  language  ele¬ 
ments  with  natural  language  elements,  so  design  lan¬ 
guage  users  might  appear  to  be  speaking  in  code. 
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TABLE  47.1 

Natural  Languages  and  Design  Languages  Compared  in  Terms  of  Primitives,  Syntax,  and  Semantics 


Natural  Language 

Design  Language 

Primitive  terms 

Centered  in  everyday  things  and  events;  abstractions  of 
experience;  common  to  a  wide  range  of  users 

Centered  in  specialized  tools,  processes,  technologies,  theories, 
or  best  practices 

Syntax 

Based  on  words  as  a  medium  of  expression  in  which 
linear  or  positional  order  is  critical 

Dependent  on  the  medium  of  problem  solving  and  solution; 
sometimes  spatial  or  view  oriented 

Semantics 

Derived  from  the  world  as  it  is  experienced  and  things 
that  can  be,  or  are  desired  to  be,  communicated 

Derived  from  the  problem  domain  and  the  context  of  problems 
in  the  domain 

Source:  Adapted  from  Gibbons,  A.S.  and  Rogers,  P.C.,  in  Instructional-Design  Theories  and  Models:  A  New  Paradigm  of  Instructional  Theory , 
Vol.  Ill,  Reigeluth,  C.,  Ed.,  Lawrence  Erlbaum  Associates,  Mahwah,  NJ,  2007. 


which  is  in  a  sense  what  they  are  doing.  It  should  be 
emphasized  that  the  shared  conceptual  sets  of  the  users 
are  what  allow  the  design  language  to  make  sense 
among  users.  These  shared  conceptual  sets  may  find 
expression  in  diagrams,  drawings,  or  other  representa¬ 
tional  forms,  so  the  syntax  of  design  language  expres¬ 
sions  may  include  spatial  and  metaphorical  dimensions 
also  peculiar  to  the  users  (such  as  blueprint  or  sche¬ 
matic  conventions). 

Design  languages  exist  along  a  continuum  of  for¬ 
mality.  It  is  difficult  to  resist  the  temptation  to  treat 
design  languages  as  a  logical  formalism,  but  design 
languages  are  at  the  same  time  a  tool  for  and  a  byproduct 
of  designing.  They  may  be  formalized  for  special  pur¬ 
poses  in  the  way  in  which  computer  languages  are,  and 
those  formalisms  may  result  in  more  precise  and  sophis¬ 
ticated  designs,  but  overemphasis  on  formalism  can 
replace  the  metaphoric  uses  of  language  and  injure  cre¬ 
ativity.  By  far  the  most  common  use  of  design  languages 
falls  short  of  formalistic  communication;  instead,  the 
more  common  use  of  design  languages  is  joint  problem 
solving  and  the  negotiation  of  ever  more  precise  mean¬ 
ing  among  designers  and  design  team  members. 


INSTRUCTIONAL  DESIGN  LANGUAGES 

Design  languages  relate  to  instructional  design  archi¬ 
tecture:  the  internal  structure  of  an  instructional  artifact 
and  its  external  structural  relationships  with  its  context. 
In  this  section  we  describe  design  languages  for 
instructional  design. 

Instructional  Design  Layers 

Gibbons  (2003)  applied  Brand’s  idea  of  layered 
designs  to  the  architecture  of  instructional  designs, 
describing  instructional  designs  in  terms  of  seven  main 
layers,  each  with  potentially  many  sublayers.  Layer 


identities  are  based  on  a  functional  rather  than  process- 
oriented  decomposition  of  the  design  problem.  The 
main  layers  named  are  content,  strategy,  controls,  mes¬ 
sage,  representation,  media-logic,  and  data  manage¬ 
ment  (Gibbons  and  Rogers,  2006).  Within  each  layer, 
multiple  instruction  design  languages  now  exist, 
implicit  in  the  practices  of  instructional  designers. 
Each  language  within  each  layer  represents  an 
approach  to  the  design  of  a  particular  function.  The 
multiplicity  of  usable  design  languages  within  a  single 
layer  shows  how  rapidly  changing  technologies,  meth¬ 
ods,  processes,  theories,  tools,  and  styles  are  absorbed 
into  languages. 

Example:  Design  Languages  of  the  Content  Layer 

The  design  languages  pertaining  to  the  content  layer 
of  instructional  designs  have  evolved  and  multiplied 
over  the  past  50  years.  These  languages  identify  pos¬ 
sible  partitionings  of  subject-matter  elements.  When 
behaviorist  influence  was  strong,  designers  thought  of 
subject  matter  in  terms  of  operants  and  operant  chains 
(Gagne,  1965).  Over  time,  the  prevailing  view  turned 
in  the  direction  of  information  processing  theories  of 
learning,  and  new  categories  of  learnable  subject  mat¬ 
ter  appeared  (Gagne,  1985).  New  design  languages  or 
design  language  migrations  occur  frequently  and  can 
be  stimulated  by  changes  in  theory,  technique,  tools, 
or  conceptual  systems.  Viewing  Gagne’s  learning  cat¬ 
egories  as  the  terms  of  a  design  language  for  content 
description  suggests  we  also  consider  the  taxonomies 
of  Bloom  (1956),  Miller  (1971),  Merrill  (1994),  and 
many  others  as  content  design  languages.  It  also  gives 
us  a  new  perspective  for  considering  the  great  variety 
of  predesign  analysis  methodologies  described  by 
Jonassen  and  colleagues  (1999),  Gibbons  (1977),  and 
Gibbons  and  colleagues  (2000).  These  attempts  to 
understand  the  nature  of  analysis  methodologies  cata¬ 
log  languages  for  content  description. 
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Figure  47.1  The  3D  model  of  design  documentation.  (From  Boot,  E.W.,  Building  Blocks  for  Developing  Instructional  Software, 
Ph.D.  dissertation.  Open  Universiteit  of  the  Netherlands,  Heerlen,  2005.  With  permission.) 


New  theoretical  developments  continue  to  spawn 
new  content  layer  languages.  Anderson’s  instructional 
designs  (Anderson,  1993;  Anderson  et  al.,  1995) 
describe  leamable  subject  matter  in  terms  of  productions 
(if-then  rules)  and  working  memory  (semantic)  ele¬ 
ments.  Anderson  and  his  associates  have  applied  this 
content  design  language  to  automated  tutors  in  multiple 
subject-matter  areas  (Anderson  et  al.,  1986,  1989; 
Koedinger  and  Anderson,  1998).  This  line  of  research 
continues  (Koedinger  and  Corbett,  2006).  Collins  and 
colleagues  (1989),  in  their  description  of  the  instruc¬ 
tional  theory  of  cognitive  apprenticeship,  defined  four 
areas  of  design  guidance,  one  of  which  they  titled  con¬ 
tent ,  referring  to  four  types  of  knowledge  designers  may 
use  to  perform  subject-matter  analyses.  Situated  learn¬ 
ing  theory,  as  described  by  Lave  and  Wenger  (1991)  and 
Wenger  (1998),  examines  participation  within  commu¬ 
nities  of  practice.  This  theory  implicitly  defines  the  prac¬ 
tices  of  the  community  as  the  content  structures. 

Practical  Application:  The  3D  Model 
of  Design  Documentation 

The  Developing  Design  Documents  (3D)  model  of 
design  documentation  (Boot  et  al.,  2005)  applies 
design  layering  and  design  language  principles  to  the 
problem  of  design  documentation.  Design  documents 
transfer  information  from  instructional  designers  to 
producers  to  describe  artifacts  to  be  produced.  Design 
documents  may  be  difficult  for  producers  to  interpret 
for  three  reasons:  (1)  the  artifact’s  instructional  and 
technical  description  does  not  readily  translate  into  the 
terms  of  the  producer;  (2)  different  levels  of  design 


detail  may  be  mixed  together  from  the  producer’s  point 
of  view;  and  (3)  design  expressions  may  be  used  in  an 
inconsistent  manner.  The  3D  model  was  introduced  to 
guide  designers  in  creating  design  documents  that  are 
better  stratified,  elaborated,  and  formalized.  The  model 
is  illustrated  in  Figure  47.1. 

•  Dimension  1.  Stratification— Stratification 
helps  the  producer  determine  the  relation¬ 
ships  between  the  functionally  different 
instructional  and  technical  structures  while 
at  the  same  time  providing  an  orderly  way 
of  representing  the  integration  of  those  struc¬ 
tures  within  the  complete  design.  The  3D 
model  defines  the  seven  design  layers  pro¬ 
posed  by  Gibbons  (2003)  for  the  stratifica¬ 
tion  of  the  design  document. 

•  Dimension  2.  Elaboration— Fowler  (2003) 
described  three  possible  levels  of  elaboration 
in  an  instructional  software  design:  (1)  a  con¬ 
ceptual  perspective,  which  describes  the 
structure  of  the  major  elements  of  the  design 
with  little  technical  detail;  (2)  a  specification 
perspective,  which  provides  sufficient 
detailed  information  for  a  skilled  and  experi¬ 
enced  producer;  and  (3)  an  implementation 
perspective,  which  describes  the  design  with 
a  high  degree  of  technical  detail.  The  level  of 
elaboration  informs  the  level  of  detail  in  the 
design  languages  that  must  be  used. 

•  Dimension  3 .  Formalization — Designers  may 
determine  the  formalization  of  their  design, 
making  their  choice  of  informal  or  formal 
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design  languages  explicit.  Using  the  3D 
model,  designers  can  determine  the  level  of 
documentation  and  the  design  languages  used 
to  communicate  with  different  specialist 
groups  taking  part  in  design. 

Instructional  Design  Languages 
and  Theories  of  Instruction 

Gibbons  and  Rogers  (2007)  described  a  relationship 
between  design  languages  and  instructional  theories: 

We  propose  that  what  an  instructional  theorist  expresses 
in  an  instructional  theory  is  a  set  of  specialized,  mutually- 
consistent  design  languages... that  are  distributed  across 
multiple  design  layers.  ...[Layers]  can  be  used  by  the 
same  observer  to  analyze  and  compare  different  instruc¬ 
tional  theories.  To  the  extent  that  different  observers  can 
agree  upon  a  common  definition  of  layers,  they  can  jointly 
and  publicly  carry  out  such  analyses  and  comparisons. 

This  relationship  between  design  languages  and  theo¬ 
ries— that  one  source  of  design  languages  is  the  terms 
introduced  or  given  special  meaning  by  theorists  — 
answers  the  questions  “Is  there  a  best  design  lan¬ 
guage?”  and  “How  can  you  tell  whether  one  design 
language  is  better  than  another?”  The  answer  to  both 
questions  is  that  it  is  not  appropriate  to  declare  a  single 
language  superior.  A  later  section  of  this  chapter 
describes  how  multiple  instructional  design  languages 
(strategy  languages,  representation  languages,  control 
system  languages,  etc.)  are  required  just  to  make  a 
single  instructional  design.  These  languages  interact 
in  ways  that  bring  out  the  strengths  of  language  com¬ 
binations.  To  say  that  one  language  is  by  itself  superior 
is  evidence  of  a  misunderstanding  of  the  composition 
of  designs.  Languages  interacting  with  other  languages 
can  be  judged  on  the  basis  of  comparative  effectiveness 
in  use.  A  given  combination  of  languages  may  be 
superior  in  a  given  context;  the  same  combination  may 
be  inadequate  to  satisfy  the  needs  of  a  different  design 
problem  within  its  context. 

Design  Languages  and  Innovation 

Polanyi  (1958,  p.  87)  described  the  role  of  language 
in  innovative  thought  where  “symbolic  operations  ... 
outrun  our  understanding  and  thus  anticipate  novel 
modes  of  thought.”  He  suggested  that  we  may  use 
linguistic  rules  and  linguistic  symbols  to  create  com¬ 
binations  that  seem  not  to  make  sense  at  first  but  that 
on  closer  examination  define  a  new  path  of  innovative 
thought,  either  through:  (1)  a  fumbling  to  be  corrected 
later  by  our  tacit  understanding,  or  (2)  a  pioneering  to 
be  followed  up  later  by  our  tacit  understanding  (Pola¬ 
nyi,  1958,  p.  93): 


We  should  say  we  are  referring  ...  to  a  state  of  mental 
uneasiness  due  to  the  feeling  that  out  tacit  thoughts  do 
not  agree  with  our  symbolic  operations,  so  that  we  have 
to  decide  on  which  of  the  two  we  should  rely  and  which 
we  should  correct  in  the  light  of  the  other. 

Designers  who  consider  design  languages  are  accept¬ 
ing  to  a  degree  what  Polanyi  would  term  a  “formalism 
[through  language]  of  thought”  (p.  94).  Just  as  a  child 
uses  natural  language,  a  design  language  user  can 
form  nonsense  expressions  using  design  language 
terms.  This  is  a  risk,  he  says,  that  comes  with  adopt¬ 
ing  any  formal  system  for  giving  public  form  to  what 
would  otherwise  remain  personal  or  tacit  knowledge, 
but  he  said  there  may  also  be  benefits  (Polanyi,  1958, 
p.  94): 

Remember  how  various  new  kinds  of  numbers — irra¬ 
tional,  negative,  imaginary,  transfinite  —  were  produced 
as  a  result  of  extending  familiar  mathematical  opera¬ 
tions  into  unexplored  regions,  and  how  these  numbers 
after  having  been  repudiated  as  meaningless,  were 
eventually  accepted  as  denoting  important  new  mathe¬ 
matical  conceptions.  Such  spectacular  gains,  achieved 
by  the  speculative  use  of  mathematical  notations  for 
purposes  not  originally  entertained,  remind  us  that  the 
major  fruitfulness  of  a  formalization  may  be  revealed 
in  its  entirely  uncovenanted  functions,  precisely  at 
points  where  the  peril  seems  greatest  of  drifting  into 
absurdity. 

The  principle,  then,  of  using  design  languages  as  a 
type  of  formalism  for  thinking  about  instructional 
designs  might  be  stated  (Polanyi,  1958,  p.  95): 

Just  as  owing  to  the  ultimately  tacit  character  of  all  our 
knowledge,  we  remain  ever  unable  to  say  all  we  know, 
so  also,  in  view  of  the  tacit  character  of  meaning,  we  can 
never  quite  know  what  is  implied  in  what  we  say. 

In  this  spirit  Botturi  and  colleagues  (2006)  pro¬ 
posed  a  two-dimensional  classification  of  design  lan¬ 
guages  according  to  their  purposes  for  communication 
and  creativity  (see  also  Stubbs,  2006).  These  dimen¬ 
sions  can  be  used  to  compare  design  languages  in 
terms  of  the  intentions  of  their  creators  to  support 
particular  aspects  of  the  designer’s  activity.  Figure  47.2 
shows  this  kind  of  comparison  among  design  lan¬ 
guages  for  learning  object  design. 


THE  RANGE  OF  INSTRUCTIONAL 
DESIGN  LANGUAGES 

Design  languages  vary  across  a  range  of  characteris¬ 
tics,  according  to  Gibbons  and  Brewer  (2005,  p.  1 13): 
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these  languages  are  defined  by:  (1)  a  productivity- 
related  instructional  design  concept  (learning  objects), 

(2)  a  programming  paradigm  (object-orientation),  and 

(3)  a  programming  tool  concept  (UML-like).  Within 
this  design  space,  these  design  language  tools  will  grow 
over  time,  informing  each  other,  migrating  to  new 
forms  like  natural  languages,  and  producing  dialects 
through  processes  described  by  McWhorter  (2003). 


WHY  STUDY  INSTRUCTIONAL 
DESIGN  LANGUAGES? 


Figure  47.2  An  example  of  the  use  of  the  communication  and 
creativity  aspects  of  design  languages  for  making  comparisons 
among  languages.  (From  Botturi,  L.  et  al.,  A  Classification 
Framework  for  Educational  Modeling  Languages  in  Instruc¬ 
tional  Design,  paper  presented  at  the  International  Conference 
on  Advanced  Learning  Technologies,  July  5-7,  Kerkrade,  the 
Netherlands,  2006.  With  permission.) 

Some  of  our  design  languages,  such  as  computer  program¬ 
ming  languages,  are  formal,  and  when  we  use  them  it  is 
a  conscious,  deliberate  design  activity.  Many  of  our  design 
languages,  however,  are  so  subtle  that  we  do  not  recognize 
them  and  realize  that  we  use  them  to  structure  designs. 

Gibbons  and  Brewer  described  several  dimensions  of 
design  language  variation: 

•  Complexity-simplicity 

•  Precision-nonprecision 

•  Formality-informality 

•  Personalization-sharedness 

•  Implicitness-explicitness 

•  Standardization-nonstandardization 

•  Computability-noncomputability 

All  of  these  characteristics  can  be  seen  in  instructional 
design  languages.  The  languages  related  to  high-tech 
media  design  and  production  tend  to  be  complex,  pre¬ 
cise,  formal,  shared,  explicit,  standard,  and  comput¬ 
able.  Languages  that  describe  instructional  strategy 
structures  tend  to  be  relatively  simple,  nonprecise, 
informal,  often  personalized  or  stylized,  implicit,  non¬ 
standard,  and  noncomputable. 

EXAMPLES  OF  SPECIALIZED 
INSTRUCTIONAL  DESIGN  LANGUAGES 

Several  research  groups  have  pursued  specialized 
design  languages  for  use  in  designing  learning  objects 
(Botturi,  2006;  Caeiro-Rodriguez  et  al.,  2006;  Derntl, 
2007;  Koper,  2002;  Koper  and  Tattersall,  2005; 
Paquette,  2005).  The  boundaries  for  the  application  of 


We  identify  four  main  reasons  for  the  serious  study  of 
instructional  design  languages:  (1)  They  encourage 
disciplined  design  practice,  (2)  they  give  organization 
to  the  growth  of  design  fields,  (3)  their  study  gives 
historical  context  to  evolving  design  fields,  and  (4) 
they  connect  practices  of  a  design  field  to  theoretical 
concepts. 

Design  Languages  Encourage 
Disciplined  Design  Practice 

In  many  design  fields,  making  design  languages  explicit 
has  resulted  in  rapid  advances  in  design  productivity. 
Computer  chip  designs  were  originally  recorded  by 
hand  in  large  and  very  detailed  drawings.  Over  time, 
highly  repetitive  and  detailed  local  design  decisions 
were  identified  and  ways  were  invented  to  express 
design  problems  in  multiple,  strictly  bounded  design 
languages.  These  languages  could  be  translated  into 
computer  programming  languages  for  solving.  The  end 
result  has  been  a  computer  chip  design  process  in  which 
most  low-level  design  decisions  are  made  automatically. 
Chip  design  is  now  faster,  and  designs  have  increased 
exponentially  in  complexity  and  sophistication.  A  return 
to  handmade  and  hand-drawn  designs  is  unthinkable 
(Brayton  and  Darringer,  2003;  Del  Man  and  Rabaey, 
2003;  Kuh  and  Hsu,  2003;  Sakallah  et  al.,  2003). 

Without  the  creation  of  design  languages  for 
problem  expression,  the  computer  would  be  irrele¬ 
vant  to  designing.  The  earliest  use  of  computers  in 
computer-aided  design  was  merely  to  record  design 
decisions  and  eliminate  drafting  tables.  Conscious 
attention  to  formal  use  of  design  languages  has  accel¬ 
erated  progress  in  many  other  design  fields  as  well, 
including  aviation,  software,  automotive,  and  archi¬ 
tectural  design,  among  others  (Kuehlmann,  2003; 
Newsome  et  al.,  1989;  Saabagh,  1996).  McDonald 
(McDonald,  2006;  McDonald  and  Gibbons,  in  press) 
described  how  design  practice  can  be  disciplined 
through  the  explicit  use  of  terms  that  describe  the 
design  practice  itself. 
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Study  of  Design  Languages  Gives  Direction 
to  the  Growth  of  Design  Fields 

Design  languages  help  organize  the  growing  knowl¬ 
edge  base  of  a  technological  field.  Design  usually 
involves  the  efforts  of  teams  of  specialists.  Planning 
and  coordination  of  their  practice  involve  the  use  of 
specialized  design  languages.  The  use  of  the  languages 
is  most  visible  in  the  jargon  of  design  team  members. 
Families  of  languages  needed  to  make  coordinated 
action  possible  include: 

•  Artifact  design  and  development  languages 

•  Languages  related  to  the  design  process 

•  Languages  related  to  development  processes 

•  Tool  languages  related  to  design  and  devel¬ 
opment  tools 

•  Production  specialty/specialist  languages 

•  Artifact  description  languages 

•  Structure  and  dimension  description  lan¬ 
guages 

•  Functional  description  languages 

•  Languages  that  describe  artifact  properties 
or  qualities 

•  Artifact  operation  languages 

•  Languages  describing  artifacts  and  their  con¬ 
texts  physically 

•  Languages  describing  modes  of  operation 
and  use 

•  Measurement  languages 

•  Instrument  design  languages 

•  Measurement  process  design  languages 

•  Measurement  concept  and  interpretation  lan¬ 
guages 

•  Status/state  description  languages 

•  Intervention  planning  languages 

•  Intervention  pattern  languages 

•  Intervention  process  languages 

Design  language  terms  support  communication  about 
specialized  objects,  dimensions,  properties,  qualities, 
and  processes  by  design  teams.  Most  instructional 
designers  are  not  aware  of  their  tacit  use  of  a  great 
variety  of  design  languages  in  written  and  spoken  com¬ 
munications,  nor  are  they  aware  of  the  important  func¬ 
tion  of  those  languages  in  making  useful  and  increas¬ 
ingly  subtle  design  distinctions  (Stubbs,  2006). 

Evolution  of  Design  Languages  Gives 
Historical  Context  to  Design  Fields 

The  evolution  of  the  design  languages  of  a  field  reflects 
the  changing  questions  of  the  field  and  their  scope.  By 
identifying  the  design  languages  of  the  past,  research¬ 


ers  and  students  can:  (1)  chart  trajectories  that  point 
to  unsolved  problems,  (2)  recognize  the  scope  and 
limitations  of  past  questions,  and  (3)  extrapolate  from 
past  questions  to  form  and  set  the  scope  of  new  ques¬ 
tions.  For  example,  successive  attempts  have  been 
made  to  codify  a  definition  for  the  instructional  tech¬ 
nology  field  and  define  its  theoretical  and  practical 
issues  (Dijkstraet  al.,  1997;  Januszewski,  2001;  Reige- 
luth,  1983,  1999;  Richey,  1986;  Seels  and  Richey, 
1994;  Snelbecker,  1985;  Tennyson  et  al.,  1997).  Such 
publications  give  a  snapshot  of  the  implicit  design 
languages  of  a  period  that  can  be  used  by  researchers 
and  students  to  understand  their  questions  in  a  histor¬ 
ical  context. 

Design  Languages  Connect  Practices  of 
a  Design  Field  to  Theoretical  Concepts 

The  study  of  design  languages  reveals  avenues  for  the 
grounding  of  instructional  design  practices  in  the  the¬ 
ories  of  related  disciplines. 

Social  Learning  Theory  and  Sociolinguistics 

Sociolinguistics  is  the  study  of  the  role  of  language  in 
social  interaction.  The  study  of  design  languages 
within  a  sociolinguistic  context  is  important  to 
researching  the  inward  conversations  of  designers  and 
the  outward  conversations  of  design  teams.  Sociolin¬ 
guistics  allows  the  design  theorist  to  consider  the 
implications  of  research  by  studying  discourse  analysis 
(Barton  and  Tusting,  2005;  Bazerman,  1999),  conver¬ 
sation  theory  (Pask,  1976),  conversation  analysis  (te 
Molder  and  Potter,  2005),  and  design  as  a  social  pro¬ 
cess  (Bucciarelli,  1994). 

Sociolinguists  Barton  and  Tusting  (2005)  proposed 
that  communities  of  practice  cannot  exist  without  a 
common  language  for  communicating  about  the  com¬ 
munity’s  shared  interest.  Wenger’s  (1998)  description 
of  communities  of  practice  anticipated  the  centrality 
of  language  to  the  function  of  communities  using  the 
concept  of  reification,  which  Wenger  (1998,  pp. 
58-59)  defined  as: 

The  process  of  giving  form  to  our  experience  by  produc¬ 
ing  objects  that  congeal  this  experience  into  ‘thingness’. 

In  so  doing,  we  create  points  of  focus  around  which  the 
negotiation  of  meaning  becomes  organized.  . .  .Any  com¬ 
munity  of  practice  produces  abstractions,  tools,  symbols, 
stories,  terms  and  concepts  that  reify  something  of  that 
practice  in  a  congealed  form. 

Barton  and  Tusting  (2005,  p.  26)  noted  that  “reification 
entails  not  only  the  negotiation  of  shared  understand¬ 
ings  but  also  enables  particular  forms  of  social  relations 
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to  be  shaped  in  the  process  of  participation.”  In  a  com¬ 
munity  of  designers,  the  formation  of  design  languages 
is  not  just  an  interesting  byproduct  but  an  essential 
process  that  makes  possible  the  existence  of  the  com¬ 
munity.  The  specialized  communications  of  the  com¬ 
munity  are  carried  out  in  the  terms  of  these  languages. 

Cognitive  Semantics  and  Linguistics 

Cognitive  semantics  (Jackendoff,  1993;  Talmy,  2001) 
is  the  study  of  how  humans  develop  and  express  mean¬ 
ing.  Design  languages  are  in  some  ways  similar  to 
other  verbal,  written,  and  symbolic  languages;  there¬ 
fore,  principles  of  cognitive  semantics,  linguistics,  and 
semiotics  may  be  used  to  study  these  specialized,  local, 
and  bounded  languages.  Semantically,  design  lan¬ 
guage  terms  take  their  meaning  and  interpretability 
from  their  context  of  use,  from  the  intentions  of  the 
designer,  and  from  the  negotiation  of  meaning  between 
designers  (Winograd  and  Flores,  1987).  This  principle 
extends  to  the  semantics  of  the  design  languages  used 
by  the  many  stakeholder  communities  that  surround 
and  support  the  use  of  a  technology.  Bazerman  (1999, 
pp.  336-337)  provided  an  account  of  the  stabilization 
of  the  new  technology  of  electrification  and  electrical 
lighting  in  the  time  of  Thomas  Edison: 

Technologists  are  tied  into  less  obvious  meaning  systems 
for  their  development,  appreciation,  production,  funding, 
operation,  maintenance,  social  control,  evaluation,  and 
distribution.  Moreover,  these  (and  possibly  other)  func¬ 
tions  are  likely  to  be  distributed  among  different  group¬ 
ings  in  the  society,  requiring  differential  distribution  of 
representations  to  various  social  components  that  may 
not  overlap.  Papers  must  be  filed  with  financial  backers, 
government  regulators,  technical  R&D  departments, 
sales  forces,  material  suppliers,  production  machinery 
producers,  and  shop  floor  designers. 

In  the  future,  descriptions  like  Bazerman’ s  applied  to 
the  stabilization  and  mass  application  of  instructional 
technologies  of  many  kinds  will  become  an  important 
factor  in  establishing  and  maintaining  them. 

Computer  Science 

Design  languages  can  also  be  related  to  the  special- 
purpose  languages  of  the  computer.  The  study  of  pro¬ 
gramming  languages  suggests  possible  connection 
points  between  computation  and  design  languages. 
Early  artificial  intelligence  research  included  experi¬ 
ments  in  intelligent  tutoring.  In  many  of  these  exper¬ 
iments,  the  goal  was  to  generate  some  portion  of  the 
instructional  experience  in  real  time,  taking  into 
account  a  number  of  factors  that  included  the  structure 


of  the  subject  matter  and  the  history  of  recent  actions 
of  the  learner.  These  experimental  systems  explored 
the  real-time  generation  of  many  elements  of  the 
instructional  experience,  including  generation  of  mes¬ 
sage  (Carbonell,  1970;  Drake  et  al.,  1998;  Stevens  and 
Collins,  1977),  generation  of  representations  (Hollan 
et  al.,  1984),  generation  of  strategy  (Buchannan  and 
Shortliffe,  1984;  Clancey,  1984),  and  generation  of 
control  systems  (Johnson  et  al.,  2000),  among  others. 

Design  languages  give  designers  a  tool  for  discuss¬ 
ing  with  greater  precision  which  instantaneous  design 
decisions  can  be  shared  with  the  computer;  for  exam¬ 
ple,  Exploring  Emergence  (Resnick  and  Silverman, 
2006)  places  side-by-side  fixed  traditional  instruc¬ 
tional  forms  and  the  computations  of  a  dynamic  graph¬ 
ical  model,  whose  unpredictable  variations  no  designer 
would  think  of  creating  and  prestoring  in  graphic  form. 

Design  languages  are  also  related  to  the  emerging 
study  of  ontologies  for  systematic  semantics  searches 
for  patterns  of  relationship  within  databases  (Niren- 
burg  and  Raskin,  2004;  Noy  and  McGuinness,  2006). 
Ontologies  will  be  pivotal  in  the  development  of  a 
Semantic  Web.  Ontology-based  searches  will  include 
finding  resources  with  standard  functional  character¬ 
istics,  such  as  those  that  contain  specific  modularized 
instructional  functions. 


THE  NATURAL  HISTORY  OF 
INSTRUCTIONAL  DESIGN  LANGUAGES 

Design  languages  come  into  being  in  different  ways. 
McWhorter  (2003,  pp.  11-12)  described  the  natural 
history  of  natural  languages  in  transformative  terms: 

Language  is  ...  analogous  to  cloud  formations.  We  look 
at  a  cloud  formation  with  full  awareness  of  its  inherently 
transitory  nature:  we  know  that  if  we  look  up  again  in  an 
hour,  the  formation  will  almost  certainly  be  different.... 

Most  design  languages  are  neither  formalized  nor  stan¬ 
dardized.  Many  of  them  enter  widespread  use  with 
only  the  most  vague  specification  and  definition  of 
terms.  Even  for  languages  whose  terms  are  clearly 
defined,  there  is  a  danger  that  the  terms  may  be  appro¬ 
priated  by  others  or  applied  with  modifications 
(McDonald,  2006).  Those  modifications  may  become 
a  de  facto  alternative  standard,  and  the  originator  of 
the  design  language  must  either  live  with  the  fuzziness 
or  issue  a  correction  reexpressing  the  original  terms 
and  definitions.  This  occurred  when  Barrows  restated 
his  definition  of  “problem-based  learning”  following 
many  years  of  definition  creep  (Barrows,  1998;  Bar- 
rows  and  Tamblyn,  1980).  Design  languages  are  in 
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constant  flux.  As  McWhorter  (2003,  p.  53)  said  of 
natural  languages,  “Dialects  is  all  there  is.” 

Primitive  terms  of  design  languages  tend  to  be 
invented,  and  invention  can  take  place  suddenly  with 
the  introduction  of  a  new  process,  tool,  or  theory.  Most 
design  language  terms  do  not  find  their  way  into  the 
lexicons  of  natural  languages.  Design  languages  may 
obey  some  of  the  rules  of  natural  languages  but  tend  to 
be  separate  from  natural  languages.  Design  languages 
are  often  promoted  by  the  originators  of  the  language, 
and  language  wars  may  occur.  Sponsors  may  include 
theoreticians,  special-interest  groups,  businesses,  or 
business  communities  with  connected  interests,  partic¬ 
ularly  in  software  tools  or  hardware  they  have  created. 

The  syntactical  rules  of  languages  emerge  slowly 
from  usage.  Syntax  gives  order  to  expressions  of  mean¬ 
ing.  Through  transformation,  words  and  phrases  can 
be  placed  in  different  surface  arrangements  that  pre¬ 
serve  meaning,  but  the  surface  form  of  the  new 
arrangement  is  still  linear.  Design  language  syntax,  on 
the  other  hand,  is  multidimensional.  The  primitive 
terms  in  a  design  language  may  be  expressed  as  words, 
but  expressions  in  a  design  must  describe  multiple 
relationships  for  every  design  element  in  adequate 
detail  to  support  creation  of  an  artifact.  Relationships 
may  be  spatial  (two-  or  three-dimensional),  temporal 
(four-dimensional),  or  more  complex  (conditional). 
Design  languages  often  require  more  complex  nota¬ 
tions  to  contain  the  amount  of  information  requisite 
for  a  design.  This  can  lead  to  three-dimensional  draw¬ 
ing  or  models,  four-dimensional  animations,  and  even 
more  complex  modeling  systems  that  involve  multiple 
representation  forms  (Stubbs,  2006).  Systems  such  as 
traditional  architectural  drawing  have  become  so  spe¬ 
cialized  that  language  notation  standards  are  created 
to  govern  the  representation  of  designs. 

Natural  language  semantics  is  derived  from  pat¬ 
terns  of  cause  and  effect  and  states  of  being  in  real- 
world  experience  or  by  metaphorical  processes  oper¬ 
ating  on  them  (Talmy,  2001).  The  semantic  of  a  design 
language  is  limited  to  a  problem  domain  within  which 
the  design  language  is  specified,  such  as  Schon’s  archi¬ 
tectural  problem-solving  domains  or  Brand’s  layers 
(Schon,  1987;  Brand,  1994).  Design  language  expres¬ 
sions  have  meanings  bounded  by  the  problem  worlds 
for  which  they  were  created. 

DESIGN  LANGUAGES 
AND  NOTATION  SYSTEMS 

Waters  and  Gibbons  (2004)  made  a  distinction 
between  a  design  language  and  public  design  repre¬ 
sentations  through  speech,  writing,  drawing,  and  ges¬ 


tures.  Private  design  languages  exist  that  do  not  have 
corresponding  symbolic  forms  and  rules  for  sharing. 
Bucciarelli  (1994,  p.  159)  described  this: 

Shared  vision  is  made  explicit  in  documents,  texts,  and 
artifacts — in  formal  assembly  and  detail  drawings,  oper¬ 
ation  and  service  manuals,  contractual  disclaimers,  pro¬ 
duction  schedules,  marketing  copy,  test  plans,  parts  lists, 
procurement  orders,  mock-ups,  and  prototypes.  But  in 
the  process  of  designing  . . .  each  participant  in  the  pro¬ 
cess  has  a  personal  collection  of  sketches,  flow  charts, 
cost  estimates,  spreadsheets,  models,  and  above  all  sto¬ 
ries —  stories  to  tell  about  their  particular  vision  of  the 
object.  The  shared  vision,  as  some  synthetic  representa¬ 
tion  of  the  artifact  as  a  whole,  is  not  in  the  documents 
or  written  plans.  To  the  extent  that  it  exists  as  a  whole, 
it  is  a  social  construction... . 

Design  languages  and  notation  systems  support  growth 
and  improvement  in  each  other.  Once  a  consistent 
notation  system  is  established,  it  can  become:  (1)  a 
tool  for  remembering  designs,  (2)  a  structured  prob¬ 
lem-solving  work  space  where  designs  can  take  form, 
and  (3)  a  laboratory  tool  for  sharpening  and  subdivid¬ 
ing  abstract  design  categories.  Through  a  continuing 
cycle  of  refinement,  both  design  language  and  notation 
system  grow  in  parallel,  and  more  sophisticated  design 
ideas  result. 


DESIGN  LANGUAGE  RESEARCH 

Gibbons  and  Brewer  (2005)  proposed  several  direc¬ 
tions  for  research  in  design  languages  and  their  poten¬ 
tial  uses: 

•  Identify,  document,  and  study  existing 
design  languages. 

•  Extract  principles  in  existing  languages  and 
learn  to  apply  them  deliberately. 

•  Obtain  better  language  grounding  through 
attachment  to  theory. 

•  Examine  the  generative  principles  of  lan¬ 
guage  that  lead  to  new  design  languages. 

•  Generate  better-defined  languages  and 
grammars. 

•  Create  tools  that  emphasize  the  designer’s 
languages  rather  than  the  computer’s. 

•  Use  design  languages  to  redefine  design  pro¬ 
cesses. 

This  list  can  be  supplemented  with  additional  ideas 
described  in  this  chapter: 

•  Explore  the  relationship  between  ontologies 
and  design  languages. 
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•  Explore  computable  instructional  design 
languages  and  their  role  in  adaptive  instruc¬ 
tional  experiences. 

•  Explore  social  dimensions  of  language  use 
during  designing. 

•  Explore  positive  economic  benefits  of  design 
language  use  for  design  teams. 

•  Explore  lessons  learned  in  other  design  com¬ 
munities  from  design  language  application. 

•  Explore  the  use  of  design  languages  in  edu¬ 
cating  new  designers. 

•  Explore  the  role  of  design  languages  in 
securing  new  technologies  and  promoting 
their  use  among  a  range  of  stakeholders. 

ANTICIPATED  BENEFITS 

Several  benefits  may  be  expected  from  the  above 
research: 

•  Improved  tools  for  the  support  of  all  stages 
of  instructional  design,  from  initial  concep¬ 
tion  to  documentation  of  the  completed 
design 

•  Clearer  relationships  between  design  deci¬ 
sions  and  structures,  features,  and  qualities 
of  designs 

•  Improved  techniques  for  facilitating  the  for¬ 
mation  of  instructional  design  teams  and  their 
rapid  movement  toward  productive  activity 

•  Improved  approaches  for  the  training  of  new 
instructional  designers 

•  A  broader  view  of  new  designers  with 
respect  to  the  spectrum  of  design  languages 
used  by  the  many  specialized  members  of 
their  design  teams 

•  New  understanding  of  the  nature  of  instruc¬ 
tional  designing  and  new  processes  that  tai¬ 
lor  the  design  process  to  the  design  problem 

CONCLUSION 

Design  languages  supply  primitives  for  private  design 
and  for  public  discourse  about  designs.  In  a  complex 
technological  world  where  designing  can  no  longer  be 
considered  a  single-person  activity,  increasing  empha¬ 
sis  should  be  placed  on  the  importance  of  design  lan¬ 
guages  as  a  vehicle  for  sharing  design  processes.  In 
addition  to  their  use  in  creating  designs,  design  lan¬ 
guages  supply  a  means  by  which  theories  used  in 
designing  can  be  examined,  understood,  compared, 
and  implemented.  In  many  design  fields,  attention  to 
design  languages  and  their  use  has  produced  enormous 


increases  in  productivity  and  creativity.  Design  lan¬ 
guages  are  not  an  invention  but  the  description  of  a 
new  way  of  seeing  what  designers  naturally  do  in  every 
field  of  design.  As  such,  it  is  a  conceptual  tool  and  not 
a  truth,  but  as  a  conceptual  tool  we  should  explore  the 
new  possibilities  design  languages  open  to  us  for 
improving  the  range,  precision,  and  sophistication  of 
our  designs.  Consideration  of  the  design  languages  of 
instructional  design  present  us  an  opportunity  to  view 
instructional  design,  instructional  theory,  instructional 
design  theory,  and  day-to-day  practice  in  a  new  way 
that  may  advance  our  understandings  of  all  of  these. 
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ABSTRACT 

Providing  a  brief  history  of  the  work  of  design  and 
development  (D&D)  teams  grounded  in  the  relevant 
literature,  this  chapter  then  offers  several  prominent 
examples  from  the  past  65  years.  These  examples 
allow  the  comparison  of  the  context  for  design  and 
development  focused  on  industry,  government,  and 
education.  Although  the  implications  for  different 
approaches  to  teamwork  for  design  and  development 
are  discussed,  we  highlight  models  for  developing 
teams  that  are  especially  useful  to  those  who  are  new 
to  the  field.  We  conclude  with  challenges  faced  by 
design  and  development  teams  specifically  in  regard 
to  educational  communications  and  technology. 
Design  and  development  teams  should:  (1)  organize  in 
ways  informed  by  the  research,  (2)  access  and  apply 
research  on  teaching  and  learning,  (3)  conduct  research 
for  ongoing  evaluation  of  technology  in  education,  and 
(4)  contribute  to  the  research  in  the  held  from  their 
own  innovations,  mindful  of  the  social  consequences 
of  design  and  development. 


KEYWORDS 

Analytical  science:  Within  this  perspective  researchers 
aim  to  develop,  test,  and  justify  theories  (D’Anjou, 
2004;  Hevner  et  al.,  2004;  Klabbers,  2003;  Simon, 
1996;  van  Aken,  2004);  see  design  science  and 
situated  science. 

Design  and  development  team:  The  group  of  people 
and  skill  sets  involved  in  the  research-based  devel¬ 
opment  of  artifacts  and  systems  in  education. 

Design  science:  Within  this  perspective  researchers 
aim  to  build  and  evaluate  artifacts  for  well- 
defined  contexts  of  use  (D’Anjou,  2004;  Hevner 
et  al.,  2004;  Klabbers,  2003;  Simon,  1996;  van 
Aken,  2004);  see  analytical  science  and  situated 
science. 

Design  in  the  large  (D1L):  Refers  to  the  goal  of  chang¬ 
ing  existing  situations  into  preferred  ones  (Klab¬ 
bers,  2003);  see  design  in  the  small  (DIS). 

Design  in  the  small  (DIS):  Refers  to  the  design  of  a 
specific  innovation  (Klabbers,  2003);  see  design  in 
the  large  (DIL). 


Knowledge,  skills,  and  abilities  (KSAs):  Used  in  the 
traditional  approach  for  specifying  (team)  compe¬ 
tencies;  recent  research  now  expands  the  skills 
beyond  those  specifically  attributed  to  the  team 
task  to  include  psychomotor  and  cognitive  compe¬ 
tencies  (Cannon-Bowers  et  al.,  1995). 

Shared  mental  models:  Crucial  in  the  work  of  a  team 
as  members  of  the  team  need  to  share  an  accurate 
model  of  the  problem  to  be  addressed  and  the  asso¬ 
ciated  task  to  assess  the  problem,  determine  their 
own  roles  and  responsibilities,  and  coordinate 
efforts  over  time  (Cannon-Bowers  et  al.,  1995). 

Situated  science:  A  perspective  in  which  the  scientific 
inquiry  and  attitude  are  carried  into  (instead  of 
applied  to)  the  field  of  the  project  and  of  practice, 
modifying  each  other  over  time  as  the  project 
unfolds  (Findeli,  2001;  Simon,  1996);  see  analyti¬ 
cal  science  and  design  science. 

Team  Evolution  and  Maturation  (TEAM)  model: 
Offers  stages  that  have  been  observed  in  team  per¬ 
formance  that  take  into  account  time,  experience, 
and  training  (Morgan  et  al.,  1998). 

Transactional  leader:  Exerts  influence  by  setting 
goals,  clarifying  desired  outcomes,  providing  feed¬ 
back,  and  exchanging  rewards  and  recognition  for 
accomplishments;  one  of  two  leadership  styles  that 
hold  the  potential  for  more  effective  team  perfor¬ 
mance,  moving  away  from  a  traditional  top-down 
directive  approach  (Bass,  1985;  Sosik  et  al.,  1997); 
see  transformational  leaders. 

Transformational  leader:  Exerts  influence  by  broad¬ 
ening  and  elevating  followers’  goals  and  providing 
them  with  the  confidence  to  go  beyond  minimally 
acceptable  expectations  specified  in  the  exchange; 
one  of  two  leadership  styles  that  hold  the  potential 
for  more  effective  team  performance,  moving  away 
from  a  traditional  top-down  directive  approach 
(Bass,  1985;  Sosik  et  al.,  1997);  see  transactional 
leaders. 


INTRODUCTION 

The  challenges  faced  by  design  and  development 
(D&D)  teams  in  regard  to  developing  information  and 
communications  technology  in  education  begin  with 
the  ability  to  conceptualize  needs,  resources,  and 
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goals.  At  the  very  end  of  the  D&D  cycle,  beyond  the 
completion  of  individual  tasks  and  products,  impact  is 
evident  in  the  social  consequences  that  unfold  even 
after  the  project  conclusion.  Technology  has  an  unde¬ 
niable  impact  on  all  aspects  of  society,  and  yet  the 
roles  and  responsibilities  of  designers  and  developers 
are  not  often  discussed  generally  nor  are  they  discussed 
specifically  with  regard  to  education. 

This  chapter  is  intended  to  offer  discussion  regard¬ 
ing  the  social  consequences  of  design  and  development 
(Bowers,  1988,  2000;  Ellul,  1964;  Postman,  1992; 
Stoll,  1999)  while  outlining  models  of  teamwork  with 
three  well-known  projects  alongside  some  of  the  most 
recent  research  in  organizational  theory  and  the  devel¬ 
opment  of  technologies  for  teaching  and  learning.  The 
uses  of  research  findings  and  research  methods  are 
described  in  ways  that  can  inform  the  development  of 
individual  projects  in  education,  as  designers  and 
developers:  (1)  aim  for  optimal  performance  of  the 
individual  and  the  team  informed  by  the  research  on 
organizational  theory,  (2)  access  and  apply  research  on 
teaching  and  learning  in  the  development  of  technolo¬ 
gies  for  education,  (3)  conduct  research  for  formative 
and  summative  evaluation  of  technology  in  education 
strengthened  and  aligned  with  a  larger  vision  for  edu¬ 
cation,  and  (4)  contribute  to  the  research  in  the  field 
from  their  own  individual  innovation  or  artifact  inform¬ 
ing  the  larger  efforts  toward  systemic  change  in  edu¬ 
cation. 


DEFINING  DESIGN 
AND  DEVELOPMENT 

Who  are  the  designers  and  developers,  and  what  is 
their  relationship  to  the  field  of  education?  Finding 
them  within  a  range  disciplines,  including  education, 
D’Anjou  (2004,  p.  213)  drew  on  the  earlier  writing  of 
Simon  (1996)  and  explained  that: 

The  science  of  design  becomes  the  broader  foundational 
paradigm  for  professional  education,  which  includes  any 
disciplines  engaged  in  the  intentional  transformation  of 
the  world,  from  bridges  to  political  programs  ...  they 
view  the  world  as  a  project,  whereas  science,  such  as 
physics,  biology,  sociology,  etc.,  views  it  as  an  object. 

The  distinction  between  the  analytical  sciences  (devel¬ 
oping,  testing,  and  justifying  theories)  and  the  design 
sciences  (building  and  evaluating  artifacts  for  well- 
defined  contexts  of  use)  has  been  made  by  Klabbers 
(2003b)  and  others  (Hevner  et  al.,  2004;  van  Aken, 
2004).  As  D&D  teams  work  within  the  design  sci¬ 
ences,  they  develop  individual  projects  that  are  then 
implemented  within  a  larger  system. 


Four  elements  are  critical  to  D&D  team  members’ 
ability  to  contribute  to  an  individual  project,  as  team 
members  need  to  share  an  understanding  of  the:  (1) 
information  structure  and  its  representation,  (2)  the 
instructional  design  and  underlying  beliefs  about 
learning,  (3)  interaction  possibilities  and  their  design, 
and  (4)  interface  mechanisms  of  their  visual  presenta¬ 
tions  (Hedberg  and  Sims,  2001).  This  definition  draws 
upon  the  foundations  of  instructional  design,  which 
over  time  has  placed  increasing  emphasis  on  the 
learner  and  on  developing  the  interactive  potential 
within  innovations  for  teaching  and  learning  (Dick  et 
al.,  2005;  Ertmer  and  Quinn,  2003;  Gagne  and  Briggs, 
1974;  Preece  et  al.,  2002;  Reigeluth,  1999;  Schank  and 
Jona,  1991).  Drawing  on  Simon  (1996)  and  the  dis¬ 
tinctions  between  science  as  applied  and  design  as 
involved,  Findeli  (2001)  cast  the  designer  as  a  stake¬ 
holder  who  participates  in  the  learning  dimension  of 
the  project  and  ends  up  transformed. 

As  stakeholders  within  systems,  designers  and 
developers  need  to  keep  in  mind  the  systemic  perspec¬ 
tive.  In  the  case  of  education,  this  requires  considering 
the  needs  and  context  in  education  at  the  beginning  of 
a  project  as  well  as  later,  evaluating  the  value  of  the 
team  work  as  a  contribution  to  this  field  (Blasi  and 
Alfonso,  2006).  Not  only  does  the  work  of  D&D 
respond  to  larger  questions  and  problems  in  society 
but  it  also  has  an  impact  on  society.  Ethicists  are  argu¬ 
ing  that,  unless  those  on  the  front  lines  become  better 
able  to  appreciate  the  ethical  dimensions  of  the  issues 
they  are  addressing  and  become  more  adept  at  carrying 
out  responsible  solutions,  "the  power  of  science  and 
technology  will  continue  to  accelerate  [while]  the  abil¬ 
ity  of  humans  and  their  institutions  to  understand  and 
respond  to  the  changes  wrought  by  science  and  tech¬ 
nology  will  remain  more  or  less  the  same”  (Sarewitz, 
2000,  p.  14). 

The  work  of  designers  and  developers  take  place 
on  two  levels,  which  Klabbers  has  distinguished  as 
design  in  the  small  (DIS),  which  refers  to  the  design 
of  a  specific  innovation,  and  design  in  the  large  (DIL), 
which  refers  to  the  goal  of  changing  existing  situa¬ 
tions  into  preferred  ones  (2003a,  p.  489).  Individual 
D&D  projects  are  part  of  an  entire  system  in  which 
the  designer  and  the  developer  are  stakeholders  and 
have  an  impact  in  terms  of  both  DIS  and  DIL.  The 
following  three  examples  offer  a  glimpse  at  the  his¬ 
tory  and  context  for  the  unfolding  work  of  D&D 
teams.  These  examples  are  followed  by  prevalent 
models  and  challenges  from  the  research  literature. 
Each  example  illustrates  a  specific  instance  in  which 
the  work  within  individual  projects  was  then  imple¬ 
mented  within  a  larger  system,  with  related  social 
consequences. 
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THREE  EXAMPLES  OF 
DESIGN  AND  DEVELOPMENT 

The  Manhattan  Project  (Government) 

The  Manhattan  Engineer  District  included  37  installa¬ 
tions  in  19  states  and  employed  at  its  peak  some  150,000 
workers  (Williams,  1999).  The  Pulitzer  Prize  winning 
The  Making  of  the  Atomic  Bomb  (Rhodes,  1995)  was 
followed  by  more  analytical  histories  that  described  the 
context  for  the  team  work  and  criticized  its  authoritar¬ 
ian,  antidemocratic,  and  coercive  aspects  (Goldberg, 
1992;  Hughes,  2004).  Badash  (2005)  argued  that  the 
lack  of  a  relevant  tradition  and  the  disparate  nature  of 
the  project  provided  little  basis  for  anticipating  prob¬ 
lems  of  social  responsibility  (Badash,  2005).  The  devel¬ 
opment  and  design  tasks  were  divided  and  administered 
separately  at  the  various  sites  of  the  project.  Many 
involved,  including  Oppenheimer,  later  became  active 
in  the  political  discussions  of  nuclear  issues  (Badash, 
2005).  The  beginning  of  the  nuclear  arms  race,  along 
with  competition  with  the  USSR  for  manned  space 
flight,  increased  the  demand  for  educational  reform  in 
the  science,  technology,  engineering,  and  mathematics 
(STEM)  fields,  as  well  as  for  innovation  in  technology 
which  included  the  advent  of  the  personal  computer. 

Apple  (Industry) 

The  work  of  the  D&D  team  that  eventually  led  to 
today’s  Apple,  Inc.,  is  well-known  for  its  humble 
beginnings  in  a  family  garage  (Linzmayer,  1999).  As 
an  example  of  design  and  development,  the  Apple 
computer  started  out  as  a  compelling  example  of  self- 
managed  teamwork:  “The  fascination  of  the  story  of 
Apple  Computer  is  that  Wozniak  and  Jobs  broke  all 
the  rules.  In  many  cases,  they  deliberately  flaunted 
convention  and  they  still  succeeded,  in  conventional 
terms,  beyond  all  expectation”  (Elbinger,  1985,  p.  90; 
see  also  Wolf,  1996).  Unlike  the  Manhattan  Project, 
with  its  time  frame  defined  in  terms  of  national 
defense,  the  development  of  the  Apple  computer  was 
fueled  by  the  developers’  and  their  own  interest  in 
establishing  a  new  market  for  personal  computers.  The 
advent  of  the  personal  computer  paved  the  way  for 
computers  in  the  classroom  and  the  design  and  devel¬ 
opment  of  specific  applications  of  technology  that 
were  fun  to  play  with  and  could  work  in  schools. 

The  Adventures  of  Jasper  Woodbury  (Education) 

The  Adventures  of  Jasper  Woodbury  was  developed  by 
the  Cognition  and  Technology  Group  at  Vanderbilt 
(CTGV,  1992;  see  also  Bransford  et  al.,  1990),  which 


was  an  interdisciplinary  team  focused  on  education. 
The  CTGV  was  focused  on  developing  a  standards- 
based  program  linked  to  specific  course  content  that 
could  run  on  laserdisc  technology  while  also  drawing 
on  computers  and  teleconferencing.  The  developers 
aligned  their  development  of  activities  and  their  con¬ 
tent  with  the  National  Council  of  Teachers  of  Mathe¬ 
matics  (NCTM)  Standards  for  School  Mathematics 
(The  Learning  Technology  Center,  1992),  which  are 
among  several  content  standards  in  U.S.  education. 
The  Jasper  program  offers  published  examples  of  team 
development  and  design  specific  to  education,  as  well 
as  research  on  impact  and  implementation  (Barron  et 
al.,  1998;  Vye  et  al.,  1997). 

Certain  characteristics  of  teamwork  evident  in 
these  three  examples  are  more  systematically  dis¬ 
cussed  in  the  text  that  follows  in  ways  that  can  be 
applied  in  team  development  and  performance,  draw¬ 
ing  on  the  published  research  regarding  organizational 
theory  (see  Table  48.1). 

CURRENT  RESEARCH  ON  TEAM 
FORMATION,  DEVELOPMENT, 

AND  PERFORMANCE 

What  is  clear  from  these  examples  is  the  multidisci¬ 
plinary  nature  of  the  problems  that  were  addressed  with 
teams  that  were  diverse  in  terms  of  expertise  and  skills. 
Through  strong  leadership  and  a  shared  vision  members 
of  the  teams  were  able  to  effectively  collaborate.  The 
three  examples  of  innovation  in  design  and  development 
can  be  discussed  in  terms  of  their  goals,  specific  to  DIS 
and  DIL  (Klabbers,  2003a),  as  well  as  in  terms  of  the 
conditions  for  team  performance  and  the  social  conse¬ 
quences  of  the  team  products.  A  review  of  the  research 
literature  more  fully  presents  some  of  the  factors  that 
challenge  or  contribute  to  effective  team  performance. 

Defining  Teamwork 

Teamwork  is  characterized  by  a  well-designed  devel¬ 
opment  strategy,  by  clearly  identified  units  of  perfor¬ 
mance,  and  by  a  common  purpose  that  can  be  trans¬ 
lated  into  specific  performance  goals.  Researchers 
have  distinguished  between  working  groups  and  teams 
and  have  documented  the  differences.  Hoffman  et  al. 
(2001)  have  noted  that  deliberative  team  development 
strategies  were  seldom  referenced  throughout  their  54 
interviews  and  survey  responses  across  6  projects. 
Team  development  implies  putting  groups  of  people 
to  work  on  common  tasks.  Hoffman  and  colleagues 
(2001)  did  not  generally  describe  specific  actions  taken 
to  strengthen  the  characteristics  that  they  associated 
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TABLE  48.1 

Examples  of  Design  and  Development  Teams  and  Questions 

The  Adventures  of  Jasper 
Woodbury  (Education) 
How  can  design  and  development 
teams  contribute  to  the  field  of 
education? 


Question  raised  for  design 
and  development: 


The  Manhattan  Project 
(Government) 

What  are  the  responsibilities  and 
the  social  consequences  of 
design  and  development? 


Apple  Computer 
(Industry) 

What  factors  need  to  be  considered  for 
team  development  and  how  do  self- 
managed  teams  aim  for  success? 


with  the  best  or  superior  project  teams:  interdepen¬ 
dence,  cohesion,  and  commitment  (Tanskanen  et  al., 
1998).  Katzenbach  and  Smith  (1993),  in  their  study  of 
50  different  teams  from  over  30  companies,  noted  that 
groups  emphasize  individual  products,  individual 
accountability,  delegating  rather  than  doing  tasks,  and 
indirect  measures  of  effectiveness  (such  as  financial 
performance)  rather  than  evaluating  actual  work  of  the 
group.  In  contrast,  teams  will  share  leadership  roles 
and  seek  to  fulfill  a  specialized  purpose  beyond  the 
larger  organization,  with  individual  and  mutual 
accountability  focused  on  collective  work  products. 

According  to  Salas  et  al.  (1992,  p.  4),  teams  can 
be  defined  as: 

...  a  distinguishable  set  of  two  or  more  people  who 
interact,  dynamically,  interdependently,  and  adaptively 
toward  a  common  and  valued  goal/objective/mission, 
who  have  each  been  assigned  specific  roles  or  functions 
to  perform,  and  who  have  a  limited  life-span  of  mem¬ 
bership. 

Distributed  teams  have  emerged  with  individuals  pos¬ 
sessing  complementary  skills  but  separated  by  great 
distances  and  coordinating  their  actions  using  collab¬ 
oration  systems  supporting  teamwork  over  time  and 
space  (Fiore  et  al.,  2003;  Lipnack  and  Stamps,  2000). 

In  research  regarding  the  formation  and  work  of 
D&D  teams,  Hoffman  et  al.  (2001,  p.  2)  noted  that 
“rigorous  definition  and  analysis  of  human  variables 
have  often  been  lacking.  This  lack  has  created  a  ten¬ 
dency  to  approach  teams  from  a  ‘guru  of  the  month' 
philosophy.  Often  the  latest  trend  is  applied  to  a  project 
team,  as  opposed  to  a  well-designed  team  development 
strategy  based  on  research  and  experience”  (p.  2). 
Katzenbach  and  Smith  (1993,  p.  112)  suggested  that 
teamwork  requires  a  shift  from  thinking  about  the 
development  and  performance  of  teams  in  terms  of 
positive  values  toward  thinking  about  it  in  terms  of 
discrete  units  of  performance  (see  also  Faraj  and 
Sproull,  2000).  Failed  teams  typically  lack  a  common 
purpose,  but  the  best  teams  “translate  their  common 
purpose  into  specific  performance  goals”  (Katzenbach 
and  Smith,  1993,  p.  113). 


A  Model  for  Team  Development 

The  development  of  the  team  has  been  shown  to  follow 
patterns  of  activity  rather  than  a  linear,  sequential  pro¬ 
cess.  The  process  of  team  formation  was  described  by 
Tuckman  (1965),  who  drew  upon  a  review  of  50  articles 
in  group  development  to  identify  four  stages:  (1)  ori¬ 
entation/testing/dependence,  (2)  conflict,  (3)  group 
cohesion,  and  (4)  functional  role  relatedness.  These 
stages  are  also  referred  to  as  forming,  storming,  norm- 
ing ,  and  performing.  Building  on  Tuckman’s  work, 
Morgan  et  al.  (1998)  proposed  the  Team  Evolution  and 
Maturation  (TEAM)  model,  which  describes  stages  that 
have  been  observed  in  team  performance  that  take  into 
account  time,  experience,  and  training  and  “provides  a 
framework  for  developing  and  evaluating  instructional 
strategies,  training  methods,  interventions,  tools,  pro¬ 
cedures,  and  instructor  behaviors  that  can  be  applied  to 
the  enhancement  of  team  development”  (p.  288). 

Approaches  to  Leadership 

Individual  leadership  has  been  emphasized  in  the  liter¬ 
ature  on  organizational  behavior.  The  leader  may  not 
necessarily  be  the  manager  of  the  team.  Barak  et  al. 
(1999,  p.  93)  distinguished  between  the  “external  man¬ 
ager  appointed  by  management”  and  the  leader  who 
emerges  in  a  group  who  is  less  aligned  with  a  manage¬ 
rial  function  and  more  closely  associated  with  expertise 
and  charisma.  The  roles  of  the  leader  in  the  development 
and  performance  of  teams  are  complex.  Individuals 
report  higher  levels  of  commitment  and  stronger  satis¬ 
faction  when  they  perceive  they  have  some  influence  in 
the  decisions  made  by  their  team  leader  (Morris  et  al., 
2000).  The  effective  sharing  of  information  held  by  indi¬ 
viduals  can  become  part  of  the  D&D  process  if  the 
appropriate  conditions  are  established  within  the  team 
by  a  knowledgeable  leader  (Schittekatte,  1996;  Stasser, 
1992).  Drawing  on  Bass  (1985),  Sosik  et  al.  (1997,  p. 
90)  suggested  that  “transactional  leaders  exert  influence 
by  setting  goals  . . .  transformational  leaders  exert  influ¬ 
ence  by  broadening  and  elevating  followers’  goals.” 
When  teams  are  self-managed,  however,  they  require 
collaborative  leadership  for  optimal  performance. 
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Preparing  for  Participation 
in  Self-Managed  Teams 

Barry  (1991,  p.  34)  noted  that  within  a  self-managed 
team  the  “distributed  leadership  pattern  that  arises  . . . 
is  necessarily  an  emergent  one.  It  normally  begins  with 
different  members  initiating  directions  in  areas  they  are 
naturally  predisposed  toward  and  that  are  needed  by  the 
team.”  Such  teams  have  been  described  by  Barry  (1991 , 
p.  32)  as  requiring  “even  more  leadership  than  conven¬ 
tional  organizational  units.  In  addition  to  needing  task- 
based  leadership,  they  require  leadership  around  group 
development  processes,”  and  “without  the  presence  of 
formal  authority,  power  struggles  and  conflict  around 
both  task  and  process  issues  surface  more  often,  adding 
to  the  overall  leadership  burden  that  must  be  handled 
by  the  group.”  Stewart  and  Manz  (1995)  provided  a 
typology  and  an  integrative  model  of  leadership  for 
self-managing  work  teams  based  on  Bandura’s  (1977) 
social  learning  theory  and  the  continuum  of  leadership 
practices  previously  identified  by  Bass  (1990).  Stewart 
and  Manz  (1995,  p.  752)  presented  four  types  of  team- 
leader  behavior:  overpowering ,  powerless ,  power  build¬ 
ing,  and  empowered.  In  these  four  types  of  leader 
behaviors,  goals  influence  each  set  of  behaviors  and 
elicit  specific  reactions  and  outcomes  from  the  teams. 

Coordination  of  Process  and  Expertise 
Key  to  Team  Performance 

In  their  study  of  69  software  development  teams,  Faraj 
and  Sproull  (2000,  p.  1554)  reported  that  expertise 
coordination  shows  a  strong  relationship  with  team 
performance  that  remains  significant  over  and  above 
team  input  characteristics,  presence  of  expertise,  and 
administrative  coordination;  although  expertise  may 
be  evident,  the  quality  of  performance  relies  on  the 
team’s  ability  to  recognize  and  draw  on  that  expertise. 
Barry  (1991,  p.  33)  noted  that  “directive  and  socially 
centered  support  functions  might  vary  as  a  group 
matures,”  with  the  need  for  direction  being  higher  at 
the  beginning  of  projects  and  later  on  the  need  for 
support,  rather  than  direction,  increasing. 

Coordination  approaches  can  include:  (1)  those  in 
which  most  coordination  happens  at  the  end  of  the 
project,  (2)  the  incremental  approach  where  coordina¬ 
tion  takes  place  in  cycles  over  time,  and  (3)  the  fault- 
driven  approach  where  coordination  takes  place  at  the 
moments  of  highest  instability  (Chiang  and  Mooker- 
jee,  2004).  In  educational  projects,  this  coordination 
includes  the  educational  and  the  technical  expertise  of 
other  members  of  the  development  team,  as  they  use 
formative  evaluation  data  emphasizing  the  role  of  the 
learner  as  an  actor  in  the  interaction  and  how  the  user 


or  learner  will  undertake  the  task,  rather  than  focusing 
solely  on  the  learning  task  (Hedberg  and  Sims,  2001). 
Ultimately,  “proper  process  design  is  the  key  to  better 
productivity”  (Chiang  and  Mookerjee,  2004,  p.  92). 

Stabilizing  a  Project  Early  in  the  Process 

Observing  that  the  coordination  process  within  D&D 
teams  is  mostly  ad  hoc,  Chiang  and  Mookerjee  (2004, 
p.  967)  noted  that: 

. .  .project  teams  that  can  stabilize  modules  fairly  quickly 
can  afford  to  coordinate  less  frequently  early  on  in  the 
project.  ...One  trait  of  top-notch  developers  is  their  abil¬ 
ity  to  acquire  project-specific  knowledge  and  avoid  major 
rework — even  in  an  unfamiliar  project  domain.  Evidence 
of  this  learning  process  is  the  fact  that  the  development 
team  stabilizes  (or  “baselines”)  newly  developed  mod¬ 
ules  more  efficiently  as  the  project  progresses. 

Considering  the  role  individuals  play  in  the  process  of 
stabilization,  it  has  been  found  that  the  preference  for 
working  in  groups  that  individuals  bring  to  the  project 
and  the  positive  perception  of  group  capabilities  that 
individuals  develop  within  the  project  both  predict 
group  performance  (Jung  and  Sosik,  1999).  Looking 
at  ways  of  improving  team  productivity,  Chiang  and 
Mookerjee  (2004)  suggested  improving  software¬ 
building  technologies  and  refinement  of  the  develop¬ 
ment  process,  but  they  also  suggest  strengthening  indi¬ 
vidual  competencies  in  addition  to  analyzing  the  roles 
individuals  play  within  the  team. 

Managing  Individual  Goals  and  Behaviors 

Beyond  the  team  leader,  the  individual  decisions  made 
within  a  team  by  designers  and  developers  contribute 
to  their  own  levels  of  commitment  and  satisfaction. 
The  individuals  on  the  team  must  be  able  to  set  indi¬ 
vidual  goals  while  managing  their  own  performance. 
Drawing  on  social  learning  theory  (Bandura,  1977), 
Manz  and  Sims  (1980,  p.  361)  emphasized  that: 

...  individuals  manage  their  own  behaviors  by  setting 
personal  standards,  evaluating  their  performance  in  terms 
of  these  standards,  and  by  self-administering  conse¬ 
quences  based  on  their  self-evaluations.  Specific  tech¬ 
niques  such  as  self-observation,  goal  specification,  cue¬ 
ing  strategies,  incentive  modification,  and  rehearsal  can 
be  used  to  exercise  self-management  behavior. 

The  leader  of  the  team  can  facilitate  individual  self¬ 
management,  modeling  behaviors,  and  the  develop¬ 
ment  of  self-management  by  subordinates  by  exerting 
external  control  and  reinforcement  while  reducing  the 
need  for  supervision. 
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Individual  knowledge,  skills,  and  abilities  (KSAs) 
have  an  impact  on  team  development  and  perfor¬ 
mance,  specific  to  the  context,  the  team,  the  task,  or 
in  some  combination  of  these  (Cannon-Bowers  et  al., 
1995,  p.  357).  A  broader  conceptualization  of  what 
constitutes  KSAs  in  a  D&D  team  is  needed  to  account 
for  the  ethical  and  societal  dimensions  and  the  need 
for  multidisciplinary  problem  solving.  Seldom  are 
team  members  trained  for  awareness  and  responsive¬ 
ness  when  faced  with  either  of  these  challenges.  At 
the  same  time,  the  ability  to  develop  shared  mental 
models  and  strategic  consensus  become  part  of  this 
wider  conceptualization  of  KSAs  working  alongside 
the  diversity  specific  to  the  team  and  its  leadership. 

Team  Competencies  and 
Training  Requirements 

Team  development,  training,  and  performance  each 
have  to  be  understood  in  terms  of  the  context  of  the 
project— specifically,  the  tasks  required  and  their  asso¬ 
ciated  environmental  conditions.  To  determine  training 
requirements,  it  is  necessary  to  evaluate  the  compe¬ 
tency  requirements  for  the  task  at  hand  (Cannon-Bow¬ 
ers  et  al.,  1995).  Cannon-Bowers  and  her  colleagues 
expanded  the  definition  of  skills  in  KSAs  to  include 
also  psychomotor  and  cognitive  competencies.  Some 
competencies  are  task  generic  and  other  competencies 
are  task  specific,  with  team- specific  competencies 
(e.g.,  perceived  efficacy  and  compensation  strategies) 
working  alongside  team- generic  competencies  (e.g., 
communications  skills,  leadership  skills,  and  attitudes 
towards  teamwork).  With  respect  to  understanding 
complex  teamwork,  the  moderating  influence  of  task 
complexity  also  must  be  examined.  Component  com¬ 
plexity  describes  the  number  of  distinct  acts  associated 
with  a  task  as  well  as  the  cues  that  must  be  processed 
(Wood,  1986).  Here,  the  emphasis  is  on  the  importance 
of  informational  cues  given  that  cue  processing  is 
increasingly  recognized  as  a  critical  component  of 
expert  situation  assessment.  Coordinative  complexity 
describes  the  degree  to  which  task  variables  must  be 
integrated  for  successful  task  completion.  Awareness 
of  such  distinctions  is  necessary  for  coordination  and 
participation  in  a  complex  teamwork  situation. 

Understanding  the  Impact  of  Shared 
Mental  Models  for  Strategic  Consensus 

Shared  mental  models  also  are  crucial  in  the  work  of  a 
team,  as  all  members  of  the  team  need  to  share  an 
accurate  model  of  the  problem  to  be  addressed  and  the 
associated  task  to  assess  the  problem,  determine  their 
own  roles  and  responsibilities,  and  coordinate  efforts 


over  time  (Cannon-Bowers  et  al.,  1995).  Researchers 
have  investigated  strategic  consensus,  or  the  degree  to 
which  mental  models  overlap  without  necessarily 
drawing  on  a  deliberative  consensus-seeking  process. 
Measuring  four  demographic  indicators  of  diversity, 
Knight  et  al.  (1999)  found  that  functional  background 
and  employment  tenure  were  negatively  related  to  con¬ 
sensus,  with  less  evidence  of  impact  evident  for  differ¬ 
ence  in  age  and  level  of  education.  The  degree  of  con¬ 
sensus  has  an  impact  upon  the  strategic  orientation  of 
the  larger  organization  and  the  process  and  products 
expected  of  individuals  in  design  and  development.  At 
the  same  time,  D&D  teams  can  be  strengthened  by 
understanding  that  differences  in  functional  background, 
employment  tenure,  age,  and  level  of  education  will  con¬ 
tribute  to  or  challenge  the  team’s  ability  to  work  within 
shared  mental  models,  and  subsequently  it  will  impact 
their  ability  to  develop  strategic  consensus.  Diverse 
leadership  styles  are  required,  especially  in  teams  that 
are  self-managed,  but  as  Barry  (1991,  p.  40)  explained, 
using  these  differences  effectively  “entails  developing 
genuine  respect  for  the  diverse  styles  and  learning.” 

Establishing  the  Optimal  Context 
for  Team  Performance 

Competencies  within  teams  are  mitigated  by  the  con¬ 
text  in  which  the  team  operates.  Looking  at  distributed 
team  performance,  Fiore  and  his  colleagues  (2003,  p. 
343)  drew  on  three  inter-related  constructs: 

(1)  the  socio-cognitive  factors  arising  from  the  group 
dynamics  inherent  in  team  environments  (e.g.,  team 
attitudes);  (2)  the  artificial  components  consisting  of  the 
technology  enabling  the  interaction  among  distributed 
team  members  (e.g.,  computer-mediated  communica¬ 
tion);  and  (3)  the  dynamic  processes  associated  with 
distributed  team  interaction  (e.g.,  coordination  efforts 
occurring  over  time). 

The  types  of  collaboration  supported  by  the  devel¬ 
opment  environment  will  have  an  impact  on  the  process 
and  the  product  of  the  D&D  team.  When  they  examined 
the  effects  of  four  modes  of  group  communication  on 
the  outcomes  of  software  requirements  determination 
using  42  groups  of  graduates  in  computer  and  informa¬ 
tion  services  (CIS)  or  information  services  (IS)  pro¬ 
grams,  Ocker  et  al.  (1998)  found  that  the  combined 
group  (face-to-face  and  asynchronous)  demonstrated 
higher  quality  and  creativity  in  their  solutions  while 
also  reporting  higher  satisfaction  with  their  solutions. 
With  the  increasing  use  of  asynchronous  communica¬ 
tion  technologies  in  the  corporate  environment,  our 
research  suggests  that  the  use  of  a  communication 
mode  according  to  the  type  of  group  work  at  a  given 
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Development  Should  begin  with  Evaluation 
functions  and  be  Informed  by  functions 


Project  conceptualization 
Design 
Development 
Implementation 
Institutionalization 
Project  reconceptualization 


Review 

Needs  assessment 
Formative  evaluation 
Effectiveness  evaluation 
Impact  evaluation 
Maintenance  evaluation 


Figure  48.1  How  designers  and  developers  use  research  methods  for  evaluation.  Note:  The  arrows  indicate  direction  of  information 
flow;  they  go  from  the  evaluation  function  to  the  development  stage.  (Adapted  from  Reeves,  T.C.  and  Hedberg,  J.G.,  in  Flexible 
Learning  in  an  Information  Society ,  Khan,  B.,  Ed.,  Information  Science  Publishing,  Hershey,  PA,  2007,  pp.  226-235.) 


point  in  time  can  greatly  affect  the  effectiveness  of  a 
group’s  work  product  (Ocker  et  al.,  1998,  p.  114;  see 
also  Linebarger  et  al.,  2005). 

CHALLENGES  FOR  DESIGN 
AND  DEVELOPMENT  TEAMS 

Design  and  development  teams  can  aim  for  optimal 
performance  drawing  on  the  research  literature  in  orga¬ 
nizational  theory.  Their  projects  can  grow  out  of  their 
ability  to  access  and  apply  the  most  up-to-date  research 
on  how  and  what  students  learn  (Donovan  et  al.,  1999; 
Kolodner,  1991;  McGinn  and  Roth,  1999;  Pellegrino, 
2002),  as  well  as  on  specific  challenges  in  education 
such  as  the  assessment  of  learning  (Hunt  and  Pelle¬ 
grino,  2002).  It  is,  however,  the  team  members’  own 
use  of  research  methods  that  will  strengthen  and  align 
the  development  of  technology  for  teaching  and  learn¬ 
ing  in  relation  to  a  larger  vision  for  education.  To  align 
projects  within  a  larger  vision  for  education,  evaluation 
is  key  throughout  the  project.  The  relationship  between 
development  and  evaluation  has  been  described  by 
Reeves  and  Hedberg  (2003)  in  Figure  48 . 1 . 

The  development  and  design  process  across  edu¬ 
cation,  the  government,  and  industry  is  often  initiated 
on  a  project-basis;  however,  researchers  have  provided 
frameworks  for  addressing  issues  of  scalability  and 
sustainability  that  consider  school  capabilities,  policy 
and  management  structures,  and  organizational  culture 
(Blumenfeld  et  al.,  2000).  Researchers  have  also 
described  the  “incompatibilities  between  the  demands 
of  the  innovations  being  introduced  by  the  research 
community  and  the  extant  culture,  capability,  and  man¬ 
agement  structures  of  schools”  (Fishman  et  al.,  2001). 


After  the  project  is  complete,  planning  for  scalability 
and  sustainability  must  draw  on  evaluation  data  to 
align  the  project  with  an  overarching  vision  for  edu¬ 
cation.  Here,  the  evaluation  data  for  effectiveness, 
impact,  and  maintenance  are  key  (Reeves  and  Hed¬ 
berg,  2003).  Four  stages  of  evaluation  within  design 
and  development  begin  with  the  educational  goal  for 
the  artifact  and  then  research  methods  are  used  by  the 
D&D  team  to:  (1)  verify  the  artifact,  (2)  check  the 
variation  of  the  content,  (3)  verify  its  use  in  context, 
and,  finally,  (4)  document  its  value  in  the  wider  field 
of  education  (Blasi  and  Alfonso,  2006). 

Beyond  applying  research  methods  for  the  forma¬ 
tive  and  summative  evaluation  of  technologies  for  teach¬ 
ing  and  learning,  researchers  have  developed  a  frame¬ 
work  for  research  regarding  innovation  that  calls  for 
research  in  the  context  of  systemic  reform  to  address 
issues  of  usability,  scalability,  and  sustainability  across 
projects  (Fishman  et  al.,  2004).  Although  evaluation 
data  can  contribute  to  systematic  change  in  education, 
the  research  and  development  process  can  also  track  the 
ways  in  which  technologies  for  teaching  and  learning, 
as  educative  materials,  actually  have  an  impact  on 
teacher  learning  as  well  as  on  student  learning 
(Schneider  and  Krajcik,  1999). 

Team  Purpose  and  Goals 
Informed  by  Research 

The  research  literature  on  D&D  teams  offers  a  frame¬ 
work  for  optimal  performance,  but  the  research  liter¬ 
ature  regarding  teaching  and  learning  can  also  be  used 
by  teams  to  inform  projects  developed  for  education 
(Quintana  et  al.,  2004;  Singer  et  al.,  2000;  Talsma  and 
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Krajcik,  2002).  When  they  use  research  in  teaching 
and  learning  to  inform  decisions  in  D&D,  designers 
and  developers  are  transforming  research  findings  into 
learning  experiences.  The  extent  to  which  teams  are 
able  to  access  and  apply  the  research  on  teaching  and 
learning  will  determine  the  extent  to  which  larger, 
long-term  social  and  economic  goals  are  met  by  design 
and  development  in  education. 

A  specific  example  of  design  and  development  that 
transforms  the  research  on  teaching  and  learning  can 
be  found  in  Alphie’s  Alley,  which  was  developed  by 
the  Centre  for  the  Study  of  Learning  and  Performance 
(CSLP,  2006).  The  Success  for  All  Foundation  (SFA) 
offers  a  comprehensive,  schoolwide  literacy  program 
called  Reading  Roots,  which  emphasizes  early  inter¬ 
vention  to  anticipate  and  solve  problems  in  reading 
(Slavin  and  Madden,  2001),  and  Alphie’s  Alley  is  the 
electronic  performance  support  system  (Gery,  2001) 
for  Reading  Roots.  It  was  developed  in  collaboration 
with  SFA  to  support  the  daily,  one-on-one  tutoring  for 
the  lowest  performing  30%  of  first  graders  in  this 
schoolwide  literacy  program  throughout  high-poverty 
Title  I  schools  that  serves  more  than  1  million  children. 

In  Alphie’s  Alley,  the  software  plays  a  supporting 
role  by  connecting  the  student  and  tutor  through  the 
computer,  informed  by  the  conviction  that  pedagogy 
must  direct  technology  (Abrami  et  al.,  2005;  Clark, 
2001;  Hipps  et  al.,  2005;  Kozma,  1994a,b,  2000),  and 
it  was  also  developed  out  of  the  research  on  early  lit¬ 
eracy  (Adams,  1990;  Slavin  and  Madden,  2001).  Built 
on  a  foundation  of  research  in  reading  and  evaluation 
using  a  randomized  control  group  design,  Alphie’s 
Alley  has  yielded  significant  positive  effects  (Slavin  et 
al.,  2006)  and  positive  tutor  acceptance  (Schmid  et  al., 
2006),  thus  serving  as  just  one  example  of  a  project 
that  transforms  the  research  in  teaching  and  learning 
into  an  experience  enabled  through  technology. 

CONCLUDING  DISCUSSION 
AND  SUMMARY 

The  Social  Consequences 
of  Design  and  Development 

As  we  have  discussed,  design  and  development  teams 
occupy  an  important  niche  in  our  society.  They  are 
responsible  for  a  form  of  creative  problem  solving  that 
spans  the  development  of  educational  software  to  the 
construction  of  the  nuclear  bomb.  We  need  to  under¬ 
stand  the  knowledge,  skills,  and  abilities  necessary  for 
those  making  up  D&D  teams.  On  the  one  hand  is  the 
issue  of  leading  such  teams;  on  the  other  hand  is  the 
level  of  competency  required  by  such  teams,  as  dis¬ 
cussed  earlier.  If  we  can  understand  how  teams  attempt 


to  understand  task  complexity  within  their  work  (DIS), 
we  can  better  help  them  understand  how  this  work 
more  broadly  has  an  impact  on  society  (DIL). 

Unfortunately,  such  teams  often  work  to  solve 
problems  of  such  complexity  that,  without  the  requisite 
team  training,  they  may  not  achieve  their  potential. 
Furthermore,  research  shows  that  when  we  attempt  to 
understand  complex  problems,  we  often  oversimplify 
the  situation  in  our  attempts  to  comprehend  as  much 
as  possible.  Indeed,  understanding  complex  constructs 
represents  an  important  challenge  to  the  welfare  of 
humankind  given  that  misinterpretation  of  factors 
within  a  system  or  mismanagement  of  these  factors 
can  have  devastating  consequences.  Scientific  analysis 
of  complex  systems  has  identified  a  number  of  factors 
that  make  them  particularly  challenging  to  understand, 
including  constant  change  coupled  with  high  interde¬ 
pendence  of  multiple  variables  (Feltovich  et  al.,  1997; 
Spiro  et  al.,  1992).  Problematically,  these  complexities 
are  often  overlooked  because  of  our  tendency  to  build 
simplified  understandings  and  explanations  that  lead 
to  misconceptions  and  possibly  errors— a  phenomenon 
termed  the  reductive  bias  (Feltovich  et  al.,  2004). 

Almost  20  years  after  an  Association  for  Educa¬ 
tional  Communications  and  Technology  (AECT) 
survey  of  80  professors  of  instructional  systems 
design  and  technology  sought  to  assess  the  current 
thinking  of  association  members  toward  ethics  in  the 
field  (Nichols,  1987),  there  is  a  growing  discussion 
about  ethics  in  terms  of  social  consequences .  Twenty 
years  ago  most  concluded  that  “they  address  ethical 
issues  in  a  variety  of  ways,  but  that  more  needs  to  be 
done”  (Nichols,  1987).  At  that  time,  most  reported 
discussing  technology  impact  “from  the  effects  of 
technology  on  learners  in  general  to  the  effects  on 
whole  cultures”  rather  than  in  regard  to  the  designer 
role  (Nichols,  1987).  Findeli  (2001,  p.  14)  illumi¬ 
nated  the  distinction  between  technicians  (focused  on 
the  DIS)  and  designers  (focused  on  the  DIS  and  DIL): 

Within  these  complex  systems,  designers  are  expected  to 
act  rather  than  to  make  ...  In  philosophical  terms,  one 
would  say  that  design  pertains  to  practical,  not  to  instru¬ 
mental,  reason;  or  else  that  the  frame  of  the  design  project 
is  ethics,  not  technology.  In  existentialist  terms,  this  could 
sound  as  follows:  design  responsibility  means  that 
designers  always  should  be  conscious  of  the  fact  that, 
each  time  they  engage  themselves  in  a  design  project, 
they  somehow  recreate  the  world,  [emphasis  original] 

History  offers  clear  examples  of  failed  innovations 
that  once  held  the  promise  of  improved  teaching  and 
learning  (Cuban,  1986, 2001).  The  documented  process 
regarding  change  and  the  diffusion  of  innovations 
shows  that  success  or  failure  relies  in  part  on  how  and 
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when  the  innovation  is  introduced  and  supported  (Hall 
and  Hord,  1987;  McLaughlin,  1990;  Rogers,  1995). 
Although  designers  and  developers  cannot  control 
what  happens  to  a  product  once  it  is  released,  it  is 
possible  to  train  for  optimal  team  work,  mindful  of 
long-term  social  consequences.  D&D  teams  can  (1) 
organize  in  ways  informed  by  the  research  from  orga¬ 
nizational  theory,  (2)  access  and  apply  the  research 
regarding  teaching  and  learning,  and  (3)  conduct 
research  toward  formative  and  summative  evaluation 
of  technology  in  education.  Through  this  process,  they 
can  also  (4)  contribute  to  the  research  in  the  field 
through  their  own  innovations .  In  this  way,  D&D  teams 
are  co-creators  within  larger  efforts  toward  systemic 
change  in  education,  mindful  of  the  social  conse¬ 
quences  of  design  and  development. 

Summary 

The  work  of  designers  and  developers  takes  place  on 
two  levels,  which  Klabbers  has  distinguished  as  design 
in  the  small  (DIS),  which  refers  to  the  design  of  a 
specific  innovation,  and  design  in  the  large  (DIL), 
which  refers  to  the  goal  of  changing  existing  situations 
into  preferred  ones  (Klabbers,  2003a,  p.  489).  Design¬ 
ers  and  developers  are  stakeholders  and  have  an  impact 
in  terms  of  both  DIS  and  DIL.  With  the  distinction 
made  between  individual  projects  and  the  entire  sys¬ 
tem,  a  glimpse  at  the  history  and  context  for  the  unfold¬ 
ing  work  of  design  and  development  teams  in  three 
examples  was  then  followed  by  prevalent  models  and 
challenges  for  teams  from  the  research  literature.  Spe¬ 
cifically,  in  this  chapter,  we  began  with  a  discussion 
of  three  examples  of  teamwork  from  the  20th  century. 
Each  example  illustrates  a  specific  instance  in  which 
the  work  within  individual  projects  was  then  imple¬ 
mented  within  a  larger  system  with  related  social  con¬ 
sequences.  Building  on  the  questions  raised  by  these 
examples,  we  discussed  some  of  the  lessons  learned 
for  team  development  and  management  drawn  from 
the  research  literature,  and  we  reviewed  the  core  con¬ 
cepts  necessary  for  effective  design  teams.  These 
include  the  specification  of  performance  factors,  such 
as  well-stated  design  purposes  and  goals;  identification 
of  the  knowledge,  skills,  and  abilities  foundational  to 
the  team’s  purpose;  and,  last,  a  shared  understanding 
of  factors  related  to  optimal  teamwork.  Our  goals  in 
this  chapter  were  to  ( 1 )  highlight  the  complexity  inher¬ 
ent  in  design  and  development,  (2)  describe  how 
design  and  development  teams  can  draw  from  and 
contribute  to  research  in  the  field  of  education,  and  (3) 
convey  an  appreciation  of  how  theory  and  methods 
from  differing  disciplines  can  aid  our  understanding 
and  improvement  of  this  important  process. 
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ABSTRACT 

This  chapter  surveys  methods,  techniques,  practices, 
and  challenging  issues  in  user-centered  design  and 
development  (UCDD).  The  traditional  instructional 
systems  design  (ISD)  approach  has  been  criticized  for 
its  bureaucratic  and  linear  nature  and  its  slow  process. 
Two  alternatives  to  that  approach  are  discussed  here: 
rapid  prototyping  and  participatory  design.  These  have 
been  put  forth  as  alternative  models  that  address  the 
many  limitations  of  the  conventional  ISD  model. 


KEYWORDS 

Participatory  design:  A  user-centered  design  approach 
in  which  users  are  actively  involved  in  the  design 
process  of  a  system  or  product  that  addresses  their 
specihc  needs. 
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INTRODUCTION 

One  of  the  most  frequent  and  important  challenges 
faced  by  instructional  technologists  is  how  to  design 
and  develop  a  product  or  program  that  both  supports 
users'  learning  and  performance  in  an  effective  and 
efficient  manner  and  also  generates  user  satisfaction. 
Recently,  new  approaches  to  the  processes  used  in 
instructional  design  have  been  proposed  and  explored. 
Many  researchers  have  pointed  out  that  the  traditional 
instructional  systems  design  (ISD)  approach  is  reduc¬ 
tionist  in  nature  and  that  it  tends  to  solve  a  problem 
by  fragmentation,  one  stage  at  a  time  (Finegan,  1994; 
Jonassen,  1990;  You,  1993).  In  Gordon  and  Zemke 
(2000)  and  Zemke  and  Rossett  (2002),  several 
researchers  and  practitioners  attacked  the  traditional 
ISD  approach  for  its  bureaucratic  and  linear  nature  ,  as 
well  as  its  slow  and  clumsy  processes. 

The  adoption  of  user-centered  design  and  develop¬ 
ment  (UCDD)  into  ISD  is  vital  for  designing  systems 


that  better  serve  users’  needs  (Willis  and  Wright, 
2000).  If  ISD  does  have  to  go  through  a  paradigmatic 
transition,  along  with  changes  in  the  educational  and 
socioeconomic  environment,  then  the  new  paradigm 
of  ISD  must  reflect  these  environmental  changes.  This 
would  mean  that  the  ISD  process  should  become  more 
user  centered,  more  cost  and  time  effective,  and  more 
performance  focused. 

The  concept  of  UCDD  is  to  place  users  at  the 
center  of  the  design  process  from  the  stages  of  plan¬ 
ning  and  designing  the  system  requirements  to 
implementing  and  testing  the  product.  UCDD 
appears  in  many  different  forms  within  design 
approaches.  In  this  chapter,  we  have  chosen  a  philo¬ 
sophical  approach  to  object  and  systems  design,  par¬ 
ticipatory  design  (PD),  and  a  particular  process, 
rapid  prototyping  (RP),  to  elucidate  the  overall  per¬ 
spective  of  user-centered  design.  First,  we  review  the 
big  picture  for  UCDD,  then  we  examine  the  partic¬ 
ipatory  design  approach— beginning  with  its  histor¬ 
ical  background  and  then  focusing  on  the  different 
participation  levels  within  this  approach.  This  is  fol¬ 
lowed  by  a  description  of  rapid  prototyping  and  a 
discussion  of  its  challenges.  Before  concluding,  the 
UCDD  approach  is  reviewed  in  light  of  instructional 
design  paradigms. 

THE  BIG  PICTURE  FOR  UCDD 
Key  Elements  of  UCDD 

What  is  UCDD?  As  Bannon  (1991,  p.  38)  stated, 
"What  the  term  user-centered  system  design  means  or 
how  it  can  be  achieved  is  far  from  clear.”  To  begin 
sorting  the  issue  out,  we  observe  that  there  are  two 
types  of  approaches  to  design  and  development:  prod¬ 
uct-oriented  and  process-oriented.  The  product-ori¬ 
ented  approach  focuses  mainly  on  the  creation  of  a 
product.  The  utilization  of  the  product  can  be  a  fixed 
and  well  understood  idea;  this  means  that  design 
requirements  can  be  determined  in  advance.  The  pro¬ 
cess-oriented  approach  requires  designers  to  view  their 
entire  process  of  development  in  the  context  of  human 
learning,  work,  and  communication  (i.e.,  use).  The 
usage  of  the  product  in  development  takes  place  in  an 
evolving  world  of  changing  needs.  This  involves  cer¬ 
tain  advantages  but  also  imposes  various  constraints. 
Because  change  is  the  norm  in  the  process,  prior  spec¬ 
ifications  for  an  end-product  are  not  predetermined 
completely.  In  UCDD,  plans  are  just  the  beginning  of 
the  process,  but  the  main  mission  is  not  conforming 
to  the  plan;  rather,  it  is  responding  to  changes  through¬ 
out  the  life  cycle  of  the  project. 
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Our  focus  here  is  on  process-oriented  approaches , 
specifically  those  that  fall  under  the  sociotechnical 
umbrella.  The  sociotechnical  perspective  considers  not 
only  technical  aspects  of  a  system  (tools,  techniques, 
procedures)  but  also  social  aspects  (people,  network 
of  roles,  relationships,  and  tasks)  (Goodrum  et  al., 
1993;  Mumford,  1983).  To  be  able  to  implement  the 
sociotechnical  approach  in  system  design,  information 
must  be  extracted  from  the  social  context. 

User-centered  design  and  development  can  be  con¬ 
sidered  a  subcircle  of  the  sociotechnical  approach. 
UCDD  and  the  sociotechnological  perspective  are 
guiding  philosophies,  not  specific  methods  or  processes 
for  design.  The  idea  is  to  approach  design  with  knowl¬ 
edge  of  and  the  will  to  utilize  social  and  cognitive 
analyses  of  human  activities.  These  become  the  basis 
of  the  given  project  and  direct  its  development;  hence, 
the  UCDD  approach  to  design  emphasizes  user  require¬ 
ments  and  strives  to  keep  those  in  mind.  Designers  are 
required  to  initiate  early  and  continuous  contact  with 
prospective  users  to  elicit  what  they  need  and  how  they 
will  learn  and  perform.  The  approach  also  stresses  that 
user-oriented  technology  in  development  must  be 
tested  for  usability.  These  tests  are  done  iteratively  as 
opposed  to  using  phased-stage  or  lock-step  testing. 
These  key  elements  of  UCDD  can  be  summarized  as 
user  participation  (mutual  learning),  contextual  inquiry, 
and  iterative  design.  Each  element  is  discussed  below. 

User  Participation 

Users  of  technology  are  simply  those  who  make  use  of 
the  tools  that  designers  create;  however,  this  term  should 
be  further  refined  for  our  present  purpose.  Maguire 
(2001)  and  McCracken  and  Wolfe  (2004)  differentiated 
primary  users  from  more  broadly  defined  users.  Primary 
users  are  those  who  will  directly  use  and  interact  with 
the  system  to  do  tasks,  and  more  broadly  defined  users 
are  stakeholders— that  is,  anyone  who  will  be  influ¬ 
enced  by  primary  users’  capabilities  to  carry  out  their 
tasks  or  who  affects  the  system  requirements.  The 
voices  of  both  primary  users  and  stakeholders  should 
be  respected  in  the  design  decision-making  process. 

User  participation  is  vital  in  UCDD  design,  so  users 
should  be  actively  involved  in  the  entire  design  pro¬ 
cess— not  simply  consulted  at  the  beginning  or  at  the 
testing  stages  of  a  product.  Users  can  contribute  impor¬ 
tant  “folk  knowledge”  derived  from  their  work  contexts 
(Walenstein,  2002,  p.  21).  In  this  regard,  designers 
should  understand  that  users  typically  know  more  than 
what  they  can  initially  verbalize.  If  properly  ques¬ 
tioned,  they  may  provide  useful  feedback  on  proposed 
design  ideas  (Nisbett  and  Wilson,  1977).  This  interac¬ 
tive  process  also  potentially  increases  the  users’  accep¬ 


tance  of  the  product  or  system  under  development. 
Designers  must  take  care  to  respect  the  users’  various 
backgrounds  and  fields  of  expertise;  this  is  a  necessary 
condition  for  mutual  learning  (Muller,  2003). 

The  methods  included  under  the  UCDD  perspec¬ 
tive  vary  as  to  the  timing  and  amount  of  user  partici¬ 
pation  they  include,  from  Carr-Chellman’s  (2007) 
insistence  on  users  being  fully  franchised  as  design 
peers  throughout  the  process  to  the  sometimes  minimal 
role  played  by  test  subjects  in  rote  usability  testing  that 
occurs  too  late  in  the  design  cycle  for  changes  to  be 
made  to  a  product  (Krug,  2005).  At  the  1994  Partici¬ 
patory  Design  Conference,  Tom  Erickson  of  Apple 
Computer  suggested  four  dimensions  of  user  partici¬ 
pation  (Kuhn  and  Winograd,  1996).  These  include 
direct  interaction  with  the  designers,  long-term 
involvement  in  the  design  process,  broad  participation 
in  the  overall  system  being  designed,  and  maintaining 
a  significant  degree  of  control  over  design  decisions. 

Contextual  Analysis 

Another  key  element  in  UCDD  is  considering  the 
users’  work  needs  in  context.  From  the  sociotechnical 
perspective,  Goodrum  et  al.  (1993)  argued  that  design¬ 
ers  must  take  into  account  the  dynamics  of  people, 
environment,  work  practices,  and  technology  to 
develop  an  enriched  learning  and  information  environ¬ 
ment.  Along  the  same  lines,  Read  et  al.  (2002)  sug¬ 
gested  various  contextual  variables  that  influence 
users’  participation  in  design  activities.  These  include 
environment,  knowledge,  skills,  and  security.  Read  et 
al.  (2002,  p.  60)  reported  that: 

•  The  cultural  and  physical  environment  in 
which  a  participatory  design  activity  takes 
place  will  affect  the  activity. 

•  Each  participant  will  bring  to  the  design 
activity  his  or  her  own  general  knowledge, 
subject  knowledge,  and  technical  knowledge. 

•  The  skills  that  will  affect  the  ability  of  indi¬ 
viduals  to  contribute  to  a  participatory  design 
activity  include  cognitive  skills,  motor  skills, 
and  articulatory  skills.  Different  participants 
will  bring  different  skills  to  any  project,  and 
it  is  likely  that  the  balance  of  skills  within  a 
group  will  affect  its  functionality. 

•  Comfort  factors,  emotional  stability,  and 
stress  also  have  an  effect  on  how  people 
contribute  to  a  group  activity.  These  factors 
can  be  quite  individual  and  are  difficult  to 
predict.  Feelings  of  security  within  a  group 
will  also  be  influenced  by  environment, 
knowledge,  and  skills. 
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Iterative  Design 

In  UCDD,  designers  are  expected  to  initiate  early  con¬ 
tact  with  potential  users  and  then  focus  continuously 
on  what  these  users  require  of  the  technology  to  be 
designed.  Testing  must  be  done  iteratively,  in  response 
to  design  questions  and  advances  rather  than  being 
carried  out  on  the  basis  of  phases  in  a  predetermined 
design  process.  The  iterative  process  is  one  of  reflec- 
tion-in-action  in  which  development  stages  are  shaped 
in  context  to  deal  intelligently  and  creatively  with 
“uncertainty,  uniqueness,  and  value  conflict”  in  a  con¬ 
stantly  changing  world  (Schon,  1987,  p.  6). 

Iterative  design  is  closely  related  to  the  concept  of 
design  space ,  an  idea  borrowed  from  the  fields  of  archi¬ 
tecture  and  graphic  design.  As  Beadouin-Lafon  and 
Mackay  (2003,  p.  1011)  explained  design  space: 

Designers  are  responsible  for  creating  a  design  space 
specific  to  a  particular  design  problem.  They  explore  this 
design  space,  expanding  and  contracting  it  as  they  add 
and  eliminate  ideas.  The  process  is  iterative:  more  cyclic 
than  reductionist.  That  is,  the  designer  does  not  begin 
with  a  rough  idea  and  successively  add  more  precise 
details  until  the  final  solution  is  reached.  Instead,  she 
begins  with  a  design  problem,  which  imposes  a  set  of 
constraints,  and  generates  a  set  of  ideas  to  form  the  initial 
design  space.  She  then  explores  this  design  space,  pref¬ 
erably  with  the  user,  and  selects  a  particular  design  direc¬ 
tion  to  pursue.  This  closes  off  part  of  the  design  space, 
but  opens  up  new  dimensions  that  can  be  explored.  The 
designer  generates  additional  ideas  along  these  dimen¬ 
sions,  explores  the  expanded  design  space,  and  then 
makes  new  design  choices. 

When  designers  expand  the  design  space  to  generate 
ideas  and  contract  it  to  select  ideas,  various  design 
tools  and  techniques  are  used.  Besides  the  most  gen¬ 
erally  used  techniques  such  as  questionnaires,  inter¬ 
views  (including  individual  interviews,  focus  groups, 
and  workshops),  and  document  analyses,  other  tools 
and  techniques  may  be  used  to  facilitate  the  iterative 
design  process.  These  include  task  analysis,  prototyp¬ 
ing  (Beadouin-Lafon  and  Mackay,  2003;  Ehn  and 
Kyng,  1991),  role-playing  activities  (Ehn,  1992),  site 
visitation  and  observation  (Ehn,  1992),  scenarios  (Car- 
roll,  1995,  2000),  personas  within  design  scenar¬ 
ios— virtual  people  who  have  jobs,  hobbies,  families, 
and  educational  accomplishments  (Grudin  and  Pruitt, 
2002)— and  virtual  reality  (Davies,  2004). 

Process  Approaches  within 
the  UCDD  Perspective 

Under  UCDD  we  place  multiple  process  approaches. 
These  include  participatory  design  (PD)  (Bodker  et 


al.,  1988),  rapid  prototyping  (RP)  (Goodrum  et  al., 
1993;  Frick  et  al.,  2005),  user-friendly  design  (Corry 
et  al.,  1997;  Dumas  and  Redish,  1993;  Norman,  1988; 
Sugar  and  Boling,  1995),  pluralistic  walkthrough 
(Bias,  1994),  contextual  design  (Beyer  and  Holtzblatt, 
1998;  Tessmer  and  Wedman,  1995),  cooperative 
inquiry  (Druin,  1999),  situated  design  (Greenbaum 
and  Kyng,  1991),  the  user-designer  approach  (Reige- 
luth,  1996),  ID2  transaction  shells  (Merrill  et  al., 

1992) ,  R2D2  model  (Willis  and  Wright,  2000),  eman¬ 
cipatory  design  (Carr-Chellman  and  Savoy,  2004),  and 
user  design  (Carr-Chellman,  2007).  Although  these 
perspectives  are  not  identical  or  equivalent,  the  com¬ 
mon  thread  among  them  is  that  in  all  of  them  users 
actively  participate  to  a  greater  or  lesser  degree  in  the 
design  of  a  system  or  a  product.  To  illuminate  the 
overall  perspective  of  user-centered  design,  we  have 
chosen  a  philosophical  approach  to  object  and  systems 
design  (participatory  design)  and  a  particular  process 
(rapid  prototyping)  to  discuss  in  further  detail. 

CHARACTERIZATION  OF 
PARTICIPATORY  DESIGN 

History  of  Participatory  Design 

Participatory  design  is  both  a  set  of  theories  for,  and 
the  practice  of,  using  users’  preferences  to  design  prod¬ 
ucts  or  systems.  As  explained  by  Greenbaum  and  Kyng 
(1991,  p.  4)  in  participatory  design,  designers  are 
required  to  take  users’  work  practices  and  needs  seri¬ 
ously;  users  are  regarded  as  "human  actors,”  not  as  cut- 
and-dried  “human  factors.”  Their  work  practices  must 
be  viewed  within  their  own  situated  contexts.  Obser¬ 
vations  of  users’  social  interactions  in  the  workplace 
are  also  employed  by  the  designer,  thus  requiring  con¬ 
tinuous  communication  between  users  and  designers. 

The  roots  of  systems  and  product-generating  par¬ 
ticipatory  design  can  be  traced  back  to  early  Scandina¬ 
vian  systems  design  efforts  in  the  1970s  (Ehn,  1988, 

1993) .  It  began  with  a  political  labor  movement  to  bring 
democracy  to  work  settings.  Early  projects  usually  took 
the  form  of  collaborations  between  computer  science 
researchers  and  union  workers.  Participatory  design 
was  pioneered  by  Kristen  Nygaard,  whose  work 
involved  collaboration  with  union  leaders  and  members 
to  create  a  Norwegian  national  agreement  to  ensure  the 
rights  of  unions  regarding  the  design  and  use  of  tech¬ 
nology  in  the  workplace  (Ehn,  1988;  Kuhn  and  Wino- 
grad,  1996).  This  triggered  other,  similar  projects  in 
Scandinavia.  In  Sweden,  the  DEMOS  project  involved 
an  interdisciplinary  team  of  researchers  who  collabo¬ 
rated  with  trade  unions.  With  collaboration  between 
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Swedish  and  Danish  researchers  and  the  Nordic  Group 
Graphic  Workers’  Union,  the  UTOPIA  project  was 
created  to  design  and  develop  a  computerized  desktop 
publishing  system  for  newspaper  graphic  designers 
(Ehn,  1992). 

The  emphasis  of  this  labor  movement  to  empower 
users  gradually  changed  in  response  to  societal 
changes.  After  reviewing  ten  participatory  design 
projects  in  the  area  of  software  development  ranging 
from  the  1970s  to  the  1980s,  Clement  and  Van  den 
Besselaar  (1993)  observed  that  the  focus  of  this  labor 
movement  shifted  from  empowering  workers  in  gen¬ 
eral  to  empowering  specifically  minority  and  female 
workers.  This  change  reflected  an  increase  in  the  pop¬ 
ulation  of  women  in  the  workplace.  When  participa¬ 
tory  design  was  eventually  applied  in  the  United  States , 
this  political  focus  was  deemphasized  (Clement  and 
Van  den  Besselaar,  1993).  Now  participatory  design 
has  widened  to  other  fields  such  as  engineering,  archi¬ 
tecture,  and  community  design  (Al-Kodmany,  1999; 
Carroll  et  al.,  2000;  Cohen,  2003). 

Different  Levels  of  User  Participation 

As  discussed  earlier,  there  are  varying  degrees  of  user 
participation  within  participatory  design.  Although  the 
definition  of  what  constitutes  participation  varies  in 
different  projects,  Kensing  offered  basic  requirements 
for  participation:  “The  employee  must  have  access  to 
relevant  information;  they  must  have  the  possibility  for 
taking  an  independent  position  on  the  problems,  and 
they  must  in  some  way  participating  in  the  process  of 
decision  making”  (cited  in  Clement  and  Van  den 
Besselaar,  1 993 ,  p .  3 1 ) .  According  to  Willis  and  Wright 
(2000,  p.  7),  there  are  “weak  participatory  design”  and 
“strong  participatory  design”  processes.  In  weak  par¬ 
ticipatory  design,  design  decision  making  is  mainly 
undertaken  by  the  designers  themselves,  even  though 
user  inputs  are  solicited  using  various  tools  and  tech¬ 
niques.  In  strong  participatory  design,  the  users’  full 
participation  is  utilized  throughout  the  entire  design 
process.  Combining  these  interpretations  with  Erick¬ 
son’s  user  participation  dimensions  (Kuhn  and  Wino- 
grad,  1996),  Table  49.1  summarizes  the  different  levels 
of  user  participation. 

With  different  combinations  of  these  dimensions, 
user  participation  levels  may  range  from  minimal  to 
full  inclusion  (Read  et  al.,  2002)  and  to  emancipatory 
design  or  user  design— empowering  stakeholders  in 
the  design  (Carr-Chellman,  2007;  Carr-Chellman  and 
Savoy,  2004).  At  the  minimal  level,  users  may  partic¬ 
ipate  in  the  design  process  for  a  limited  time  or  with 
a  limited  scope  of  influence.  At  the  full  inclusion  level 
and  the  emancipatory  level,  users  are  empowered  to 


TABLE  49.1 

Levels  of  User  Participation 

Weak  Participation  Strong  Participation 

Interaction  Indirect  Direct 

Length  Short  Long 

Scope  Small  Large 

Control  Very  limited  Very  broad 

participate  in  the  design  process  by  cooperating  with 
researchers  and  developers  or  carrying  out  the  design 
themselves  with  primarily  facilitation  provided  by 
trained  designers. 

Application  of  Participatory  Design 

In  Clement  and  van  den  Besselaar’s  1993  article, 
many  successful  cases  of  participatory  design  projects 
are  surveyed.  These  are  cases  of  projects  in  system 
design  for  work  settings  (e.g.,  computer  center, 
human-centered  office,  local  government)  conducted 
since  the  1970s,  including  architecture,  urban  plan¬ 
ning,  and  community  design  (Al-Kodmany,  1999; 
Cohen,  2003),  as  well  as  recordkeeping  in  healthcare 
training  (Carr-Chellman  et  al.,  1998).  It  should  be 
noted  that  participatory  design  projects  in  education 
are  relatively  under-researched  (Carroll  et  al.,  2000). 
In  this  section,  we  briefly  illustrate  one  research  and 
design  project  that  has  successfully  integrated  partic¬ 
ipatory  design  for  computer  system  designs  in  the 
education  field;  however,  we  encourage  the  interested 
reader  to  refer  also  to  the  case  examples  above.  We 
begin  with  a  participatory  design  project  of  5  years’ 
duration  which  involved  the  design  and  development 
of  a  network-based  collaborative  learning  system  for 
middle-school  physical  science  and  high-school  phys¬ 
ics.  The  purpose  of  this  example  is  to  (1)  illustrate 
how  participatory  design  was  carried  out  in  a  specific 
instance,  including  what  methods  were  used  and 
when,  and  (2)  consider  the  effectiveness  of,  efficiency 
of,  and  participants’  satisfaction  with  the  participatory 
design  methodology  used  as  well  as  to  consider  the 
challenges  encountered  during  the  project. 

Case  Studies 

Carroll  et  al.  (2000)  presented  an  example  of  how 
participatory  design  was  applied  in  the  design  of  a 
virtual  school  to  support  collaborative  learning  in  mid¬ 
dle-school  and  high-school  physical  science.  The  case 
provides  powerful  insights  into  the  transition  of  par¬ 
ticipants’  roles  over  the  course  of  the  project.  This 
5-year  project,  called  LiNC  (Learning  in  Networked 
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Communities),  began  as  a  small-scale  project  involv¬ 
ing  teachers  from  one  middle  school  and  one  high 
school  physics  class  and  was  supported  by  a  U.S. 
National  Science  Foundation  grant. 

The  main  players  in  the  LiNC  project  were  four 
middle  and  high  school  physics  teachers  and  eight 
university  research  team  members  (four  human-com¬ 
puter  interaction  specialists  and  four  computer  scien¬ 
tists).  The  project  was  a  partnership  between  Virginia 
Tech  University  and  the  public  schools  of  Montgomery 
County  in  Virginia  to  support  collaborative  science 
learning.  During  the  project,  physics  classes  were 
offered  every  other  year  to  very  small  classes  (three  to 
hve  students).  The  purpose  of  the  project  was  to  bring 
systemic  change  to  public  education  through  a  new 
computer  networking  infrastructure. 

The  project  team  observed  developmental  changes 
in  participant  teachers’  roles  as  the  project  progressed, 
beginning  with  “practitioner-informant”  and  trans¬ 
forming  along  the  way  to  “analyst,”  then  “designer,” 
and  finally  “coach.”  From  the  beginning,  the  university 
project  team  was  mindful  of  employing  participatory 
design  in  conceptualizing  the  project,  foreseeing  that 
the  teachers’  active  participation  must  be  continued 
even  after  the  project  ended  to  bring  the  sustainable 
systemic  change  to  public  education  that  the  project 
originally  set  forth  as  its  main  purpose. 

Although  this  project  resulted  in  an  enviable  level 
of  acceptance  and  use  for  the  designed  product,  it  is 
worth  noting  that  Carroll  et  al.  (2000)  questioned 
whether  it  had  to  take  5  years  to  work  effectively  with 
teachers.  In  their  view,  some  stages  of  the  project  could 
have  been  more  efficient— for  example  ,  by  assigning 
a  lead  teacher  or  by  helping  teachers  attain  prerequisite 
skills  in  design.  They  cautioned,  however,  that  com¬ 
pressing  the  timeline  for  such  a  project  would  “com¬ 
promise  the  coordination  of  participatory  and  ethno- 
graphically  driven  approaches  to  requirements 
development”  (Carroll  et  al.,  2000,  p.  248)  and  noted 
that  it  takes  time  to  build  the  trust  and  mutual  under¬ 
standing  required  to  carry  out  effective  design  work. 
Indeed,  participatory  design  is  a  philosophical  per¬ 
spective  rather  than  a  circumscribed  set  of  methods. 
Within  such  a  perspective,  the  inherent  value  of  user 
participation  and  the  presumed  benefits  resulting  from 
that  participation  are  held  to  be  of  greater  ultimate 
importance  than  the  efficiency  of  the  method. 

Perhaps  with  a  different  kind  of  preparation  them¬ 
selves,  the  trained  designers  on  such  a  project  could 
become  more  effective  at  facilitating  the  participation 
of  users  and  designers  in  such  a  project,  but  this  obser¬ 
vation  also  requires  us  to  step  back  from  the  case  and 
consider  what  is  necessary  for  such  a  shift  in  the  train¬ 
ing  of  designers.  If  the  inherent  worth  of  user  partici¬ 


pation  in  design  is  great  enough,  then  overhauling  the 
training  provided  to  designers  of  educational  systems 
might  be  seen  as  feasible. 

One  last  aspect  of  this  case  to  consider  is  that  the 
users/participants  appear  to  have  been  only  the  teach¬ 
ers  who  would  incorporate  the  system  into  their  class¬ 
rooms.  The  students,  who  would  presumably  also  be 
users  of  the  system,  were  not  included  as  participating 
designers,  although  they  may  have  been  included  sec¬ 
ondarily  as  part  of  the  very  small  classes  conducted 
during  the  development  of  the  system.  Although  a  case 
like  this  one  describes  a  potentially  effective,  albeit 
costly,  process  approach  for  bringing  about  change  in 
classroom  teaching,  it  is  important  to  discuss  seriously 
the  circumstances  in  which  it  is  possible  and  desirable 
to  apply  this  philosophy  and  the  methods  it  requires. 

A  further  example  of  participatory  design  is  the 
work  being  done  by  Reigeluth  and  Duffy  (2007)  in  the 
Decatur  Township  school  district.  The  participants 
include  school  teachers,  administrators,  students,  their 
parents,  and  community  members,  as  well  as  the 
design  leaders.  Although  this  is  an  effort  in  systemic 
change,  it  is  also  a  good  example  of  participatory 
design  in  which  the  stakeholders  play  major  roles 
throughout  the  process,  the  goal  of  which  is  the  real¬ 
ization  of  their  vision  regarding  what  they  want  their 
school  system  to  become.  This  process  will  occur  over 
several  years,  as  did  the  FiNC  project  described  earlier. 

CHARACTERIZATION  OF 
RAPID  PROTOTYPING 

Background  of  Rapid  Prototyping 

Rapid  prototyping,  a  methodology  used  in  software 
design  (and  also  in  fabrication  techniques  in  manufac¬ 
turing  via  CAD/CAM)  holds  potential  for  addressing 
many  of  the  limitations  of  the  conventional  ISD  model. 
Since  rapid  prototyping  was  introduced  as  a  design 
methodology  in  the  ISD  held  (Tripp  and  Bichelmeyer, 
1990),  conflicting  descriptions  of  how  rapid  prototyp¬ 
ing  applies  to  instructional  development  have 
appeared.  This  situation  has  resulted  in  an  inconsistent 
view  of  this  methodology  in  the  literature. 

Tessmer  (1994)  and  Northrup  (1995),  in  the  held 
of  instructional  technology,  argued  that  rapid  prototyp¬ 
ing  should  be  considered  as  an  alternative  method  of 
formative  evaluation  in  the  design  and  development 
phases.  This  is  consistent  with  the  role  of  prototyping 
described  in  many  studies  in  human-computer  inter¬ 
action  (HCI)  and  software  design.  Many  people  in  the 
held  of  instructional  technology,  however,  perceive 
rapid  prototyping  as  a  new  paradigm  of  instructional 
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design  methodology  (Dorsey  et  al.,  1997;  Jones  and 
Richey,  2000;  Rathbun  et  al.,  1997;  Tripp  and  Bich- 
elmeyer,  1990) .  In  this  chapter,  our  position  is  the  latter 
perspective,  which  views  rapid  prototyping  as  an  alter¬ 
native  to  the  conventional  ISD  process.  Note  that, 
when  rapid  prototyping  is  practiced  as  an  alternative 
to  traditional  ISD  processes,  it  can  also  be  character¬ 
ized  as  a  comparatively  weak  form  of  participatory 
design  (Kuhn  and  Winograd,  1996;  Willis  and  Wright, 
2000).  This  does  not  imply  that  the  rapid  prototyping 
process  is  weak,  but  rather  that  the  level  of  user  par¬ 
ticipation  in  RP  may  be  less  than  in  other  forms  of  PD. 

Customizations  of  rapid  prototyping  methods  to  fit 
the  instructional  design  field  have  been  based  on  two 
perspectives  on  design.  One  is  Simon’s  (1996)  theo¬ 
retical  view  that  “artificial  science”  differs  from  natural 
science.  Basically,  the  instructional  design  and  soft¬ 
ware  design  arenas  share  the  same  design  theory, 
which  holds  that  design  is  a  problem-solving  process 
that  uses  optimization  procedures.  The  other  perspec¬ 
tive  is  that  of  Schon  (1987),  who  viewed  the  design 
process  as  an  iterative  process  of  reflection  in  action. 
Design  plans  are  not  to  be  predetermined  so  as  to  lead 
to  a  predefined  goal,  but  should  instead  be  a  process 
that  deals  creatively  with  “uncertainty,  uniqueness,  and 
value  conflict”  (Schon,  1987,  p.  6). 

The  purpose  of  rapid  prototyping  is  to  demonstrate 
possibilities  quickly  by  building  an  inexpensive  series 
of  mock-ups  so  designers  are  able  to  obtain  early  feed¬ 
back  from  which  they  may  respond  to  user  require¬ 
ments  This  is  particularly  true  in  the  following  three 
types  of  situations:  (1)  cases  that  involve  complex  fac¬ 
tors,  which  can  make  predictions  difficult;  (2)  cases 
already  examined  by  conventional  methods  without 
satisfactory  results;  and  (3)  new  situations,  which  do 
not  offer  a  lot  of  experience  to  draw  from  (Tripp  and 
Bichelmeyer,  1990).  Thus,  rapid  prototyping  is  appro¬ 
priate  for  developing  electronic  performance  support 
systems  (Gery,  1995;  Gustafson  and  Branch,  1997; 
Gustafson  and  Reeves,  1990;  Law  et  al.,  1995;  Witt 
and  Wager,  1994),  conference  video  designs  (Appel- 
man  et  al.,  1995),  software  designs  (Dumas  and 
Redish,  1993;  Sugar  and  Boling,  1995),  and  computer- 
based  instruction  (Tripp  and  Bichelmeyer,  1990).  It  is 
also  useful  in  Web  design  (Boling  and  Frick,  1997; 
Corry  et  al.,  1997;  Frick  et  al.,  2005)  and  for  collabo¬ 
rative  learning  (Goodrum  et  al.,  1993;  Tessmer,  1994). 

As  proponents  of  rapid  prototyping  have  noted, 
however,  it  is  not  a  panacea  and  can  lead  to  an  undis¬ 
ciplined  design-by-repair  approach  that  ignores  initial 
analysis  and  planning.  Although  Sugar  and  Boling 
(1995)  described  conceptual  prototyping  for  nonexist¬ 
ent  technologies,  rapid  prototypes  cannot  easily  be 
used  to  develop  prototypes  for  many  common  instruc¬ 


tional  applications,  such  as  lectures,  workshops,  and 
televised  instruction  sessions,  because  the  prototyping 
effort  may  be  prohibitive  with  regard  to  both  time  and 
cost  (Tessmer,  1994;  Tripp  and  Bichelmeyer,  1990). 
Tripp  and  Bichelmeyer  (1990)  pointed  out  further  cau¬ 
tions  in  the  use  of  rapid  prototyping,  including  the  need 
for  tools  that  support  building  prototypes  efficiently, 
choice  of  optimal  methods  for  both  design  and  evalu¬ 
ation  of  prototypes,  and— most  importantly— knowl¬ 
edgeable  and  experienced  designers. 

Frick  et  al.  (2005)  added  important  front  and  back 
ends  to  the  rapid  prototyping  process.  Their  inquiry- 
based,  iterative  design  process  was  developed  and 
improved  through  formative  research  methods  and 
includes  needs  assessment  of  the  stakeholders,  rapid 
prototyping  on  paper  with  usability  testing,  further 
rapid  prototyping  on  computers  with  more  usability 
evaluation,  and  creating  and  maintaining  the  product 
designed  (Reigeluth  and  Frick,  1999,  p.  21).  Although 
their  focus  was  on  Web  design,  their  work  demonstrates 
that  more  than  rapid  prototyping  itself  is  needed  for 
designing  products  that  work  well  with  intended  users. 

Definition  of  Rapid  Prototyping 

As  Boling  and  Bichelmeyer  (1998)  have  noted,  rapid 
prototyping  has  been  used  in  many  different 
approaches  to  design  and  development.  Examples 
include  rapid  prototyping  (Tripp  and  Bichelmeyer, 
1990),  the  participatory  design  process  (Goodrum  et 
al.,  1993),  rapid  collaborative  prototyping  (Dorsey  et 
al.,  1997),  user-centered  design  (Corry  et  al.,  1997; 
Dumas  and  Redish,  1993;  Sugar  and  Boling,  1995), 
context-sensitive  design  (Tessmer  and  Wedman,  1995), 
and  ID2  transaction  shells  (Li  and  Merrill,  1990).  All 
of  these  include  a  rapid  series  of  iterative  tests  and 
revision  cycles,  coupled  with  the  direct  participation 
of  users  to  result  in  a  product  that  is  shaped  until  an 
acceptable  version  is  created  (see  Table  49.2). 

Even  though  these  various  approaches  share  the 
use  of  rapid  prototyping  methodologies,  the  definition 
of  what  a  prototype  is  differs  somewhat  from  one 
approach  to  another.  Tripp  and  Bichelmeyer  (1990) 
asserted  that  a  prototype  should  include  a  required 
database,  the  major  program  modules,  screen  displays, 
and  input  and  output  for  interfacing  systems.  This  def¬ 
inition  emphasizes  the  availability  of  computer  soft¬ 
ware  that  offers  modularity,  which  allows  for  flexibility 
in  adding,  removing,  or  modifying  a  segment  of  the 
instruction  without  introducing  severe  interactions  in 
the  other  segments.  Modularity  also  provides  plastic¬ 
ity,  which  refers  to  the  ability  to  change  aspects  of  a 
unit  of  instruction  with  only  minimal  time  and  cost 
(Tripp  and  Bichelmeyer,  1990,  p.  38). 
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TABLE  49.2 

A  Comparison  of  Instructional  Systems  Design  (ISD)  Approaches  That  Include  Rapid  Prototyping 


Tripp  and  Bichelmeyer 
(1990) 

Jones  et  al.  (1992) 

Dorsey  et  al.  (1997) 

Tessmer  and  Wedman  (1995) 

Model  name 

Rapid  prototyping 

ID2 

Rapid  collaborative  prototyping 

Context-sensitive  ID  model 

Meaning  of 

A  working  model  that 

An  incomplete  but 

Tangible  solution  ideas  that 

A  working  portion  of  the  final 

a  prototype 

includes  a  required  database, 
the  major  program  modules, 
screen  displays,  and  input 
and  output  for  interfacing 
systems 

essentially  executable 
version  of  the  final  product 

have  different  amounts  of 
fidelity 

product  that  is  immediately 
implemented  with  a  group  of 
learners  or  is  reviewed  by 
experts 

Processes 

Assess  needs  and  analyze 

content. 

Set  objectives. 

Construct  prototype. 

Utilize  prototype. 

Install  and  maintain  system. 

Analyze  knowledge. 

Analyze  audience  and 
environment. 

Analyze  strategies. 

Specify  transaction 
configurations. 

Develop  transaction  details. 
Implement. 

Create  visions. 

Explore  conceptual  prototypes. 
Experiment  with  mock-ups. 

Pilot  test  working  prototypes. 
Implement  product. 

Analyze  layers. 

Specify  instructional  scenarios. 
Develop  alternative  prototypes. 
Negotiate  prototype. 

ISD 

New  paradigm  of  ISD  process 
model 

Large-component  prototype 
approach 

Co-ownership  of  designers 
and  users 

New  form  of  ISD 

Jones  et  al.  (1992)  argued  that  a  prototype  is  an 
incomplete  but  essentially  executable  version  of  the 
final  product.  Tessmer  and  Wedman  (1995)  defined  a 
prototype  as  a  working  portion  of  the  final  product  that 
is  immediately  implemented  with  a  group  of  learners 
or  is  reviewed  by  experts.  Both  definitions  emphasize 
the  aspect  of  a  quick,  working  version  of  a  final  prod¬ 
uct;  therefore,  a  prototype  does  not  have  to  include 
everything  that  the  final  version  will  contain.  Finally, 
Dorsey  et  al.  (1997)  and  Sugar  and  Boling  (1995) 
viewed  a  prototype  as  a  tangible  idea  of  possible  solu¬ 
tions  that  have  different  range  of  fidelity  from  low  to 
high.  Their  definition  is  very  different  from  the  others, 
in  that  even  a  conceptual  version  of  a  solution  could 
be  a  prototype,  and  it  is  closest  to  that  used  in  the 
software  design  community  (Rudd  et  al.,  1996). 

CHALLENGING  ISSUES 

Designers  face  many  challenging  issues  when  attempt¬ 
ing  to  implement  UCDD,  such  as  effective  incorporation 
of  user  participation  in  the  design  process,  control  issues 
over  resources,  and  the  practical  implementation  of  the 
approach  utilized.  These  issues  are  discussed  below. 

Issue  1.  Effective  Incorporation 
of  User  Participation 

One  of  the  most  difficult  challenges  of  UCDD  is  the 
effective  incorporation  of  user  participation  in  the 
design  process.  Determination  of  which  voices  will  be 


heard  and  how  the  users’  preferences  will  be  reflected 
in  the  design  is  a  values-based  decision  and  is  rarely 
easy.  This  is  especially  true  in  large-sized  commercial 
projects  targeted  at  a  range  of  users  from  different 
backgrounds  and  settings.  Along  with  the  issue  of  who 
gets  to  participate  comes  the  issue  of  how  to  recruit 
users  who  will  represent  the  potential  target  user 
groups  appropriately  when  those  groups  are  very  large 
or  very  diverse.  In  addition,  when  user  participation  is 
limited  only  to  a  certain  stage,  the  users’  role  will  end 
up  being  that  of  information  providers  rather  than 
codesigners  of  the  project. 

Even  when  the  goal  of  UCDD  is  to  place  users  at 
the  center  of  the  design  process,  in  many  situations  the 
negotiation  between  the  “designed  for”  approach,  in 
which  the  designers  assume  leadership  in  the  design 
process,  and  the  “designed  with”  approach,  in  which 
the  users  assume  ownership  in  the  process,  can  become 
both  a  philosophical  and  a  practical  consideration. 
How  much  user  participation  is  too  much?  Even  for 
designers  who  consider  active  user  participation 
throughout  the  entire  process  to  be  the  ideal,  some 
researchers  have  encountered  practical  difficulties  with 
the  process.  In  a  participatory  design  project  intended 
to  build  a  community  learning  network  using  open 
source  tools,  Luke  et  al.  (2004)  observed  that  early 
group  brainstorming  heightened  the  users’  expectations 
and  demands,  and  these  demands  were  furthermore 
unmoderated  by  any  realistic  conceptions  of  the  time 
and  costs  they  would  require.  When  the  first  prototype 
was  released  (past  its  due  date),  these  same  heightened 
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expectations  turned  into  general  disappointment.  The 
authors  attributed  this  problem  to  both  the  designers 
and  the  users,  who  were  “too  participatory  and  too 
open”  (Luke  et  al.,  2004,  p.  1 1).  They  then  warned  that 
user  participation  in  the  early  stages  of  a  project  can 
be  disadvantageous  if  it  is  not  balanced  with  realistic 
constraints.  Processes  can  be  developed  to  ameliorate 
or  eliminate  these  kinds  of  problems,  but  the  potential 
for  them  to  arise  remains. 

Some  researchers,  however,  have  been  able  to  elicit 
user  participation  positively,  even  in  long-term,  large- 
scale  PD  projects.  Letondal  and  Mackay  (2004)  con¬ 
ducted  participatory  design  activities  with  research 
biologists,  bio-informaticians,  and  programmers  at  the 
Institut  Pasteur  in  Paris  over  a  period  of  7  years.  The 
focus  of  their  project  was  the  development  of  tools  to 
support  end-user  programming.  They  did  observe  some 
tensions  between  different  groups  of  participants;  how¬ 
ever,  overall  the  participatory  design  worked  in  that 
context.  The  main  reason  for  this  success  was  attributed 
by  the  authors  to  maintaining  a  balance  between  “low- 
responsibility”  and  “useful  results”  (Letondal  and 
Mackay,  2004,  p.  39).  Again,  it  appears  that  designs 
committed  to  the  UCDD  perspective  require  experi¬ 
ence  and  skill  to  carry  it  out  effectively. 

Issue  2.  Control  over  Resources: 

Money,  Time,  Tools,  and  Space 

Another  challenging  issue  in  UCDD  is  acquiring  and 
maintaining  control  over  enough  resources  to  support 
a  project— money,  time,  tools,  and  space.  Acquiring 
and  allocating  these  resources  can  cause  a  great  deal 
of  tension.  Even  after  full  members  in  the  design  team 
have  been  identified,  the  question  still  remains:  How 
can  the  team  elicit  full  user  participation  when  the 
users  may  also  have  to  fulfill  their  own  full-time  job 
duties?  In  their  review  of  ten  different  participatory 
design  projects,  Clement  and  Van  den  Besselaar  (1993) 
observed  that,  although  some  projects  provided  funds 
for  the  users  to  hire  temporary  staff  to  take  their  places 
while  they  were  working  with  the  design  team,  users 
in  other  projects  had  to  perform  their  regular  job  duties 
while  concurrently  contributing  to  the  project.  Design 
teams  seeking  only  intermittent  and  short-term 
involvement  from  users  may  face  lower  barriers  but 
may  still  have  trouble  recruiting  users  who  can  afford 
to  take  the  time  necessary  to  participate  in  the  test  of 
a  prototype— or  repeating  such  sessions. 

Sugar  (2001)  pointed  out  that  one  of  the  common 
misconceptions  among  researchers  in  UCDD  is  that 
designers  should  relinquish  all  of  their  authority  and 
allow  the  participating  users  to  make  all  of  the  design 
decisions.  He  warned  that  users  are  not  expert  design¬ 


ers,  and  designers  should  not  expect  users  always  to 
know  exactly  what  they  want  to  use.  He  pointed  out 
that  they  may  not  be  right  all  the  time,  either,  and  that 
even  though  users’  opinions  must  be  respected  design¬ 
ers  need  to  present  the  possibilities  and  limitations  of 
proposed  solutions  properly.  Although  this  is  true  for 
any  design  project.  Sugar  claimed  that  the  governing 
responsibility  of  designers  is  certainly  crucial  in 
UCDD.  To  implement  this  approach  effectively, 
designers  must  also  delve  beyond  the  surface  of  these 
issues  and  carefully  consider  each  of  them  by  means 
of  in-depth  analyses  (Sugar,  2001).  Carr-Chellman 
(2007)  has  offered  multiple  suggestions  for  carrying 
out  user  design  activities  in  which  users  function  as  the 
primary  designers  and  trained  designers  as  facilitators 
but  also  pointed  out  that  the  process  can  be  very  diffi¬ 
cult  and  is  not  suitable  for  every  context  or  situation. 

Raskin  (2000),  who  played  a  major  role  in  the 
design  of  the  Macintosh  computer  interface  at  Apple 
Computer,  emphasized  that  what  users  prefer  in  a 
design  is  not  necessarily  what  is  most  efficient  and 
effective.  He  cited  several  empirical  studies  where 
users  actually  performed  more  poorly  with  interface 
designs  they  preferred  than  they  did  with  others  they 
did  not  prefer.  This  illustrates  the  tension  between  what 
users  want  compared  with  what  is  best  for  them  based 
on  scientifically  proven  principles,  similar  to  the  prob¬ 
lem  with  what  people  prefer  to  eat  vs.  what  is  good 
for  them  in  terms  of  nutritional  value  and  their  long¬ 
term  health. 


DISCUSSION 

When  designers  select  a  design  approach,  their  choice 
is  influenced  by  their  philosophies  (Visscher-Voerman 
and  Gustafson,  2004).  In  their  attempt  to  understand 
how  designers  carry  out  instructional  design  projects 
in  reality,  Visscher-Voerman  and  Gustafson  (2004) 
found  that  all  12  of  their  examined  designers  (from  6 
different  settings  in  the  initial  study)  integrated  a  tra¬ 
ditional  ISD  model  into  their  work.  The  ways  in  which 
they  incorporated  this  model,  however,  were  diverse 
and  varied.  In  their  second  study,  Visscher-Voerman 
and  Gustafson  developed  four  alternative  design  para¬ 
digms  (or  conceptual  frameworks)  that  are  anchored  in 
philosophy:  instrumental,  communicative,  pragmatic, 
and  artistic.  Table  49.3  shows  the  characteristics  of  each 
of  these  paradigms.  In  general,  the  UCDD  approach 
seems  to  be  related  to  communicative  and  pragmatic 
paradigms  in  the  sense  that  UCDD  puts  an  emphasis 
on  users  as  codevelopers  in  the  design  process.  This  is 
achieved  by  means  of  the  nonlinear  and  iterative  anal¬ 
ysis/design/evaluation  format  of  cooperation.  Rapid 
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TABLE  49.3 

Four  Alternative  Design  Paradigms 


Communicative 

Pragmatic 

Artistic 

Instrumental  Paradigm 

Paradigm 

Paradigm 

Paradigm 

Emphasis 

Aligned  goals,  learning 

Communication  between 

Repeated  testing  and  revision 

Creative  design 

situations,  process,  and 

designers  and  users  to  reach 

outcome  of  the  design 

consensus 

Designer’s  role 

Expert  (responsible  for 

Facilitator  (shares  responsibility 

Expert  (responsible  for  design) 

Artist  (fully  responsible 

design) 

with  users) 

for  design) 

User’s  role 

Information  provider  and 

Information  provider;  codesigner 

Information  provider;  product  user 

Product  user 

approval  for  action 

Design  process 

Typically  linear 

Nonlinear  and  iterative 

Nonlinear  and  iterative 

Linear  or  nonlinear 

Sources:  Adapted  from  Visscher-Voerman,  I.  and  Gustafson,  K.L.,  Educ.  Technol.  Res.  Technol. ,  52(2),  86,  2004. 


prototyping,  when  used  as  the  cornerstone  of  an  alter¬ 
native  ISD  model,  may  be  closer  in  philosophy  to  the 
pragmatic  paradigm.  In  either  case,  UCDD,  in  both 
strong  and  weak  forms,  represents  some  shift  in  phi¬ 
losophy  for  instructional  designers  who  employ  it.  To 
the  extent  that  UCDD  gathers  momentum  in  the  teach¬ 
ing  and  practice  of  instructional  design,  we  can  expect 
to  see  changes  in  logistics,  methods,  and  power 
dynamics  in  design  projects  within  this  held. 
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ABSTRACT 

Today’s  instructional  designers  can  choose  from  a  wide 
variety  of  authoring  and  online  learning  development 
tools.  With  the  evolution  of  learning  technology,  expec¬ 
tations  from  outside  the  learning  and  training  discipline 
are  looking  to  the  technology  to  provide  instructional 
design  guidance  and  allow  novice  developers  to  create 


instructionally  sound  online  learning.  This  chapter 
explores  the  tools  that  instructional  designers  and  nov¬ 
ice  developers  use  most  often  to  create  online  learning 
from  both  a  design  and  development  perspective,  the 
types  of  instructional  design  support  that  can  be  found 
in  different  types  of  development  tools,  and  how  these 
tools  may  (or  may  not)  impact  the  ability  to  meet 
instructional  needs.  A  classification  system  is  used  to 
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analyze  tools  as  (1)  standard  authoring  tools,  (2)  front- 
end  design  and  automated  instructional  design  (AID) 
tools,  (3)  simulation  and  gaming  tools,  or  (4)  team 
development  or  publishing  tools  such  as  learning  con¬ 
tent  management  systems  (LCMSs)  and  groupware 
authoring.  Although  instructional  design  guidance,  in 
some  form  or  another,  is  now  embedded  inside  several 
classifications  of  development  tools,  considerable 
innovation  and  research  are  still  required  to  create 
holistic  systems  that  negate  or  minimize  the  need  for 
instructional  design  expertise,  allowing  nontrained 
designers  to  create  online  learning  courseware  without 
assistance. 


KEYWORDS 

Asynchronous  learning:  Communication  between 
learners  and  instructors  that  does  not  take  place 
simultaneously  or  real  time.  An  example  of  asyn¬ 
chronous  learning  is  when  a  learner  engages  in  a 
self-paced,  self-service  learning  module  without 
communicating  with  another  person. 

Authoring  tool:  A  software  application  used  by  non¬ 
programmers  to  assemble  digital  media  hies  into 
displays,  presentations,  and  interactive  exercises. 

Automated  instructional  design  (AID):  Leveraging 
technology  to  automate  the  instructional  design  as 
part  of  the  process  of  learning  content  develop¬ 
ment. 

Dynamic  pretest:  A  test  delivered  online  that  can  lit¬ 
erally  adapt  courseware  to  learner  deficiencies 
identified  through  online  testing  and  scale 
courseware  to  match  specific  learner  needs. 

Groupware  authoring:  Concurrent  process  of  creating 
learning  content  in  a  team  environment  with  mul¬ 
tiple  authors  who  often  play  different  roles,  such  as 
writers,  graphic  artists,  or  instructional  designers. 

Learning  content  management  system  (LCMS):  A 
multi-user  software  application  in  which  learning 
developers  can  create,  store,  reuse,  manage,  and 
deliver  digital  learning  content  from  a  central 
object  repository  (database). 

Novice  developer:  A  person  who  has  little  or  no  train¬ 
ing  in  instructional  design  yet  has  the  responsibility 
of  creating  learning  content  using  authoring  tools. 

Soft-skills  simulation:  Simulations  designed  for  the 
specific  purpose  of  teaching  interpersonal  skills 
such  as  leadership,  coaching,  facilitating,  and  so  on. 

Storyboard:  A  document  that  details  and  specifies  on¬ 
screen  text,  narrative  scripts,  and  interaction  in  a 
paper-based  format  before  it  is  converted  into  an 
online  course. 


INTRODUCTION 

What  if  William  Shakespeare  would  have  had  access 
to  modem  word-processing  software?  Would  it  have 
made  him  a  better  writer?  This  answer  is  likely  an 
emphatic  no.  Taking  the  analogy  one  step  further,  what 
if  the  vendor  who  produced  the  word-processing  soft¬ 
ware  added  an  embedded  wizard  to  assist  Shakespeare 
in  creating  sonnets  with  perfect  iambic  pentameter? 
How  much  impact  might  this  wizard  have  had  on 
improving  the  quality  and  quantity  of  his  work? 
Finally,  what  if  a  vendor  touted  new  software  that 
encapsulates  the  genius  of  Shakespeare  such  that  any¬ 
one  (even  novice  writers)  who  uses  it  could  write  clas¬ 
sic  material  with  the  same  style  and  effectiveness  of 
Shakespeare  himself?  Like  most  people,  you  would 
likely  be  highly  skeptical. 

Seems  like  a  series  of  ridiculous  scenarios;  yet, 
believe  it  or  not,  similar  arguments  still  rage  on  about 
the  role  of  software  tools  used  by  instructional  design¬ 
ers  to  design  and  develop  online  learning  courseware. 
In  addition,  many  organizations  have  heightened 
expectations  that  relatively  novice,  nontechnical  devel¬ 
opers,  and  even  subject-matter  experts  (SMEs)  should 
be  able  to  use  such  tools  to  create  significant  amounts 
of  both  traditional  and  online  learning.  In  many  cases, 
the  developers  have  little  or  no  instructional  design 
training  (Nantel  and  Vipond,  2006),  relying  instead  on 
design  and  development  tools  to  provide  guidance 
throughout  the  process. 

Software  tools  unquestionably  play  a  valuable 
role  in  the  process  of  instructional  development,  but 
what  levels  of  instructional  design  support  can  we 
expect  to  come  from  the  tools,  and  what  part  must 
still  come  from  those  who  use  the  tools?  The  purpose 
of  this  chapter  is  to  explore  what  tools  instructional 
designers  and  novice  developers  most  often  use  to 
create  online  learning  from  both  design  and  devel¬ 
opment  perspectives,  what  types  of  instructional 
design  support  can  be  found  in  different  types  of 
development  tools,  and  how  these  tools  may  (or  may 
not)  impact  the  ability  to  meet  instructional  needs. 
This  chapter  focuses  on  four  different  tool  classifi¬ 
cations; 

•  Standard  authoring  tools 

•  Front-end  design  and  automated  instruc¬ 
tional  design  (AID)  tools 

•  Learning-activity,  focused  design  tools ,  such 
as  simulations  and  games 

•  Team  development/publishing  tools,  such  as 
learning  content  management  systems 
(LCMSs)  and  groupware  authoring  tools 


672 


Tools  for  Design  and  Development  of  Online  Instruction 


Figure  50.1  Percentage  of  instructional  design  job  postings  requiring  authoring  tool  experience. 


STANDARD  AUTHORING  TOOLS 

Times  have  changed  for  instructional  designers  who 
work  on  online  learning  courses.  In  the  early  days  of 
computer-based  training  (CBT)  and  through  the  early 
transition  to  Web-based  delivery  of  asynchronous,  self- 
paced  learning,  instructional  development  teams  were 
most  often  organized  with  clear  delineation  between 
those  who  designed  the  instruction  and  those  who  pro¬ 
grammed  or  authored  course  material.  Instructional 
designers  were  responsible,  primarily,  for  creating  the 
specifications,  such  as  design  documents,  scripts,  sto¬ 
ryboards,  etc.,  while  authoring  specialist  and  program¬ 
mers  focused  almost  exclusively  on  assembling  online 
learning  materials  to  match  the  specifications.  Over 
time,  however,  the  lines  between  these  roles  continued 
to  blur,  with  increased  pressure  on  instructional 
designers  to  assume  a  more  active  role  in  the  authoring 
stages  of  development. 

To  illustrate  current  demand  for  required  author¬ 
ing  skills,  a  random  sampling  of  instructional  design 
job  postings  was  taken  from  monster.com,  Yahoo! 
hotjobs.com,  and  CareerBuilder.com  from  April  to 
May  2006.  Of  the  postings  studied,  68%  specifically 
listed  the  ability  to  use  authoring  tools  as  a  require¬ 
ment  for  the  job ,  in  most  cases  naming  specific  author¬ 
ing  tools,  such  as  Adobe  Flash,  Dreamweaver, 
Authorware,  and  ToolBook  (see  Figure  50.1).  The 
postings  came  from  a  wide  array  of  organizations, 
both  large  and  small,  including  The  Johns  Hopkins 
University,  Becton  Dickinson,  OfficeMax,  Fidelity 
Investments,  Walgreen’s,  Sears,  The  Hartford,  Merck, 
and  the  Federal  Reserve  Bank.  In  another  study,  orga¬ 
nizations  that  create  custom  courseware  for  internal 
training  reported  on  the  tools  they  use  to  create  self- 
paced  learning  materials.  Table  50.1  shows  the  top  ten 
authoring  tools  used  most  frequently,  listed  in  order 
of  the  highest  percentage  of  use  (Chapman,  2006b). 
Many  of  these  tools  have  existed  in  one  form  or 


another  for  over  a  decade,  including  Flash,  Dream¬ 
weaver,  Authorware,  Director,  Lectora,  ToolBook, 
and  Quest.  Standard  authoring  tools  really  have  not 
changed  much  since  the  early  days  of  technology- 
based  training  (Foshay  and  Preese,  2005),  even  pre¬ 
dating  the  movement  from  hard-drive-delivered  and 
CD-ROM  courses  to  Web-based  delivery. 

Authoring  tools  were  originally  designed  for  the 
purpose  of  allowing  nonprogrammers  to  easily  assem¬ 
ble  media  objects  and  preconstructed  scripting  code  to 
build  instructional  learning  applications.  Most  stan¬ 
dard  authoring  tools  use  the  metaphor  of  a  blank  page, 
allowing  authors  (users  of  authoring  tools)  as  much 
flexibility  as  possible  to  create  visually  appealing  lay¬ 
outs  and  designs  while  also  providing  a  scripting  lan¬ 
guage  for  further  extensibility  in  the  creation  of  com¬ 
plex  interactions.  For  the  most  part,  authoring  tools 
operated  as  advertised,  although  they  still  often 
required  a  steep  learning  curve  in  order  to  take  advan¬ 
tage  of  their  features  (Merrill,  1997).  In  many  ways. 


TABLE  50.1 

Most  Frequently  Used  Standard  Authoring 
Tools  for  Online  Courseware  Development 


Authoring  Tool 

Vendor 

Companies 
Using  Tool 

(%) 

Flash 

Adobe 

92 

Dreamweaver 

Adobe 

67 

Authorware 

Adobe 

34 

Director 

Adobe 

27 

Lectora  Publisher 

Trivantis 

21 

Captivate 

Adobe 

18 

ToolBook 

SumTotal 

18 

Breeze 

Adobe 

6 

Total  LCMS 

SumTotal 

4 

Quest 

Allen  Communication 

4 

673 


Bryan  L.  Chapman 


-Ifll  XI 


Figure  50.2  CourseBuilder  for  Dreamweaver  template  wizard  for  creating  a  drag-and-drop  exercise. 


constructing  courses  with  just  an  authoring  tool  is 
much  like  building  an  entire  house  stick  by  stick. 
Because  of  the  open  nature  of  authoring  tools  and  the 
blank  page  metaphor,  originally  there  was  little  or  no 
instructional  design  guidance  to  assist  novice  develop¬ 
ers  in  creating  instructionally  sound  courseware. 

The  next  evolutionary  stage  of  authoring  tools  was 
the  introduction  of  templates,  designed  for  two  primary 
purposes:  (1)  to  facilitate  rapid  development  of  course 
content,  and  (2)  to  provide  additional  instructional 
design  guidance  for  novice  developers.  As  an  example 
of  the  types  of  templates  found  in  authoring  tools,  a 
set  of  templates  named  CourseBuilder  was  created  as 
an  add-on  for  Macromedia’s  Dreamweaver,  which  was 
initially  designed  for  standard  Webpage  development 
and  not  for  the  creation  of  learning  material.  The  learn¬ 
ing  templates  are  accessed  from  the  menu  bar  as  a 
wizard  to  walk  novice  developers  through  the  process 
of  creating  complex  learning  interactions.  The  tem¬ 
plates  can  be  used  to  create  a  variety  of  test  question 
formats,  discovery  exercises,  and  sophisticated  drag- 
and-drop  exercises  (see  Figure  50.2).  The  templates 
have  built-in,  automatic  controls  to  record  scoring 
information,  answer  judging,  feedback,  and  remedia¬ 
tion  branching  based  on  learner  selection. 

Templates  also  serve  another  valuable  role  in  pro¬ 
viding  examples  of  good  design  and  ideas  for  different 
types  of  instructional  interventions;  for  example. 


Articulate  Presenter  is  a  popular  tool  for  converting 
PowerPoint  presentations  into  e-learning  courses.  To 
expand  on  its  core  technology.  Articulate  released  the 
Rapid  E-Learning  PowerPoint  Template  Kit,  which 
includes  dozens  of  professionally  designed  templates, 
including  screen-layout  templates  and,  even  more 
importantly,  examples  of  how  the  template  may  be 
applied  to  instruction  (see  Figure  50.3). 

Although  templates  provide  much  needed  addi¬ 
tional  design  guidance  to  authors,  desktop  authoring 
tools  still  lack  the  core  functionality  to  fully  ensure 
adherence  to  sound  instructional  design  principles, 
especially  in  the  early  stages  of  front-end  analysis, 
objective  writing,  and  designing  appropriate  interac¬ 
tions  across  an  entire  course.  Bell  (1998)  concluded 
that  authoring  tools  support  many  possible  instruc¬ 
tional  applications,  some  of  which  may  be  well  exe¬ 
cuted  and  some  poorly  executed,  but  none  will  be 
created  with  very  much  guidance  from  the  tool. 

Authoring  tools,  however,  do  play  an  important 
role  in  the  overall  instructional  development  process, 
as  evidenced  by  the  demand  for  authoring  tool  expe¬ 
rience  among  posted  instructional  design  job  descrip¬ 
tions.  Instructional  designers,  who  also  may  use 
authoring  tools,  must  apply  their  own  design  expertise 
in  the  early  stages  of  instructional  development,  then 
use  the  authoring  tools  to  carry  out  the  design  accord¬ 
ing  to  specifications. 
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□  Example:  Classifying  Igneous  Rock 

Igneous  rocks  are  classified  by  the  chemical 
constituents  and  texture 


Granites 


Felsic  Rocks 

Rhyolite 
Andesite 


Obsidian 


Diorite 


Coarse¬ 

grained 


Gabbro 


Fine-grained 


Basalt 


Peridotite 


Komatiite 


Mafic  Rocks 


Template  with  Content 


Figure  50.3  Articulate  instructional  design  template  used  for  providing  classification  examples  across  multiple  domains. 


Figure  50.4  Typical  level  of  instructional  design  support  offered  by  standard  authoring  tools. 


Figure  50.4  illustrates  the  level  of  instructional 
design  support  one  might  expect  from  standard  author¬ 
ing  tools.  Most  authoring  tools  provide  little  to  no  guid¬ 
ance  in  the  areas  of  needs  analysis,  objective  writing, 
high-level  (course)  design,  script/storyboarding,  and 
instructional  design  guidance;  instead,  they  are  more 
appropriately  used  during  the  development  phases  to 
create  interaction,  which  provides  feedback  and  oppor¬ 
tunities  for  learners  to  control  their  own  learning  expe¬ 
rience  (learner  control),  rapid  development  of  online 
course  materials  (as  opposed  to  programming  from 


scratch),  and  support  for  the  creation  of  automated 
assessment.  By  understanding  that  standard  authoring 
tools  provide  limited  instructional  design  guidance, 
organizations  can  correctly  position  their  use  appropri¬ 
ately  in  the  broader  context  of  instructional  develop¬ 
ment  process  by  applying  other  methods  and  resources 
to  the  front-end  design  work.  This  discussion  also  sug¬ 
gests  that  standard  authoring  tools  may  not  be  best 
suited  for  novice  development  of  learning  content  when 
expectations  exist  that  the  tool  will  provide  guidance 
and  structure  through  development. 
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Needs  Analysis  |  □  Full  Support  ■  AID 


Figure  50.5  Coverage  of  design  guidance  supported  by  automated  instructional  design  tools  (AIDs). 


FRONT-END  DESIGN  AND  AUTOMATED 
INSTRUCTIONAL  DESIGN  TOOLS 

Approaching  the  design  and  development  task  from 
the  opposite  end  of  the  spectrum  is  a  set  of  specialized 
software  applications  that  provide  guidance  through 
the  front-end  of  instructional  development.  None  of 
these  tools  has  achieved  the  same  mass  adoption  rate 
experienced  by  standard  authoring  tools.  Some  were 
initially  created  as  experimental  projects,  some  as  in- 
house  development  toolsets  supporting  custom  devel¬ 
opment  of  CBT  and  online  courseware,  and  still  others 
simply  as  performance  support  tools  designed  to  fill 
in  the  gap  left  by  standard  authoring  tools.  These  appli¬ 
cations  also  range  significantly  in  scope  of  function¬ 
ality  and  purpose.  Some  are  designed  to  create  high- 
level  instructional  design  documents,  while  others 
span  the  entire  development  process  from  front-end 
design  through  the  production  phases  of  development. 
Holistic  systems  that  also  provide  production  of 
courseware  are  often  referred  to  as  automated  instruc¬ 
tional  design  (AID)  tools.  This  moniker  is  what  sets 
them  apart  from  other  design  tools  (Gros  and  Spector, 
1994).  As  a  classification  of  design  and  development 
tools,  this  category  provides  the  most  instructional 
design  guidance  targeting  development  by  novice 
designers  and  subject  matter  experts  and  others  respon¬ 
sible  for  developing  instruction  (Muraida  and  Spector, 
1993).  Information  about  the  frequency  of  use  of  front- 
end  design  and  AID  solutions  is  difficult  to  obtain, 
because  most  studies  of  instructional  design  and  devel¬ 


opment  tools  focus  on  commercial  products.  Table 
50.2  identifies  some  of  the  most  high-profile  projects 
and  solutions  in  this  area.  The  common  thread  among 
these  tools  is  that  they  all  serve  a  useful  purpose  for 
environments  where  instructional  design  expertise 
may  be  lacking  or  in  short  supply  and  for  situations 
where  subject  matter  experts  and  others  are  primarily 
responsible  for  developing  instruction  (Muraida  and 
Spector,  1993).  Figure  50.5  illustrates  where  front-end 
design  and  AID  solutions  provide  the  most  support  for 
instructional  development,  with  strong  emphasis  on 
needs  analysis,  objective  writing,  high-level  design, 
script/storyboarding,  and  ISD  guidance. 

A  simple  subclassification  of  front-end  design  and 
AID  tools  was  presented  by  Kasowitz  (1998):  ( 1 )  those 
systems  that  focus  on  cognitive  aspects  of  instructional 
design— in  other  words,  systems  that  prescribe  instruc¬ 
tional  interventions  based  on  best  practice  theories 
(e.g.,  IDExpert,  ADG),  and  (2)  those  systems  that  pro¬ 
vide  support  and  guidance  for  the  procedural  steps  of 
instructional  design  (e.g.,  AIM  II,  The  Designer’s 
Edge,  GAIDA,  DesignWare,  Coursewriter).  As  an 
example,  IDExpert  (ID2  Research  Group,  led  by  Dr. 
M.  David  Merrill)  was  based  on  instructional  transac¬ 
tion  theory  (Cline  and  Merrill,  1995).  Briefly,  IDEx¬ 
pert  included  the  intelligence  to  bring  together  ele¬ 
ments  of  knowledge  representation,  best  practice 
instructional  strategies,  and  instructional  design  pre¬ 
scriptions,  automatically  producing  all  of  the  necessary 
interactions  for  a  student  to  acquire  a  particular  knowl¬ 
edge  or  skill  (Merrill,  1999).  IDExpert  users  followed 


676 


Tools  for  Design  and  Development  of  Online  Instruction 


TABLE  50.2 

Some  of  the  Most  Recognized  Front-End  Design  and  Automated  Instructional  Design  Tools 


Design  or  AID  Tool 
AIM  II 

The  Designer’s  Edge  (Allen  Communication) 
GAIDA 


IDExpert 

ADG  (acronym  for  “didactic  engineering 
workbench”  in  French) 

DesignWare  (Langevin) 

CourseWriter  (Darryl  Sink  and  Associates) 


Reason  Created 

Created  for  the  Navy  to  promote  uniform  instructional  design  practices  and  as  a  repository  for 
course  design  specifications 

Created  as  a  commercial  application  focusing  on  front-end  analysis,  objective  writing,  design 
specifications,  story  boarding,  and  evaluation 

Created  by  Armstrong  Laboratories  at  Brooks  Air  Force  Base  for  the  purpose  of  providing 
instructional  design  guidance  and  examples  during  the  front  end  of  instructional  development; 
based  on  Gagne’s  nine  events  of  instruction 

Created  at  Utah  State  University  through  grants  and  sponsorships  to  put  into  practice  the  theory 
of  automated  instructional  design 

Created  as  joint  project  of  research  and  academic  institutions  as  an  intelligent  support  system 
for  course  design 

Created  as  a  commercial,  support  tool  to  accompany  Langevin’s  train-the-trainer  workshops; 
focuses  on  guidance  and  organization  course  design  specifications 

Created  as  a  commercial  tool  focused  on,  for  example,  front-end  design,  objective  writing,  and 
course  organization 


steps  provided  by  the  system  to  enter  appropriate  con¬ 
tent,  and  the  system  automatically  generated  and  deliv¬ 
ered  the  appropriate  interactions  until  mastery  was 
achieved  by  learners. 

By  contrast.  The  Designer’s  Edge  (Allen  Com¬ 
munication)  uses  a  performance  support  approach  by 
emulating  the  analyze,  design,  develop,  implement. 


and  evaluate  (ADDIE)  model  of  instructional  design 
as  its  primary  interface  (see  Figure  50.6).  Instead  of 
prescribing  specific  design  methodology,  the  system 
walks  novice  and  expert  designers  through  the  com¬ 
mon  steps  of  instructional  design,  such  as  conducting 
a  needs  analysis,  creating  an  audience  profile,  using 
an  embedded  wizard  to  create  appropriately  formatted 


Radiology .des  -  Designer's  Edge 


File  Edit  View  Options  Help 

D  |  &  |  y  |  | Analyze  Needs 


dSld 


~3  n|a|  »l*l .  Ivifill 


Rif 

Analyze  & 

Outline 
Content 

P  Completed  P  Completed  P  Completed  P  Completed  P  Completed  P  Completed 


r  Completed  |  Completed  P  Completed  P  Completed  P  Completed  P  Completed 


d 


IT)  Foims 


/■  Edit 


y  Edil 


J  Reporl 


Edil 


r  Gather  Information  and  Collect  Data 
P  Identity  Needs 

P  Write  and  Prioritize  Instructional  Goals 
P  Write  Needs  Analysis  Report 
P  Gain  Approval 


Ready 


Figure  50.6  The  Designer’s  Edge  interface  based  on  the  standard  ADDIE  model. 
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performance  objectives,  creating  a  master  course 
map,  and  providing  extensive  tools  for  creating 
scripts  and  storyboards  to  be  used  by  multimedia 
programmers  when  creating  online  learning. 

To  provide  guidance  beyond  the  ADDIE  model 
interface,  the  system  also  serves  up  context-sensi¬ 
tive  instructional  design  advice  and  guidance  from 
leaders  in  the  field,  linked  to  tools  that  can  be  used 
to  complete  the  task.  Although  Allen  Communica¬ 
tion  has  added  tools  to  transmit  the  learning  content 
from  the  storyboarding  tool  directly  into  standard 
authoring  tools  such  as  Authorware,  ToolBook,  and 
Dreamweaver,  the  primary  focus  of  the  system  is 
on  the  front-end  design  tasks  of  instructional  devel¬ 
opment. 

Among  the  examples  of  successful  front-end 
design  developed  by  utilizing  AID  tools  is  GAIDA 
(Guided  Approach  to  Instructional  Design  Advising), 
based  on  Gagne’s  nine  events  of  instruction  (Gagne, 
1991).  GAIDA  was  put  through  a  series  of  tests 
where  novices  created  course  designs  with  only  guid¬ 
ance  from  the  support  tool.  The  results  concluded 
that  the  tool  was  “generally  effective”  in  creating 
instructional  sound  learning  content  (Spector  and 
Whitehead,  1994).  The  theory  behind  the  tool, 
according  to  Gagne  himself,  was  that  it  would  “pro¬ 
vide  novices  with  easily  understood,  high-level  guid¬ 
ance  along  with  completely  worked  examples,  and 
they  will  perform  as  if  they  were  advanced  appren¬ 
tices”  (Gagne,  1991).  The  tool  is  still  available  for  a 
nominal  fee  from  Cyberlearning  (www.cyberlearn- 
ing.com).  The  name  of  the  product  has  been  changed 
to  GUIDE. 

AIM  (Authoring  Instruction  Materials)  II  has 
also  proven  to  be  a  successful  endeavor  with  the  U.S. 
Navy;  in  fact,  it  is  now  the  largest  single  database 
of  training  in  the  Navy  (Arnone,  2001).  In  addition 
to  providing  instructional  design  guidance,  it  also 
serves  as  a  primary  content  management  system  and 
monitors  the  workflow  of  instructional  development 
projects.  The  Navy  also  uses  the  system  to  maintain 
design  consistency  for  internal  development  and 
managing  outsource  course  developers  (Johnson, 
2005). 

With  the  promise  and  proven  success  (in  part)  of 
front-end  design  and  AID  solutions,  it  is  a  wonder 
why  these  systems  have  not  garnered  more  wide¬ 
spread  adoption,  especially  as  more  and  more  orga¬ 
nizations  look  to  novice  developers  and  subject  mat¬ 
ter  experts  as  course  designers.  This  will  continue  to 
be  an  area  of  further  research  and  study,  and  these 
early  precursor  projects  will  help  set  the  stage  for 
the  next  generation  of  front-end  design  and  AID 
systems. 


LEARNING-ACTIVITY, 

FOCUSED  DESIGN  TOOLS 
(SIMULATIONS,  GAMES) 

Another  classification  of  tools  used  by  instructional 
designers  consists  of  simulation  and  gaming  tools, 
designed  for  a  precise  purpose  and  to  achieve  very 
specific  instructional  outcomes.  From  the  very  begin¬ 
ning  of  computer-facilitated  instruction,  simulations 
and  games  have  been  an  important  instructional  inter¬ 
vention  to  support  experiential  learning  and  to  reinforce 
higher  level,  cognitive  learning  methodologies.  Early 
simulations  and  games  were  often  custom  programmed 
at  the  coding  level  or  assembled  through  sophisticated 
use  of  standard  authoring  tools.  Of  course,  simulations 
have  been  around  for  many  years  in  a  variety  of  forms, 
including  large-scale  flight  simulators,  role-playing 
simulations  in  the  classroom,  computer-delivered  sim¬ 
ulations,  and,  most  recently,  high-fidelity  simulations 
that  can  be  delivered  through  cyberspace. 

In  a  recent  study,  organizations  indicated  that  they 
would  like  to  use  simulations  and  games  as  part  of  a 
blended  curriculum,  but  they  feel  that  the  barriers  to 
entry  (such  as  high  development  costs  and  difficulty 
of  creating  simulations  on  their  own)  are  simply  too 
prohibitive  (Chapman,  2005).  In  fact,  the  study  found 
that  the  average  development  time  required  to  create 
1  hour  of  simulation  was  750  hours  (ranging  up  to 
1300:1),  compared  to  an  average  ratio  of  220:1  for 
standard  e-learning  courses.  This  explains  why  simu¬ 
lations  and  games  are  currently  underutilized  in  online 
learning  courses;  however,  this  desire  to  utilize  simu¬ 
lations  has  also  resulted  in  an  explosion  of  a  whole 
new  line  of  instructional  design  and  development  tools 
that  focus  on  the  development  of  simulations  and 
games.  The  following  taxonomy  lists  the  major  group¬ 
ings  of  tools  found  on  the  market  today  (representing 
over  106  simulation  tool  providers): 

•  Software  simulation 

•  Soft-skills  simulations 

•  Role-playing  (e.g.,  conversation  with  on¬ 
screen  characters) 

•  Business  skills  (e.g.,  making  decisions  to 
play  out  a  business  scenario  such  as  running 
a  mock  business) 

•  Business  modeling/analytical  (e.g.,  setting 
variable  conditions  and  observing  the  out¬ 
come  based  on  business  rules;  learning  how 
to  interpret  data) 

•  Story-problem/scenario-based  (e.g.,  setting 
up  story  problems  and  having  learners  make 
decisions  to  solve  the  problems) 
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•  Sales  process  simulator  (simulating  mock 
sales  scenarios) 

•  Hard  skills/technical 

•  Troubleshooting/diagnostic  (making  deci¬ 
sions  and  observing  the  outcome  of  each 
action) 

•  Procedural  walk-through  (learner  performs 
steps  in  a  procedure) 

•  Simulating  physical  systems  (e.g.,  simulat¬ 
ing  pieces  of  equipment  or  other  objects, 
setting  up  a  computer  network) 

•  Simulating  concepts  (e  .g . ,  simulating  a  sche¬ 
matic  diagram,  simulating  how  weather  pat¬ 
terns  work  using  a  diagram) 

•  Emergency  response  simulations  (e.g.,  per¬ 
forming  actions  as  a  result  of  an  emergency) 

•  Virtual  worlds/spatial  relationships  (e.g., 
flight  simulators  or  simulating  an  office  envi¬ 
ronment,  cockpit,  or  factory) 

Some  of  the  leading  tools  in  each  of  these  sectors  are 
shown  in  Table  50.3. 

Software  Simulation  Tools 

Software  simulation  tools  allow  novice  developers  to 
record  screen  interactions  while  walking  through  a 
procedure.  The  tool  remembers  each  mouse  click  and 
keystroke  for  the  purpose  of  later  creating  interactive 
simulations— for  example,  registering  the  coordinates 
and  creating  a  click  area  for  each  mouse  click.  Simu¬ 
lation  authors  must  simply  clean  up  the  recording  and 
add  feedback  and  remediation  to  complete  the  simu¬ 
lation.  This  is  considerably  quicker  than  using  standard 
authoring  tools  to  (1)  capture  screens,  (2)  manually 
add  touch  areas  and  text  input  fields,  (3)  create  prompts 
and  instructions  for  the  learners,  (4)  wire  each  click 
or  correct  key  entry  into  a  branching  pattern,  etc.  Soft¬ 
ware  simulation  tools  have  significantly  reduced  devel¬ 
opment  time,  while  preserving  a  high  degree  of  fidelity 
and  interactivity  in  the  learning  event. 

Soft-Skills  Simulation  Tools 

Soft-skills  tools  (Table  50.4)  are  usually  based  on  a 
template  model  for  quickly  creating  specific  types  of 
interactions  such  as  developing  a  dialog,  choosing  on¬ 
screen  characters,  and  creating  a  role-play  simulation 
with  graded  responses  and  comparisons  to  expert  paths 
through  the  simulation  (how  they  may  have  handled 
the  situation).  The  difficulty  for  makers  of  simulation 
tools  is  that  they  must  try  to  provide  the  flexibility 
necessary  to  vary  the  simulation  for  different  purposes 
while  keeping  focused  on  delivering  high-fidelity. 


TABLE  50.3 
Commercially  Available 
Software  Simulation  Tools 


Product 

Captivate 

OnDemand 

Firefly 

STT  Trainer 

Assima 

SoftSim 

RapidBuilder 


Company 

Adobe 

Global  Knowledge 
KnowledgePlanet 
Kaplan  IT  Learning 
Assima 
OutStart 

XStream  Software 


TABLE  50.4 

Commercially  Available  Soft-Skills  Tools 


Product 

Redwood  development  platform 
Forio  simulation  development 
software 
KDSimStudio 
Experience  Builder 
RealCall 
StarTrainer 
Simulated  Role  Play 


Company 

Redwood  e-Learning  Systems 
Forio 

Knowledge  Dynamics 
ExperienceBuilders,  LLC 
SIVOX 
Knowlagent 
SIMmersion,  LLC 


workplace-relevant  scenarios  created  in  a  very  short 
amount  of  time.  Figure  50.7  shows  an  example  of  a 
sophisticated  simulation  in  which  the  learner  makes 
decisions  about  how  society  may  potential  reverse  the 
effects  of  global  warming.  The  simulation,  from  Forio, 
is  capable  of  simulating  elapsed  time  as  the  learner 
may  run  the  scenario  over  months  or  even  years  (com¬ 
pressed  to  an  hour  or  less  online) . 

Hard-Skills/Technical  Simulation  Tools 

The  hard-skills/technical  simulations  category  (Table 
50.5)  covers  a  broad  range  of  skill  areas.  Think  of 
systems  in  this  category  as  being  capable  of  modeling 
physical  systems,  such  as  a  piece  of  machinery  or  an 
electronic  measurement  system.  This  category  also 
covers  task  simulators  designed  to  teach  and  measure 
a  learner’s  ability  to  follow  steps  in  a  procedure  with 
various  levels  of  guidance.  A  good  example  of  how 
this  might  work  is  the  flight  simulators  often  used  in 
the  aviation  industry.  Many  airlines  have  figured  out 
how  to  pass  performance  data  from  sessions  in  a  flight 
simulator  back  to  a  central  learning  management  sys¬ 
tem,  which  keeps  performance  records  for  classroom- 
based  learning  and  e-leaming  courses. 


679 


Bryan  L.  Chapman 


New*?  i*  there 


D«t« 


0J  http:/Awww .^lobAlwarimn^mterArtive.cofn  interface  sdev  Microsoft  Internet  Fvplorcr 


~uipjta' 


— - 


Current  Issues 

Proolsm;  ahead 
P  Sea  Levels  on  the  rise 
S  Reecvrw  running  out 
S  Fossil  fuel  use  still  nigh 


ft 


Submit 


i  Did  you  see  this  item  rrcm  lest  mgrrs  Ation 
anything  yw  car  do  5o  reverse  Bss  trend 


Communications 


"  ; 


* i.fr  .• 


-’opu  31  l  •:  mon 


4)Baw 


Figure  50.7  Global  warming  simulation  developed  using  Forio. 


TABLE  50.5 

Hard-Skills/Technical  Simulation 
Development  Tools 


TABLE  50.6 

Popular  Development  Tools  for  Instructional 
Games 


Product 

Multigen  Creator 
KDCalc 

NGRAIN  Producer 
Visual  Purple 


Company 

Multigen-Paradigm 
Knowledge  Dynamics 
NGRAIN 
Visual  Purple 


Product 

Gameshow  Pro  (Web) 

Composica  Enterprise 
Games2Train 

Game  Development  Environment 


Company 

Learningware 
Composica 
Games2Train 
Galaxy  Scientific 


Gaming  Tools 

Games  play  a  unique  role  in  deploying  an  enterprise¬ 
wide  learning  strategy  (Table  50.6).  Although  not  pri¬ 
marily  designed  to  deliver  an  entire  training  or  teach¬ 
ing  job  alone  (Kirk  and  Belovics,  2004),  games  pro¬ 
vide  a  unique  level  of  motivation  in  learning  areas  such 
as  classification  (matching  games),  reinforcing  factual 
information  (question  and  answer  games),  and  rote 
learning  (memorization  games).  In  addition,  as  the  new 
gaming  generation  enters  the  workforce,  lecture-based 
training  will  likely  be  less  effective  (Kirk  and  Belovics, 
2004).  What  makes  simulation  and  gaming  develop¬ 
ment  tools  unique  is  that  instructional  design  guidance 
is  not  often  obvious  through  the  interface;  rather,  guid¬ 
ance  comes  from  the  intelligence  and  flexibility  of  the 
simulation  or  game  designed  to  achieve  a  specific 
learning  outcome.  The  primary  instructional  benefits 


of  using  such  tools  is  the  depth  of  interaction  and  rich 
feedback  available  for  learners,  achievable  through  rel¬ 
ative  rapid  development  processes  and  minimizing  the 
authoring  savvy  required  on  the  part  of  instructional 
developers  (see  Figure  50.8). 


TEAM  DEVELOPMENT/ 

PUBLISHING  TOOLS 

Since  the  beginning  of  2000,  a  new  classification  of 
instructional  development  tools  has  evolved  that  is 
beginning  to  gain  traction  as  commercially  viable 
options  for  developing  online  learning.  Many  of  the 
most  popular  traditional  authoring  tools  available  today 
are  desktop  applications,  meaning  that  they  typically 
can  be  used  by  only  one  instructional  developer  at  a 
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□  Full  Support  ■  Simulation  &  Gaming 


Figure  50.8  Instructional  design  support  offered  by  focused,  learning-activity  tools  (such  as  simulation  or  gaming). 


time  to  create  interaction.  In  contrast,  this  new  classi¬ 
fication  of  tools  is  based  on  groupware  projects  based 
on  a  publishing  model  for  online  learning  development, 
using  multiple,  concurrent  content  contributors,  each 
working  on  learning  content  and  interactive  exercises 
that  are  part  of  a  larger  course  or  curriculum.  To  illus¬ 
trate  the  point,  consider  the  following  question:  Could 
the  publishers  of  The  Los  Angeles  Times  use  Microsoft 
Word  to  lay  out  and  organize  each  daily  newspaper? 
The  answer  is  “yes.”  How  efficiently  could  it  be  pro¬ 
duced,  though?  The  answer  is  “not  very  efficiently.” 

These  groupware  publishing  systems  are  called 
learning  content  management  systems  (LCMSs).  By 
definition,  an  LCMS  is  a  multideveloper  environment 
where  developers  can  create,  store,  reuse,  manage,  and 
deliver  learning  content  from  a  central  object  reposi¬ 
tory  (Chapman,  2006a).  Although  many  LCMS  solu¬ 
tions  have  built-in  authoring  capabilities,  most  are  also 
designed  to  assemble  individual  learning  activities  that 
may  have  been  created  using  a  variety  of  traditional 
authoring  tools  and  storing  the  learning  events  as  learn¬ 
ing  objects.  A  learning  object  is  a  reusable  learning 
activity  that  can  be  (1)  metadata  tagged  for  easy 
retrieval,  (2)  standards  that  can  communicate  with 
other  learning  technologies,  or  (3)  objects  combined 
or  clustered  with  other  learning  objects  to  create  new, 
derivative  learning  structures  such  as  lessons,  units,  or 
entire  courses.  Table  50.7  lists  some  of  the  most  fre¬ 
quently  used  commercial  LCMS  solutions  to  date 
(Chapman,  2006b). 

Beyond  the  rapid  development  capabilities  offered 
by  these  systems,  LCMS  tools  often  also  include  a 
wide  range  of  embedded  instructional  design  guid- 


TABLE  50.7 

Most  Frequently  Used 

LCMS  Solutions 


Product 

Evolution 

TotalLCMS 

ForceTen 

Saba  LCMS 

Learn.com 

Generation  21 

TopClass 


Company 

Outstart 

SumTotal 

Eedo 

Saba 

Leam.com 
Generation  21 
WBT  Systems 


ance,  at  times  bordering  on  a  hybrid  between  standard 
authoring  and  front-end  design  and  automated  instruc¬ 
tional  design  (AID)  tools.  Following  are  some  of  the 
areas  of  instructional  design  guidance  that  can  be 
found  inside  commercial  LCMS  solutions. 

Interactivity  Templates 

Learning  content  management  system  tools  offer 
even  more  extensive  capabilities  for  using  and  reus¬ 
ing  design  templates.  One  example  would  be  a  gam¬ 
ing  template  that  is  driven  by  a  bank  of  test  questions. 
The  template  provides  the  interface,  look  and  feel, 
game  show  host,  game  boards,  etc.  Because  all  learn¬ 
ing  content  in  an  LCMS  is  stored  in  a  database,  the 
test  questions  can  be  randomly  and  automatically 
extracted  by  the  database  each  time  a  new  version  of 
the  game  is  played;  hence,  the  same  template  could 
yield  thousands  of  variations  of  the  same  gaming 
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Figure  50.9  Script/storyboarding  tool  available  in  QMIND. 


template.  Although  most  LCMS  solutions  are 
designed  to  be  instructional-theory  agnostic,  there  is 
evidence  in  most  systems  that  they  prescribe  to  cer¬ 
tain  aspects  of  instructional  theory.  Although  this 
sounds  like  a  weakness,  it  actually  helps  to  create 
more  focused  design  templates.  Standardizing  on  a 
specific  design  theory  is  what  allows  template  design¬ 
ers  to  create  templates  (Foshay  and  Preese,  2005). 
The  flip  side  of  this  logic  is  that  templated  authoring 
tools  are  very  restrictive,  unless  you  happen  to  pre¬ 
scribe  to  that  specific  theory. 

Dynamic  Pretesting 

Most  LCMS  products  use  a  dynamic  pretesting  model, 
meaning  that  individual  test  questions  can  be  linked 
with  learning  content  (stored  as  learning  objects). 
Based  on  learner  performance  in  the  test,  the  LCMS 
can  dynamically  create  a  new  version  of  the  course 
based  on  demonstrated  performance  in  the  test,  so 
learners  do  not  receive  instruction  on  content  they  have 
already  mastered.  When  used  correctly,  dynamic  pre¬ 
test  can  dramatically  shorten  course  contact  time. 

Learning  Objects 

Although  most  references  to  learning  objects  focus 
on  their  reusability,  there  is  a  more  significant  instruc¬ 
tional  tie-in.  Learning  objects  frequently  are  based 


on  a  single,  enabling  learning  objective,  thus  provid¬ 
ing  focus  and  clarity  on  teaching  and  assessing 
learner  performance  against  their  mastery  of  the 
objective.  Learning  objects— with  a  one-to-one  rela¬ 
tionship  with  enabling  objective— can  be  clustered 
with  other  enabling  objectives  to  create  new  paths 
for  working  toward  a  broader  terminal  objective  (the 
purpose  of  a  lesson  or  entire  course).  Learning  object 
design  helps  focus  instructional  designers  on  their 
ultimate  goal. 

Storyboard  Specifications 

In  the  instructional  development  process,  storyboards 
represent  a  detailed  design  specification  for  the  course 
as  an  elaboration  of  high-level  design  documents. 
LCMS  technology  provides  methods  that  allow  learn¬ 
ing  objects  to  be  based  on  objectives  and  then  designed 
into  delivery  specifications  via  storyboarding  in  sup¬ 
port  of  audio/video  production,  message  design,  and 
keeping  track  of  complex  details  of  interactivity.  Fig¬ 
ure  50.9  shows  an  example  of  an  LCMS -generated 
storyboard  interface  for  QMIND. 

Links  between  Assessment  and  Competencies 

The  makeup  of  LCMS  technology  makes  it  possible 
to  create  sophisticated  links  between  test  questions 
and  competencies  (skills,  knowledge,  and  attitude). 
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□  Full  Support  ■  LCMS 


Figure  50.10  Instructional  design  support  offered  by  learning  content  management  systems  (LCMSs). 


Programming  numerous  connections  between  learn¬ 
ing  performance  and  competencies  is  possible  using 
standard  authoring  tools,  but  the  complexity  makes 
it  a  barrier  to  measuring  learning  performance  in  a 
meaningful  way  for  novice  developers.  In  general, 
LCMSs  provide  less  guidance  than  front-end  and 
automated  design  tools  (see  Figure  50.10).  The  struc¬ 
ture  of  most  systems  is  based  on  the  notion  that  front- 
end  analysis  and  high-level  design  have  already  pre¬ 
viously  been  completed  and  that  the  purpose  of  the 
LCMS  is  to  support  the  remaining  development  and 
delivery  process.  They  do  encourage  developmental 
practices  but  still  come  up  short  in  supporting  the  full 
range  of  instructional  design  guidance  across  the 
entire  spectrum;  however,  of  all  the  commercially 
viable  learning  technologies,  LCMS  technology 
appears  to  be  evolving  toward  a  platform  that  may 
show  promise  in  providing  both  instructional  design 
guidance  and  development  system. 


CONCLUSIONS 

At  present,  myriad  online  learning  development  tools 
cover  a  broad  spectrum  of  choice  and  flexibility;  yet, 
the  shear  number  of  these  tools,  plus  new  emerging 
technologies  such  as  wikis,  blogs,  and  podcasts, 
leaves  organizations  in  a  state  of  confusion  when 
attempting  to  develop  a  cohesive  authoring  and  devel¬ 


opment  strategy.  Although  instructional  design  guid¬ 
ance,  in  some  form  or  another,  is  now  embedded 
inside  several  classifications  of  development  tools, 
considerable  innovation  and  research  are  still 
required  to  create  holistic  systems  that  negate  or  min¬ 
imize  the  need  for  instructional  design  expertise, 
allowing  nontrained  designers  to  create  online  learn¬ 
ing  courseware  without  assistance. 

Perhaps  software  developers  will  never  create  a 
wizard  that  allows  writers  to  fully  write  like  Shakes¬ 
peare,  and  the  same  may  be  true  for  intelligent, 
instructional  design  software,  but  that  does  not  mean 
that  instructional  design  guidance  and  principles 
should  not  be  embedded  in  the  evolution  of  existing 
content  development  tools  or  future  technologies .  In 
the  meantime,  it  is  incumbent  on  every  organization 
that  creates  online  learning  to  find  the  appropriate 
balance  and  mix  between  internal  expertise  (people 
resources)  and  available  tools  and  technology  for 
designing  instruction. 

Great  appreciation  is  extended  to  those  who  con¬ 
tinually  push  the  envelope  and  introduce  instructional 
design  savvy  into  the  tools  of  the  trade.  They  have 
laid  a  great  foundation  for  future  innovators  to  draw 
upon  to  create  new  methods,  new  tools,  and  new 
solutions,  as  long  as  they  keep  their  vision  clearly 
focused  on  the  ultimate  goal— namely,  the  design  and 
creation  of  effective  learning  experience  that  result 
in  optimal  knowledge  transfer. 
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ABSTRACT 

Both  process  and  product  artifacts  are  collected  and 
disseminated  in  many  fields  of  design  as  a  source  of 
design  knowledge  in  the  form  of  precedent,  which 
includes  both  the  artifacts  themselves  and  the  memo¬ 
ries,  or  schema,  retained  as  a  result  of  interaction  with 
or  examination  of  the  artifacts.  The  emergence  of  seri¬ 
ous  study  into  the  practice  of  design  in  our  field  will 
extend  beyond  the  study  of  process  models  and  may  be 
productively  informed  by  understanding  how  artifacts 
are  used  as  tools  by  designers. 


KEYWORDS 

Artifacts:  Designed  objects  or  systems,  including  those 
created  in  the  process  of  design  and  those  resulting 
from  the  act  of  design. 

Precedent:  Artifacts  and/or  memory  of  designing  them 
or  examining  them. 


INTRODUCTION 

In  recent  years,  there  has  been  much  discussion  in  the 
instructional  design  and  technology  literature  regarding 
the  ways  we  practice  and  teach  design.  Although  the 
field  has  a  significant  history  of  developing  process 
models  to  guide  and  structure  design  activities  (such  as 
those  reviewed  by  Andrews  and  Goodson,  1980) ,  recent 
criticisms  of  these  sorts  of  models  (Gordon  and  Zemke, 
2000;  Zemke  and  Rossett,  2002)  have  encouraged 
reconsideration  of  basic  assumptions  underlying  prac¬ 
tice  in  the  field  and  have  emphasized  the  need  for  more 
flexible  approaches  to  complex  design  problems  (Nel¬ 
son  and  Stolterman,  2003).  Authors  have  emphasized 
the  importance  of  defining  the  field’s  value-added  prop¬ 
osition  in  the  face  of  rapidly  changing  academic  and 
corporate  settings  (Hill  et  al.,  2004),  and  for  some  years 
authors  have  openly  wondered  if  the  field  could  be 
strengthened  by  looking  at  processes  and  habits  of 
thought  in  allied  design  fields  (Bichelmeyer  et  al.,  2006; 
Johnson,  2005;  Murphy,  1992;  Rowland  et  al.,  1994). 
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In  this  environment  of  reflection,  we  anticipate  the 
need  for  increased  attention  to  research  in  the  area  of 
design  science  (Cross,  2001),  with  a  view  toward 
expanding  what  Bichelmeyer  (2003)  termed  instruc¬ 
tional  design  theory,  which  she  said  was  required  to 
systematically  study  the  work  of  instructional  design 
and  the  impact  of  such  designs. 

We  propose  to  begin,  in  part,  by  examining  the 
ways  in  which  other  design  fields  (such  as  art,  archi¬ 
tecture,  and  engineering)  teach  and  practice  design. 
We  do  not  suggest  that  instructional  design  is  entirely 
the  same  as  other  design  fields,  or  that  it  should  be; 
however,  we  do  suggest  that  considering  practices 
from  related  fields,  both  successful  and  unsuccessful, 
and  pursuing  the  research  questions  generated  by  com¬ 
parisons  between  their  practices  and  those  of  instruc¬ 
tional  systems  design  (ISD)  is  a  potentially  fruitful 
avenue  for  expanding  our  own  design  science.  This 
chapter  examines  one  aspect  of  design  practice  in  fields 
that  share  characteristics  with  ISD:  the  use  of  artifacts 
as  tools  for  carrying  out  and  teaching  design. 


ARTIFACTS  AS  TOOLS 

A  striking  feature  of  many  design  fields  is  the  wide¬ 
spread  use  of,  and  attention  given  to,  artifacts  as  tools 
in  the  design  process.  These  artifacts  fall  into  two 
primary  categories:  process  artifacts  and  product  arti¬ 
facts.  Process  artifacts  include  sketches  of  several 
kinds  (conceptual  sketches,  decision  diagrams,  formal 
or  visualizing  sketches,  and  so  on)  as  well  as  docu¬ 
mentation,  generally  written  reports  created  at  virtually 
any  stage  of  the  design  process.  Product  artifacts  are 
comprised  of  design  work  already  completed,  includ¬ 
ing  instructional  materials,  final  and  end-user  docu¬ 
mentation,  presentation  materials,  and  episodic  mem¬ 
ories  of  reviewing  or  experiencing  designed  products . 
These  two  categories  overlap,  as  the  type  of  an  artifact 
can  depend  on  how  it  is  being  used  by  designers  at  a 
given  time. 

Some  process  artifacts  and  product  artifacts  are 
propositional;  that  is,  they  are  largely  verbal  and  largely 
abstract.  Even  when  a  design  report  describes  the  plans 
for  an  instructional  experience,  for  example,  such  a 
description  does  not  tend  to  be  a  concrete,  or  tangible, 
representation  of  that  experience.  It  may  be  very 
detailed  but  does  not  as  often  take  the  form  of  a  narra¬ 
tive,  a  symbolic/diagrammatic  representation,  or  a  spa¬ 
tial/verbal  diagram— forms  currently  being  explored  by 
Parrish  (2006),  Appelman  (2005),  and  Botturi  (2006). 
In  some  of  the  related  design  fields  under  discussion, 
artifacts  are  more  often  descriptive  and  concrete ,  visual 
or  constructed.  They  include  sketches,  concept  dia¬ 


grams,  prototypes,  and,  to  a  greater  degree  than  is  pub¬ 
licly  practiced  in  our  held,  finished  artifacts  (products) 
or  published  representations  of  such  artifacts. 

Visual  Process  Artifacts  as  Cognitive  Tools 

One  of  the  most  ubiquitous  forms  taken  by  process 
artifacts  in  many  fields  of  design  is  sketching.  Fish  and 
Scrivener  (1990)  detailed  the  visual  cognition  process 
involved  in  sketching.  They  described  the  interplay 
between  the  two  modes  of  mental  representation— 
propositional  (largely  symbolic)  and  analog  (“quasi¬ 
pictorial,  spatially  depictive”)— required  for  sketching 
(p.  121).  As  individuals  sketch,  the  percept  (what  they 
are  seeing)  always  undergoes  mental  manipulation 
before  it  is  represented  as  marks;  in  turn,  the  marks 
“generate  mental  images  that  may  in  turn  influence  the 
sketch”  (p.  120).  Although  they  address  sketching 
directly  from  life  for  the  most  part,  they  also  state 
"there  is  objective  evidence  that  spatially  depictive 
images  generated  from  memory  have  many  of  the 
properties  needed  to  explain  the  ability  of  artists  and 
designers  to  generate,  manipulate,  combine  and 
inspect  in  imagination  non-existent  visual  objects”  (p. 
122). 

Goldschmidt  (1991)  built  on  the  work  of  Fish  and 
Scrivener  to  study  the  use  of  sketching  in  the  design 
process,  finding  what  she  calls  an  oscillation  between 
propositional  thinking  and  descriptive  thinking.  She 
observed  designers  using  two  types  of  argumentation, 
based  on  these  two  types  of  thinking,  during  the  pro¬ 
cess  of  design.  The  first  is  seeing  as  argumentation, 
which  is  depictive  and  sketch  based.  The  second  is 
seeing  that  argumentation,  which  is  nonfigural.  The 
use  of  these  two  types  of  argumentation  oscillates, 
particularly  in  skilled  designers,  over  many  iterations. 
Goldschmidt  stated  that  "the  order  in  which  arguments 
switch  modalities  is  not  important.  What  is  significant 
is  the  fact  that  the  shift  occurs  both  ways.  ...[It]  helps 
translate  the  particulars  of  form  into  generic  qualities, 
and  generic  rules  into  specific  appearances”  (pp. 
138-139).  In  nonsketching  protocols,  Goldschmidt 
reported  that  “‘seeing  as’  arguments  are  by  far  in  the 
minority”  (p.  140). 

Fish  and  Scrivener  examined  direct  sketching  from 
reality,  while  Goldschmidt  turned  her  attention  to 
sketching  from  the  internal  eye  during  the  process  of 
architectural  design.  Laseau  (1986)  did  not  conduct 
formal  studies  but  brought  decades  of  observational 
and  professional  experience  to  bear  on  his  discussions 
of  visual  artifacts  that  represent  nonformal  properties 
of  a  design  space.  His  work  makes  it  clear  that  visual 
artifacts  are  not  produced  solely  for  the  sake  of  pic¬ 
turing  an  essentially  visual  product.  Suwa  and  Tversky 
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(1996),  in  their  study  of  sketching  among  architecture 
students  and  professionals,  underscored  Laseau’s 
experiential  insights.  They  concluded  that  study  par¬ 
ticipants  were  able  to  use  their  sketches  to  explore  not 
only  visual  relationships  among  parts  but  also  to 
explore  functional  relationships  (such  as  lighting  or 
circulation).  They  noted  that:  “This  analysis  has 
revealed  that  sketches  make  apparent  not  only  percep¬ 
tual  relations  but  also  inherently  non-visual  functional 
relations  to  both  advanced  design  students  and  prac¬ 
ticing  architects”  (p.  192). 

Goel  (1995)  offered  an  explanation  for  the  mech¬ 
anism  whereby  sketching  supports  cognition  during 
the  design  process.  He  explained  that  sketches  sup¬ 
port  design  cognition  by  leaving  options  open 
because  the  elements  of  the  symbol  system  used  in 
sketching  are  non-notational.  He  drew  on  Goodman’s 
(1976)  taxonomy  of  symbol  systems  in  which  non- 
notational  systems  are  defined  as  ambiguous  with 
respect  to  what  a  given  mark  may  stand  for.  As  an 
example,  a  circle  in  a  sketch  may  represent  a  sun,  a 
wheel,  a  plate,  or  anything  round.  This  ambiguity 
allows  the  designer  to  defer  specific  interpretation  of 
the  sketch  (or  diagram,  as  described  by  Laseau,  1986) 
and  entertain  or  realize  alternatives  within  the  image 
that  may  not  have  been  intended  at  the  time  of  its 
creation.  Fish  and  Scrivener  (1990)  labeled  this  same 
quality  as  indeterminacy  and  speculated,  based  on 
the  visual  theories  of  Kosslyn,  a  key  proponent  of  the 
neurophysiologic  view  of  representations  in  the  brain 
as  quasi-pictorial  and  manipulable,  that  “sketch  inde¬ 
terminacy  may  trigger  innate  recognition  search 
mechanisms  into  generating  a  stream  of  imagery  use¬ 
ful  to  invention”  (pp.  121-123). 

In  his  exhaustive  discussion  of  the  types  of  knowl¬ 
edge  required  for  design  engineers,  Vincenti  (1990,  p. 
220)  pointed  out  that  nonverbal  thinking  in  the  form 
of  sketching,  drawings,  graphs,  and  models  is  a  critical 
way  of  thinking  that  falls  under  the  knowledge  cate¬ 
gory  design  instrumentalities.  In  his  discussion  of  how 
knowledge  builds  in  the  field  of  engineering,  Vincenti 
also  discussed  visible  sketching  and  doodling,  which 
serve  as  vicarious  overt  trials.  These  trials  are  mech¬ 
anisms  for  selecting  variation  in  a  larger  model  that 
includes  generating  blind  variations  of  design  solu¬ 
tions  or  components  of  solutions  (blind  in  the  sense 
that  their  outcomes  cannot  be  known  until  they  are 
actually  built  and  tried),  employing  consistent  pro¬ 
cesses  for  selecting  variations,  and  preserving  or  prop¬ 
agating  the  variations  that  are  selected.  When  discuss¬ 
ing  his  hypothesis  that  “instruments,  on  a  par  with 
theory,  bear  knowledge,”  Baird  (2004,  p.  xvii)  criti¬ 
cized  Vincenti  for  not  going  further  in  his  exploration 
of  engineering  knowledge  to  accord  artifacts  them¬ 


selves  (including,  presumably,  sketches)  the  status  of 
knowledge.  Baird’s  work,  however,  focused  primarily 
on  scientific  instruments,  and  more  exploration  of  this 
area  is  needed  to  establish  solid  links  between  his 
detailed  analysis  and  process  artifacts. 

Use  of  Product  Artifacts 

In  his  discussion  of  the  design  knowledge  used  by 
architects,  Lawson  (1980)  stressed  designers’  reliance 
on  episodic  memory  rather  than  general  principles  or 
problem-solving  algorithms  (i.e.,  process  models)  to 
generate  design  solutions.  Episodic  memory  includes 
direct  experience  with  designed  artifacts  (designing 
them  or  experiencing  the  designs  of  others)  and  vicar¬ 
ious  experience  with  descriptions  and  representations 
of  designed  artifacts.  In  more  recent  writing,  Lawson 
(2004b,  p.  443)  explained  that  “precedent  stored  in  the 
form  of  episodic  memory,”  either  of  one’s  own  design 
or  of  the  designed  artifacts  of  others,  “is  used  by 
experts  to  recognize  design  situations  for  which  gam¬ 
bits  are  available,”  where  gambits  are  defined  as  “pat¬ 
terns  known  to  have  certain  properties  and  to  offer 
certain  capabilities”  (p.  449). 

In  his  widely  cited  analysis  of  design  science,  or 
the  study  of  design.  Cross  (2001,  p.  54)  listed  experi¬ 
ence  in  designing  and  manufacturing  artifacts  as  two 
sources  of  design  knowledge  and  the  “knowledge 
inherent  in  the  artifacts  of  the  artificial  world”  as  the 
third.  Vincenti  (1990,  pp.  208)  pointed  to  “fundamental 
design  concepts”  as  the  first  category  of  design  knowl¬ 
edge  for  engineers.  These  concepts  include  the  opera¬ 
tional  principles  underlying  a  device  (how  the  device 
works)  and  “normal  configuration”  or  the  “general 
shape  and  arrangement  that  are  commonly  agreed  to 
best  embody  the  operational  principle.”  These  funda¬ 
mental  concepts  are  learned  through  experience  even 
before  students  begin  to  study  design  and  are  also 
“learned  deliberately  by  the  design  community.”  Epi¬ 
sodic  memory,  knowledge  inherent  in  artifacts,  and 
fundamental  design  concepts  are  all  derived  in  one  way 
or  another  from  the  conscious  examination  of  product 
artifacts.  “Designers  appear  to  be  able  to  browse  freely 
and  associatively  between  multiple  precedents  [and] 
browsing  enables  the  discovery  of  new,  often  unantic¬ 
ipated,  concepts  in  precedents”  (Oxman,  1994). 

In  much  of  design  education,  the  use  of  product 
artifacts  is  considered  to  be  critical.  The  authors  con¬ 
ducted  a  simple  analysis  of  published  design  class 
syllabi  in  graphic  design  and  illustration  (Heller,  1998; 
Heller  and  Arisman,  2000).  Each  syllabus  was  exam¬ 
ined  to  determine  whether  or  not  there  were  explicit 
references  to  the  use  of  artifacts  in  any  one  of  three 
ways  listed  below: 
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TABLE  51.1 

Instances  of  Syllabi  Explicitly  Calling  for  Use  of  Artifacts  in  Graphic  Design  and 
Illustration  Courses  Surveyed 


Number 
of  Syllabi 

Type  of  Syllabus  Surveyed 
Illustration  courses  21 

Design  courses  8 


Number  of  Syllabi  Explicitly  Calling  for  Use  of  Artifacts 


Artifacts  Created 
by  Others 


Artifacts  Created 
by  Students 


Student  Memory  of 
Artifacts  or  Experiences 


13/21  (62%) 
3/8  (38%) 


14/21  (67%) 
1/8  (13%) 


2/21  (10%) 
0 


•  Students  are  instructed  to  use  artifacts  that 
have  been  created  by  others. 

•  Students  participate  in  critiques  of  their  own 
designs  in  process  as  represented  by  presen¬ 
tation  artifacts. 

•  Students  are  expected  to  draw  upon  memo¬ 
ries  of  artifacts  or  experiences. 

Our  findings  are  summarized  in  Table  51.1.  In  each 
collection  of  syllabi,  courses  covering  professional 
issues  (e.g.,  business  skills  for  designers)  and  theoret¬ 
ical  or  philosophical  issues  (e.g.,  Green  Design) 
appeared  together  with  the  studio  courses  focused  on 
design  practice;  the  former  typically  did  not  include 
the  use  of  artifacts.  Many  other  courses  did,  including 
over  half  the  courses  in  the  sample  of  syllabi  from 
illustration  programs  that  required  students  to  use  their 
own  artifacts  or  those  created  by  others  as  part  of  their 
design  education.  In  the  smaller  sample  of  graphic 
design  syllabi,  almost  40%  required  the  use  by  students 
of  artifacts  created  by  others.  It  may  not  be  surprising 
to  see  few  requirements  for  using  memories  of  artifacts 
in  design  courses  given  the  emphasis  on  hands-on 
experience  in  such  courses  and  the  possibility  that 
instructors  assume  students  will  use  episodic  memory 
while  designing  as  a  matter  of  course. 

Anthony’s  (1999)  extended  criticism  of  the  design 
jury  (a  public  form  of  critique  often  incorporating  exter¬ 
nal  reviewers  or  panels  of  reviewers  to  comment  on 
students’  work)  in  architecture  education,  which  ques¬ 
tions  the  forms  that  this  pedagogical  practice  takes,  also 
makes  clear  that  explaining  one’s  own  artifacts  and 
experiencing  criticism  of  them  is  considered  an  inte¬ 
gral,  essential  part  of  design  education  in  her  discipline. 
The  most  compelling  evidence  of  her  belief  in  the  use 
of  artifacts  for  design  education  is  that  the  alternatives 
to  the  classic  design  jury  that  she  proposes  ah  include 
the  consideration  and  critique  of  designed  artifacts. 

One  of  the  most  visible  aspects  of  most  design 
fields  is  their  production  of  formal  precedent:  publica¬ 
tions,  juried  competitions,  design  collections,  case 
studies,  and  so  on.  Precedent  in  the  sense  we  are  using 


it  here  does  not  carry  the  connotation  of  restriction  on 
future  decision-making  as  it  does  when  used  as  a  legal 
term.  It  does  include  any  and  every  designed  artifact 
from  which  a  designer  might  draw  understanding  of 
solutions  in  specific  contexts  or  inspiration  for  seeing 
as  argumentation  that  might  apply  to  a  current  design 
situation.  Episodic  memory  is  a  form  of  precedent 
individual  to  each  designer,  and  it  is  informal  in  the 
sense  that  it  is  idiosyncratic  to  the  experiences  and 
focus  of  each  designer.  Episodic  memory  is  greatly 
extended  by  the  production  of  formal  precedent  which 
allows  individual  designers  to  examine  the  artifacts  of 
many  more  peers  than  would  be  available  in  the 
absence  of  published  and  collected  representations  of 
design  products.  Vincenti  (1990)  took  the  existence  of 
both  formal  and  informal  precedent  and  its  role  in 
disseminating  design  knowledge  so  much  for  granted 
that  he  said,  “The  methods  of  propagating  and  preser¬ 
vation  of  engineering  knowledge  (journals,  hand¬ 
books,  textbooks,  engineering  school  teaching,  design 
traditions,  word  of  mouth,  and  so  forth)  are  obvious 
in  our  cases  and  do  not  require  elaboration”  (p.  242). 

A  lot  of  time  and  money  goes  into  disseminating 
precedent  and  maintaining  adequate  access  to  precedent 
for  designers  and  design  students  in  most  fields  of  design. 
A  brief  search  for  design  publications  in  well-stocked 
bookstores  or  online  and  a  survey  of  these  publications 
can  make  this  point  evident.  To  explore  a  fraction  of 
the  publications  and  competitions  in  one  held  of  design, 
the  reader  may  visit  the  website  of  the  American  Insti¬ 
tute  of  Graphic  Artists  (AIGA;  http://www.aiga.org/). 

Rowe  et  al.  (2005)  reported  on  their  visits  to  prac¬ 
ticing  designers  from  three  fields  (sculpture,  editorial 
illustration,  and  graphic  design)  in  their  work  spaces 
to  observe  the  forms  of  precedent  that  they  collect  and 
used.  These  included  annual  publications  featuring  the 
works  of  designers  who  pay  to  be  included;  specialty 
journals  featuring  representations  of  product  artifacts 
and  process  artifacts  with  commentary;  books  in  which 
multiple  samples  of  specific  product  types  are  col¬ 
lected,  often  with  descriptions  of  the  design  situations 
and  decisions  made;  collections  of  illustrated  books 
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and  individual  images  from  all  kinds  of  sources;  online 
image  collections;  and  catalogs  from  competitive 
shows  of  work.  Many  of  these  sources  of  precedent 
clearly  require  widespread  financial  and  logistical  sup¬ 
port  for  collecting,  publishing,  and  distributing  the 
materials  as  well  as  a  strong  commitment  on  the  part 
of  individual  designers  to  contribute  examples  of  their 
work  and  participate  in  competitions  and  shows  of 
work.  The  designers  in  this  small  study  reported  using 
precedent  materials  specifically  as  sources  of  technical 
and  conceptual  learning,  springboards  for  design  inspi¬ 
ration,  and  vehicles  for  building  fundamental  skills. 

COLLECTING  AND  DISSEMINATING 
PRECEDENT  IN  ISD 

Several  noteworthy  efforts  at  collecting  and  dissemi¬ 
nating  precedent  exist  in  the  field  of  instructional 
design.  The  St.  Louis  Educational  Museum,  founded 
in  1905  (eventually  the  Audiovisual  Department  of  the 
St.  Louis  Public  Schools  and  subsumed  into  other 
departments)  was  an  early  such  effort.  Allen  (2005) 
reported  that  the  museum  first  housed  key  educational 
exhibits  from  the  St.  Louis  World’s  Lair  of  1904  and 
subsequently  collections  of  visuals,  realia,  and,  in  time, 
audio-visual  materials  and  educational  radio  shows. 
Much  of  this  material  was  donated  in  about  2003  to 
the  Missouri  Historical  Society  and  the  Academic  Lilm 
Archive  of  North  America.  In  1946,  Ole  Larson  at 
Indiana  University  started  an  instructional  film  library 
by  purchasing  materials  from  Encyclopedia  Britannica 
and  other  sources.  These  were  rented  to  schools  and, 
in  turn,  funded  the  production  of  new  audio-visual 
educational  materials  (Instructional  Systems  Technol¬ 
ogy  Department,  2007).  Although  both  of  these  col¬ 
lections  still  exist  in  archive  form,  neither  was  origi¬ 
nally  intended  to  serve  as  precedent  material  for 
designers,  and  neither  is  now  readily  available  as  such. 

Hooper  et  al.  (2002),  of  the  University  of  Minnesota 
Curriculum  and  Instruction  and  Design,  Housing,  and 
Apparel  programs,  developed  and  coordinated  the  Uni¬ 
versity  of  Minnesota  Learning  Software  Design  Com¬ 
petition,  which  ran  from  2000  to  2002,  the  goals  of 
which  were  to  “promote  innovative  educational  soft¬ 
ware  design  [and]  developing  resources  for  instructional 
design  students  and  practitioners”  (p.  5).  Pacificorp  and 
the  Design  and  Development  Division  of  the  Associa¬ 
tion  for  Educational  Communications  and  Technology 
(AECT)  have  collaborated  on  the  Pacificorp  Design  and 
Development  Award  competition  since  2004.  The  goals 
of  this  competition  are  to  promote  and  recognize  inno¬ 
vative  design  in  adult  instruction  and  to  promote  collab¬ 
oration  and  mentoring  between  students,  faculty  and 


practitioners  of  instructional  design  (AECT,  2007). 
There  is  not  an  explicit  commitment  to  collection  and 
dissemination  of  material  from  the  competitions  to  serve 
as  precedent  in  the  field.  Since  the  May/June  2004  issue, 
TechTrends  (the  membership  journal  of  AECT)  has  pub¬ 
lished  articles  under  the  section  heading  “The  Instruc¬ 
tional  Design  Portfolio.”  The  stated  purposes  of  this 
feature  are  to  “offer  practitioners  in  the  field  of  ID  a 
chance  to  see  the  work  of  other  designers”  and  to  “pro¬ 
vide  for  the  sharing  of  design  knowledge  that  is  not 
always  available  in  other  formats”  (AECT,  2004). 

Apart  from  these  efforts  (not  all  of  them  yielding 
published  precedent  and  some  industry  efforts  not 
being  available  to  designers  outside  the  companies  or 
design  groups  involved),  the  authors  are  not  aware  of 
a  volume  of  relevant  precedent  materials,  or  discipline¬ 
wide  efforts  at  disseminating  such  precedent,  in  the 
instructional  design  field  comparable  to  those  evi¬ 
denced  in  the  work  spaces  of  the  designers  visited 
during  our  pilot  study. 


IMPLICATIONS  FOR  ISD 

We  are  a  design  field.  Murphy  (1992)  came  to  this  con¬ 
clusion  in  reviewing  Lawson’s  (1980)  work  on  the  think¬ 
ing  and  knowledge  of  designers.  Goel  (1995),  in  study¬ 
ing  the  cognition  of  design,  made  the  case  that 
instructional  design  is  a  prototypical  design  activity, 
because  its  core  activity— the  design  of  instruction— 
shares  high-level  characteristics  with  other  prototypical 
design  endeavors.  His  analysis  of  prototypical  design 
fields  holds  strong  face  validity  for  those  who  have  prac¬ 
ticed  instructional  design.  He  enumerated  12  character¬ 
istics  of  design-task  environments  and  explained  that  the 
degree  to  which  an  environment  exhibits  these  charac¬ 
teristics  is  the  degree  to  which  it  resembles  a  prototypical 
design  environment.  These  characteristics  are  (1)  a  lack 
of  information  exists  about  the  start  state,  the  goal  state, 
and  the  transformation  required  to  go  from  one  to 
another;  (2)  constraints  on  the  task,  either  natural  or 
manmade,  do  not  constitute  or  define  the  task;  (3)  prob¬ 
lems  are  large  and  complex;  (4)  problems  do  not  decom¬ 
pose  into  distinct  units  except  as  the  designer  or  as  cus¬ 
tom  dictate;  (5)  components  of  the  problem  are  only 
connected  by  contingent  interdependency;  (6)  there  are 
no  right  and  wrong  answers;  (7)  informational  inputs  to 
and  outputs  from  the  problem  conform  to  certain  cate¬ 
gories;  (8)  feedback  loops  need  to  be  simulated  before 
the  designed  artifact  exists;  (9)  errors  carry  costs;  (10) 
the  artifact  will  have  to  function  independent  of  the 
designer;  (11)  specification  and  delivery  are  distinct  from 
each  other;  and  (12)  specification  and  delivery  are  sep¬ 
arated  in  time  (pp.  85-87).  Using  these  characteristics, 
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most  instructional  design  problem  spaces  can  be  seen  to 
be  close  to  the  prototypical  design  case,  which  places 
instructional  design  close  to  architecture,  design  engi¬ 
neering,  graphic  design,  and  other  fields  of  design  and 
allows  us  to  ask  whether  their  practices  may  have  rele¬ 
vance  for  us. 

If  they  do,  much  research  lies  ahead  in  multiple 
areas.  In  the  area  of  artifacts  as  tools  for  design, 
research  is  needed  to: 

•  Examine  further  how  the  use  of  sketching 
and  precedent  might  be  incorporated  effec¬ 
tively  into  instructional  design  activities. 

•  Determine  what  might  constitute  rigor  and 
applicability  in  representations  of  instruc¬ 
tional  designs  for  use  as  precedent. 

•  Explore  the  barriers  and  facilitating  factors 
present  in  the  held  for  investing  in  the  pro¬ 
duction  and  use  of  precedent. 

•  Establish  pedagogical  guidelines  for  incor¬ 
porating  “designerly  ways  of  knowing” 
(Cross,  2001)  into  ISD  education. 
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ABSTRACT 

The  purpose  of  this  chapter  is  to  introduce  current 
research  and  theory  on  systems  design  for  change,  or 
systemic  change,  in  the  fields  of  education  and  training. 
Systems  design  is  the  process  for  determining  what 
characteristics  a  new  system  should  have,  resulting  in 
a  model  of  the  new  system  and  a  plan  for  creating  it. 
Systemic  change  is  the  process  of  changing  a  system 
from  one  paradigm  to  another  by  applying  systems 
thinking  and  systems  theory.  Repeated  calls  for  massive 
reform  of  current  educational  and  training  practices 
have  consistently  been  published  over  the  last  several 
decades.  This  has  resulted  in  an  increasing  recognition 
of  the  need  for  systemic  change  in  education  as  numer¬ 
ous  structured,  piecemeal  approaches  to  education 
reform  have  been  implemented  and  failed  to  signifi¬ 
cantly  improve  the  state  of  education.  This  chapter  first 
presents  a  description  of  design  theory  and  systems 
theory  as  the  foundations  for  systems  design.  Design 
theory  is  the  concepts  and  principles  that  help  to 
develop  strategies  and  methods  for  designing.  The  term 
system  has  been  defined  in  various  ways  by  different 
researchers,  but  the  core  concept  is  one  of  relationships 
among  components  comprising  the  whole. 


KEYWORDS 

Design  theory:  The  concepts  and  principles  that  help 
to  develop  strategies  and  methods  for  designing. 

Systemic  change:  The  process  of  changing  a  system 
from  one  paradigm  to  another  by  applying  systems 
thinking  and  systems  theory. 

Systems  design:  The  process  for  determining  what 
characteristics  a  new  system  should  have. 

Systems  theory:  An  interdisciplinary  held  with  appli¬ 
cations  in  both  the  hard  and  soft  sciences;  it  focuses 
on  understanding  relationships  among  components 
comprising  the  whole. 


INTRODUCTION 

The  purpose  of  this  chapter  is  to  introduce  current 
research  and  theory  on  systems  design  for  change,  or 
systemic  change,  in  the  fields  of  education  and  training. 
Systems  design  is  the  process  for  determining  what 
characteristics  a  new  system  should  have,  resulting  in 
a  model  of  the  new  system  and  a  plan  for  creating  it 
(Banathy,  1996).  Systemic  change  is  the  process  of 
changing  a  system  from  one  paradigm  to  another  by 
applying  systems  thinking  and  systems  theory;  however 
systemic  change  has  different  meanings  for  different 
people  in  education.  Squire  and  Reigeluth  (2000)  iden¬ 
tified  four  different  meanings:  statewide  changes,  dis¬ 
trictwide  changes,  schoolwide  changes,  and  ecological 
changes.  Ecological  systemic  change  is  based  on  an 
understanding  of  interrelationships  and  interdependen¬ 
cies  with  the  system  and  between  the  system  and  its 
systemic  environment.  This  meaning  more  fully  imple¬ 
ments  the  concepts  of  systems  theory  and  systems 
thinking  by  embracing  the  organic,  interconnected 
nature  of  social  systems,  and  it  encompasses  the  other 
three  meanings,  so  it  will  be  the  focus  of  this  chapter. 

A  systems-thinking  or  systemic  approach  to  design 
views  problems  and  their  solutions  from  the  perspec¬ 
tive  of  the  whole  system.  A  system  is  composed  of 
many  parts,  all  of  which  relate  to  each  other.  Systems 
design  takes  into  account  the  interrelationships  among 
these  parts,  rather  than  isolating  individual  problems 
and  simplifying  solutions  by  decomposing  and  frag¬ 
menting  reality  into  an  easier-to-understand  but  incom¬ 
plete  view. 

The  next  section  of  this  chapter  presents  the  foun¬ 
dations  of  systems  design  by  illustrating  the  need  for 
systems  design  in  education  and  training  and  summa¬ 
rizing  the  major  literature  in  design  theory  and  systems 
theory.  The  third  section  provides  a  synthesis  of  sys¬ 
tems  design  principles.  The  fourth  and  final  section 
presents  a  number  of  current  systems  design  models 
in  the  literature. 
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FOUNDATIONS  OF  SYSTEMS  DESIGN 

Need  for  Systems  Design  and  Systemic  Change 

Systemic  change  is  concerned  with  the  creation  of  a 
completely  new  system  rather  than  a  mere  retooling 
of  a  current  system.  It  entails  a  paradigm  shift  as 
opposed  to  piecemeal  change  .  Repeated  calls  for  mas¬ 
sive  reform  of  current  educational  and  training  prac¬ 
tices  have  consistently  been  published  over  the  last 
several  decades.  This  has  resulted  in  an  increasing 
recognition  of  the  need  for  systemic  change  in  educa¬ 
tion  as  numerous  piecemeal  approaches  to  education 
reform  have  been  implemented  and  failed  to  signifi¬ 
cantly  improve  the  state  of  education.  But,  is  a  true 
paradigm  shift  needed  to  better  meet  the  needs  of 
today’s  learners? 

Numerous  publications  have  discussed  the  shift  of 
society  from  the  industrial  age  into  what  many  call  the 
information  age  (Reigeluth,  1994;  Senge  et  al.,  2000; 
Toffler,  1980).  The  current  educational  and  training 
systems  were  built  to  fit  the  needs  of  an  industrial-age 
society,  where  the  focus  was  on  sorting  learners  rather 
than  on  learning  (Reigeluth,  1994).  Learners  in  the 
industrial  age  were  expected  to  learn  the  same  amount 
of  material  in  the  same  time,  thereby  forcing  the  slower 
students  to  accumulate  learning  deficits  and  eventually 
fail.  In  the  industrial  age,  it  was  important  to  sort 
learners  into  management  or  worker  roles,  and  the 
teacher-centered,  standardized  paradigm  of  education 
was  well  suited  for  this  purpose  (Joseph  and  Reigeluth, 
2002). 

In  the  current  information  age,  however,  the  major¬ 
ity  of  jobs  entail  knowledge  work  that  requires  learners 
to  master  such  skills  as  communication,  problem-solv¬ 
ing,  critical  thinking,  and  teamwork.  Furthermore, 
employees  are  more  and  more  expected  to  show  ini¬ 
tiative,  manage  themselves,  and  cooperate  with  others; 
therefore,  training  and  education  now  must  have  a 
customized  and  learner-centered  focus  that  the  old  par¬ 
adigm  does  not  offer.  Systemic  change  seeks  to  shift 
from  a  paradigm  in  which  time  is  held  constant, 
thereby  forcing  achievement  to  vary,  to  one  designed 
specifically  to  meet  the  needs  of  information-age  learn¬ 
ers  and  their  communities  by  allowing  students  as 
much  time  as  each  needs  to  reach  proficiency.  Systems 
design  focuses  on  creating  a  new  system  to  meet  the 
new  educational  and  training  needs  of  the  information 
age. 

The  foundations  of  systems  design  are  systems 
theory  and  design  theory.  The  following  two  sections 
summarize  these  theories  and  their  relation  to  systems 
design. 


Design  Theory 

One  foundation  of  systemic  change  and  systems  design 
is  design  theory.  Design  theory  is  the  concepts  and 
principles  that  help  to  develop  strategies  and  methods 
for  designing.  A  number  of  different  design  theories 
in  the  literature  provide  insight  on  the  complex  and 
challenging  task  of  designing  organizations  or  educa¬ 
tional  systems.  These  include  Nelson  and  Stolterman’s 
(2003)  design  way,  which  is  applicable  to  all  kinds  of 
organizations;  Ackoff’s  (1999)  idealized  design  for 
corporations;  Banathy’s  (1996)  social  systems  design 
for  all  kinds  of  organizations;  and  Reigeluth's  (2006a) 
leveraged  emergent  approach  for  educational  systems. 

The  Design  Way 

Nelson  and  Stolterman  (2003)  noted  that,  fundamen¬ 
tally,  design  is  a  creative  act,  resulting  in  the  creation 
of  something  that  has  not  previously  existed.  It  focuses 
on  making  choices  to  create  the  best  design  for  a  very 
specific  system.  As  such,  it  examines  a  real,  natural, 
complex  world  that  requires  systems  thinking  to  take 
into  account  the  interdependent  relationships  at  work. 
Design  is  service  oriented,  a  creative  expression  of 
what  is  desired,  and  it  relies  on  relationships  between 
formalized  roles  among  the  participants  in  a  collabo¬ 
rative  social  system. 

Idealized  Design 

The  key  concept  behind  Ackoff’s  (1999)  idealized 
design  is  the  selection  of  ideals  to  create  an  ideal  vision 
of  what  the  new  system  should  be.  An  idealized  design 
should  be  technologically  feasible,  operationally  via¬ 
ble,  and  capable  of  rapid  learning  and  development.  It 
is  the  most  effective  system  of  which  the  designers  can 
conceive,  and  its  vision  should  be  shared  by  all  par¬ 
ticipants. 

Social  Systems  Design 

Banathy  (1996)  viewed  design  as  a  creative,  iterative, 
holistic,  decision-oriented  process  resulting  in  a  model 
of  a  new  system.  It  is  key  to  understand  that  designers 
must  transcend  current  approaches  and  solutions  to 
design  a  completely  new  model  of  a  system  appropri¬ 
ate  for  the  specific,  unique  context. 

Leveraged  Emergent  Approach 

Reigeluth's  (2006a)  leveraged  emergent  approach  pos¬ 
its  that  it  is  difficult  to  make  the  drastic  changes  to  a 
new  paradigm  all  at  once,  but  piecemeal  change  is 
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likely  to  be  unsuccessful;  therefore,  high-leverage 
structural  changes  should  be  implemented  that  can 
resist  the  pull  of  the  current  system  to  return  to  the 
status  quo  and  can  exert  leverage  to  change  the  rest  of 
the  system.  These  high-leverage  changes  will  be 
guided  by  a  few  principles,  and  the  remaining  changes 
will  emerge  over  time  as  the  need  for  them  becomes 
apparent  and  resources  become  available.  This 
approach  yields  frequent  visible  progress  to  sustain 
momentum  and  win  over  skeptics. 

Systems  Theory 

Systems  theory  was  established  in  the  mid-20th  cen¬ 
tury  by  a  multidisciplinary  group  of  researchers  who 
shared  the  view  that  science  had  become  increasingly 
reductionist  and  the  various  disciplines  isolated.  Ber- 
talanffy  (1968)  was  among  the  first  to  establish  a  gen¬ 
eral  systems  theory,  which  noted  the  existence  of  prin¬ 
ciples  and  laws  that  could  be  generalized  across 
systems  and  their  components  regardless  of  the  type 
of  system  or  its  relationship  to  other  systems. 

The  term  system  has  been  defined  in  various  ways 
by  different  researchers,  but  the  core  concept  is  one  of 
relationships  among  components  comprising  the 
whole.  Ultimately,  systems  theory  is  an  interdiscipli¬ 
nary  held  with  applications  in  both  the  hard  and  soft 
sciences.  Hard  systems  thinking  is  appropriate  for 
closed,  engineered  systems,  while  soft  systems  think¬ 
ing  is  appropriate  for  the  complexities  of  social  sys¬ 
tems  (Cheekland  and  Scholes,  1990).  Nelson  and  Stol- 
terman  (2003)  argued  that  there  are  no  set  types  or 
categories  of  systems;  instead,  the  view  of  a  system  is 
a  matter  of  perspective  and  choice. 

Flood  (1990)  took  this  viewpoint  even  further  in 
his  liberating  systems  theory  (LST)  by  firmly  focusing 
systems  theory  with  a  critical  viewpoint.  LST  is  related 
to  critical  systems  theory,  which  draws  from  Habermas 
(1973)  and  seeks  emancipation  of  humans  in  systems 
that  promote  subjugation  and  dominance  (Flood, 
1990).  LST  uses  a  post-positivist  approach  to  analyze 
social  conditions  to  liberate  the  oppressed  while  also 
seeking  to  liberate  systems  theory  from  tendencies 
such  as  self-imposed  insularity,  cases  of  internal  local¬ 
ized  subjugations  in  discourse,  and  liberation  of  sys¬ 
tem  concepts  from  the  inadequacies  of  objectivist  and 
subjectivist  approaches  (Flood,  1990). 

Banathy  (1991)  applies  systems  theory  to  social 
systems  design  by  examining  the  design  of  educational 
systems.  He  suggested  examining  systems  through 
three  lenses:  a  “still  picture  lens,”  used  to  understand 
the  components  comprising  the  system  and  their  rela¬ 
tionships;  a  “motion  picture  lens,”  used  to  understand 
the  processes  and  dynamics  of  the  system;  and  a 


“bird’s-eye  view  lens  ”  to  understand  the  relationships 
between  the  system  and  its  peer  and  suprasystems 
(Banathy,  1992). 

Senge  (1990)  applied  systems  theory  to  organiza¬ 
tional  learning.  Systems  thinking  is  the  fifth  and  most 
important  of  five  disciplines  of  a  learning  organization, 
according  to  Senge.  He  suggested  that  learning  orga¬ 
nizations  help  their  members  to  view  the  organization 
as  a  complex  system  of  interrelated  parts,  rather  than 
as  isolated  departments. 


PRINCIPLES  OF  SYSTEMS  DESIGN 

Of  the  many  principles  of  systems  design,  the  ones 
described  in  this  section  include  systems  thinking, 
design  theory,  idealized  design,  broad  stakeholder 
involvement,  mindset  and  culture,  participatory  lead¬ 
ership,  shared  vision,  learning  organization,  and 
strange  attractors  and  leverage. 

Systems  Thinking 

An  important  principle  of  systems  design  is  a  systems 
thinking  approach  to  design.  Systems  thinking  is  a 
framework  for  seeing  patterns  and  interrelationships 
in  a  complex  organization  or  system.  Banathy ’s  (1992) 
three-lens  view  described  earlier  is  a  useful  framework 
for  examining  the  system  through  its  components,  its 
processes  and  relationships,  and  its  peer  systems  and 
suprasystems.  The  complexity  of  systems  can  be  over¬ 
whelming,  but  systems  thinking  makes  these  complex 
realities  more  manageable  while  still  retaining  a  true 
view  of  reality,  rather  than  the  fragmented  and  inac¬ 
curate  view  of  a  systematic  analysis. 

Design  Theory 

Also  important  is  the  recognition  that  systems  design 
is  about  design,  meaning  the  creation  of  a  new  system 
that  has  not  previously  existed  (Banathy,  1996;  Nel¬ 
son  and  Stolterman,  2003).  This  is  a  shift  to  an 
entirely  new  paradigm,  a  transcendence  of  the  current 
system  rather  than  a  piecemeal  approach  of  modify¬ 
ing  the  existing  system.  Furthermore,  the  design  pro¬ 
cess  should  be  holistic  and  iterative,  rather  than 
sequential  and  systematic  (Banathy,  1996;  Nelson 
and  Stolterman,  2003)  by  beginning  with  a  “rough 
sketch”  (Ackoff,  1999)  or  “fuzzy  image”  of  the  new 
system  and  proceeding  to  gradually  work  out  more 
detail  for  the  whole  new  system,  one  level  at  a  time 
(Banathy,  1991).  It  is  important  to  be  able  to  generate 
feedback  on  the  emerging  vision  or  model  of  the  new 
system  through  the  development  of  artifacts  and  the 
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implementation  of  accountability  (Banathy,  1996; 
Nelson  and  Stolterman,  2003;  Reigeluth,  2006a). 

Idealized  Design 

Related  is  the  principle  of  idealized  design.  The  result 
of  the  design  process  should  be  a  model  of  the  ideal 
system  (Ackoff,  1981;  Banathy,  1996).  This  ideal 
design  should  be  the  designer’s  best  expression  of  what 
is  desired  by  the  clients  or  stakeholders,  labeled 
“desiderata”  by  Nelson  and  Stolterman  (2003).  Design 
is  driven  by  hope  and  vision,  which  motivates  the 
participants  to  make  the  leap  to  a  new  paradigm,  and 
the  newly  created  system  should  meet  the  desired 
results  as  best  as  possible  (Banathy,  1996;  Nelson  and 
Stolterman,  2003). 

Broad  Stakeholder  Involvement 

An  important  principle  to  consider  in  systems  design 
is  the  involvement  of  those  who  will  be  affected  by 
any  changes  in  the  system.  This  is  an  ethical  issue,  but 
it  is  also  a  quality-of-design  issue.  The  different  stake¬ 
holders  bring  diverse  perspectives  to  the  process  of 
systemic  change.  Different  professions,  cultures, 
understandings,  and  skills  should  come  together  to 
create  a  rich  environment  for  design  decisions  in 
schools  and  other  organizations.  Hutchins  (1996) 
pointed  out  that,  because  systems  are  complex,  they 
usually  serve  multiple  purposes— often  changing  over 
time  and  in  conflict  with  each  other.  Because  of  the 
complex  nature  of  any  system,  each  stakeholder’s  per¬ 
spective  needs  to  be  heard  and  addressed  for  more 
positive  change  in  schools  to  occur. 

The  role  of  the  community  is  extremely  important 
in  organizational  change  as  well  (Jenlink  et  al.,  1998). 
A  system  does  not  exist  in  isolation,  and  the  commu¬ 
nity  impacts  the  schools  and  organizations  within  it. 
The  roles  of  higher  education  institutions  and  commu¬ 
nity  organizations  and  businesses  are  often  overlooked 
but  should  be  included  when  designing  a  new  para¬ 
digm  of  education.  Furthermore,  rather  than  just 
involving  stakeholders,  it  is  important  to  create  a  col¬ 
laborative  social  system  (Nelson  and  Stolterman, 
2003).  This  entails  developing  a  design  culture  where 
participants  become  knowledgeable  about  design, 
develop  design  competence,  and  empower  themselves 
(Banathy,  1996). 

Mindset  and  Culture 

Mindsets  or  mental  models  are  one  of  the  most  impor¬ 
tant  things  to  consider  in  systemic  change.  One  aspect 
of  mindset  is  the  nature  of  the  change  process.  Edu¬ 


cators  are  used  to  thinking  in  terms  of  piecemeal 
reforms  imposed  from  the  top  down.  A  successful 
systemic  change  process  requires  a  different  mindset, 
one  of  empowerment,  collaboration,  consensus  build¬ 
ing,  and  trust.  It  is  important  to  help  stakeholders 
evolve  their  mindsets  to  participate  effectively  in  the 
change  process  (Banathy,  1996;  Nelson  and  Stolter¬ 
man,  2003). 

The  other  aspect  of  mindset  entails  helping  par¬ 
ticipants  to  shift  their  mindsets  from  the  standardized, 
time-based,  industrial-age  paradigm  of  training  or 
education  to  the  customized,  attainment -based,  infor¬ 
mation-age  paradigm,  and  from  piecemeal  change  to 
systemic  change.  Senge  (1994)  noted  that  good,  new 
ideas  rarely  get  put  into  practice,  often  because  these 
new  ideas  conflict  with  participants’  subconscious, 
internal  images  of  the  world.  If  stakeholders  are  oper¬ 
ating  on  different  mental  models  of  what  the  designed 
system  should  be  or  how  the  process  for  creating  that 
system  should  be  enacted,  then  they  will  resist  or 
oppose  the  changes  and  perpetuate  the  existing  par¬ 
adigm. 

Culture  for  change  can  be  viewed  as  the  collective 
mindsets  of  participants  in  the  change  process.  A  cul¬ 
ture  of  collaboration,  consensus-building,  empower¬ 
ment,  and  trust  among  the  stakeholders  is  crucial  for 
true  paradigm  change  to  succeed  in  an  organization. 
The  roles  of  leaders  (such  as  principals,  school  district 
administrators,  or  CEOs)  are  particularly  important. 
To  move  forward  in  the  beginning  of  a  systemic  change 
process,  leaders  must  establish  the  culture  of  the  sys¬ 
tem  and  set  examples  by  taking  the  courageous  first 
steps. 

Participatory  Leadership 

Leaders  of  the  change  process  have  big  responsibilities 
in  systemic  change.  They  must  not  only  evolve  their 
own  mindsets  and  guide  the  change  process  but  also 
involve  other  stakeholders  in  the  decision-making  pro¬ 
cess  and  share  responsibilities.  Most  leaders  have  expe¬ 
rienced  the  authority -centered  approach  of  leadership 
in  systems;  however,  leaders  in  systemic  change  pro¬ 
cesses  need  a  different  paradigm  of  leadership.  Leaders 
must  be  open  to  new  ideas,  be  self  aware,  and  pay 
individual  attention  to  followers  to  empower  them, 
help  them  grow,  and  stimulate  their  intellectual  abilities 
(Duffy  et  al.,  2000;  Senge,  1994).  Furthermore,  it  is 
important  for  leaders  to  be  actively  involved  in  the 
change  process.  A  design  team  must  have  the  full 
authority  it  needs  to  design  and  implement  changes. 
Without  the  support  of  the  top  leaders  in  a  system, 
design  decisions  will  always  run  the  risk  of  being  over¬ 
turned;  it  is  therefore  important  to  have  the  top  leaders 
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as  members  of  the  design  team  (Nelson  and  Stolter- 
man,  2003).  Moving  beyond  the  traditional  top-down 
approach  of  most  organizations  and  the  “principal-do- 
right”  (Senge,  2000)  leadership  model  in  schools  is 
important  to  achieve  paradigm  change  in  a  system. 

Shared  Vision 

Shared  vision  is  another  vital  notion  in  systemic 
change.  There  must  be  values  and  beliefs  that  indi¬ 
vidual  stakeholders  in  the  school  system  come  to 
share  to  change  the  paradigm.  Senge  (1994)  pointed 
out  that  no  organization  becomes  great  without  goals, 
values,  and  a  mission  that  become  shared  throughout 
the  organization.  A  vision  statement  or  the  leader’s 
charisma  cannot  be  enough.  Shared  vision  is  funda¬ 
mental  for  learning  organizations  that  want  to  help 
their  employees  to  fundamentally  change  their  work. 
The  overarching  vision  establishes  not  just  commit¬ 
ment  but  also  new  ways  of  thinking  and  acting  and 
consequently  fosters  risk  taking  and  experimenting 
in  the  organization  (Senge,  1994).  This  shared  vision 
is  related  to  the  earlier  discussion  of  a  shared  ideal¬ 
ized  design. 

Learning  Organizations 

Educational  systems  also  must  transform  into  learning 
organizations  to  succeed  in  systemic  change.  Accord¬ 
ing  to  Senge  (1994),  a  learning  organization  requires 
its  members  to  acquire  competency  in  five  disciplines: 
systems  thinking,  personal  mastery,  mental  models, 
shared  vision,  and  team  learning.  These  disciplines  are 
provided  to  help  the  members  and  teams  of  the  orga¬ 
nization  in  shifting  their  mental  models  to  understand 
their  system  as  a  whole  rather  than  as  parts  and  con¬ 
sequently  to  move  toward  a  shared  vision.  Further¬ 
more,  they  must  become  design  knowledgeable  and 
competent  to  fully  participate  in  the  design  process 
(Banathy,  1996).  These  disciplines  also  consider  all 
members  as  contributors  to  their  own  personal  growth 
as  well  as  the  growth  of  their  organization  and  team 
as  a  whole  (Senge,  1994). 

Strange  Attractors  and  Leverage  Points 

Fractals  and  strange  attractors  are  another  important 
notion  in  the  systemic  change  process.  Fractals  are 
patterns  that  recur  at  all  levels  of  a  system  (Wheatley, 
1999),  which  in  education  are  core  ideas,  values,  and 
beliefs  (Banathy,  1991;  Reigeluth,  2006b,  2007). 
Strange  attractors  are  a  kind  of  fractal  that  has  a 
powerful  influence  over  the  processes  and  structures 


that  emerge  during  transformation  (Wheatley,  1999). 
These  are  similar  to  memes ,  which  are  the  social 
counterpoints  to  genes  in  the  physical  organism  and 
have  the  power  to  organize  a  system  in  a  specific  way 
(Caine  and  Caine,  1997).  These  strange  attractors 
become  essential  in  the  mindsets  or  mental  models 
held  by  a  critical  mass  of  participants;  hence,  they 
are  an  essential  part  of  the  culture  of  the  organization 
(Reigeluth,  2007). 

Leverage  points  are  important  in  terms  of  the  effi¬ 
ciency  and  effectiveness  of  the  change  process.  They 
are  related  to  Senge’s  (1994)  notion  of  small  changes 
being  capable  of  producing  big  results.  They  are  cer¬ 
tain  elements  of  the  system  that  have  a  large  impact 
in  the  entire  organization  (Reigeluth,  2006a).  For  the 
systemic  change  process  to  happen  more  quickly  yet 
still  effectively,  it  is  important  to  identify  the  leverage 
points  in  the  school  system;  however,  Senge  (1994) 
also  talked  about  how  the  areas  of  highest  leverage  are 
often  the  least  obvious.  Investigating  school  systems 
and  organizations  to  identify  leverage  points  is  a  cru¬ 
cial  step  in  systemic  change. 


SYSTEMS  DESIGN  MODELS 

This  chapter  has  described  systems  theory  and  design 
theory  as  the  foundations  of  systems  design  and  pre¬ 
sented  systems  design  principles  to  guide  the  process 
of  designing  educational  and  training  systems.  This 
final  section  presents  an  overview  of  major  systems 
design  models  in  the  literature  .  These  models  present 
specific,  elaborate  processes  for  designing  education 
and  training  systems.  These  models  include  Jenlink  et 
al.’s  Guidance  System  for  Transforming  Education 
(GSTE),  Duffy’s  Step-Up-to-Excellence,  Schlechty’s 
guidelines  for  leadership  in  school  reform.  Hammer 
and  Champy’s  business  process  redesign/reengineer¬ 
ing,  and  Ackoff’s  idealized  systems  design. 

Guidance  System  for  Transforming  Education 

The  Guidance  System  for  Transforming  Education 
(GSTE)  (Jenlink  et  al.,  1996,  1998)  is  a  process  model 
for  facilitating  systemic  change.  The  GSTE  was 
designed  to  provide  process  guidelines  to  a  facilitator 
engaging  in  a  K-12,  districtwide  systemic  change 
effort.  The  GSTE  does  not  provide  any  suggestion  of 
what  changes  should  be  made  in  the  school  district; 
rather,  it  provides  the  facilitators  with  process  guide¬ 
lines  to  help  the  school  district  and  its  community 
make  decisions  about  what  changes  should  be  made. 
The  GSTE  is  comprised  of: 
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•  Core  values  about  the  change  process 

•  Discrete  events  (a  chronological  series  of 
activities  for  engaging  throughout  much  or 
all  of  the  change  process) 

•  Continuous  events  (activities  that  must  be 
addressed  continuously  throughout  much  or 
all  of  the  change  process) 


TABLE  52.1 

Revised  Discrete  Events  in  the  GSTE 

Phase  I.  Initiate  a  systemic  change  Facilitators  assess  and  enhance  their  own  readiness  for  the  process  and  form  a  support  team, 
effort  Facilitators  establish  or  redefine  a  relationship  with  a  school  district  and  discuss  per  diem  payment 

for  Event  3 . 

Facilitators  assess  and  enhance  district  readiness  for  change. 

Negotiate  and  sign  a  contract/agreement  with  the  superintendent  and  board  for  Phase  II. 

Phase  II.  Develop  starter  team  Facilitators  and  superintendent  form  the  starter  team. 

Hold  a  retreat  to  develop  the  starter  team  dynamic. 

Develop  starter  team  understanding  of  systems,  design,  mental  models,  systemic  change  process, 
dialog,  and  small-group  facilitation. 

Assess  and  enhance  district  and  community  capacity  for  change  (identify  assets  and  barriers,  and 
use  community  forums  if  needed). 

Develop  an  agreement/contract  with  the  starter  team  and  school  board  for  Phase  III,  scope  out 
resource  needs,  and  plan  a  budget  for  internal  funding  and  a  proposal  for  external  funding. 

Phase  III.  Develop  the  districtwide  Starter  team  expands  into  the  leadership  team;  starter  team  becomes  facilitators;  facilitator 
framework  and  capacity  for  change  becomes  an  advisor  and  “critical  friend.” 

Hold  a  one-day  retreat  to  develop  the  leadership  team  dynamic. 

Facilitators  develop  leadership  team  understanding  of  systems,  design,  mental  models,  systemic 
change  process,  dialog,  and  small-group  facilitation  (address  throughout  Events  13  to  17). 
Leadership  team  develops  a  district-wide  framework  with  broad  stakeholder  participation 
(community  forums).  This  includes  identifying  changes  in  the  community’s  educational  needs 
and  using  them  to  develop  a  mission,  vision,  and  core  values  for  an  ideal  school  system.  It  takes 
this  opportunity  to  assess  and  enhance  district  and  community  interest  in,  and  culture  for, 
systemic  change.  It  develops  pyramid  groups  for  broad  stakeholder  involvement. 

Leadership  team  identifies  current  and  recent  change  efforts  and  decides  what  relation  those  should 
have  with  this  effort. 

Leadership  team  develops  a  change  process  strategy,  including  capacity  building  and  funding; 
advisor’s  role  is  defined  and  funded  for  Phase  IV. 

Leadership  team  forms  and  capacitates  building-level  design  teams  and  conducts  a  workshop  on 
the  framework. 

Design  teams  create  building-level  designs  and  systems  for  evaluating  those  designs  with  broad 
stakeholder  involvement;  leadership  team  supports  and  monitors  the  design  teams. 

Leadership  team  forms  and  capacitates  a  district-level  design  team. 

Design  team  creates  a  design  for  ideal  district  administrative  and  governance  systems  and  systems 
for  evaluating  that  design,  with  broad  stakeholder  involvement;  leadership  team  supports  and 
monitors  this  design  team. 

Design  teams  create  building-level  processes  for  evolving  as  close  as  possible  to  their  ideal 
designs;  leadership  team  supports  and  monitors  the  design  teams. 

Carry  out  implementation  plans,  formative  evaluations,  and  revisions  of  the  evolving  designs  and 
the  implementation  processes. 

Source:  Reigeluth,  C.M.  and  Duffy,  F.M.,  in  Trends  and  Issues  in  Instructional  Design  and  Technology,  2nd  ed..  Reiser,  R.A.  and 
Dempsey,  J.V.,  Eds.,  Prentice  Hall,  Upper  Saddle  River,  NJ,  2007.  With  permission. 


Phase  IV.  Create  ideal  designs  for  a 
new  educational  system 


Phase  IV.  Create  ideal  designs  for  a 
new  educational  system 


The  GSTE,  originally  developed  by  Jenlink  et  al.  (1996, 
1998),  has  undergone  additional  development  based  on 
Reigeluth's  experience  using  it  in  the  Metropolitan 
School  District  of  Decatur  Township  in  Indianapolis, 
Indiana.  The  discrete  events  listed  in  Table  52.1  reflect 
these  tentative  revisions  of  the  GSTE.  These  events  are 
guided  by  underlying  principles  and  suggested  activi- 
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ties  that  help  one  to  understand  and  engage  in  them. 
The  core  values  underlying  the  GSTE  are: 

•  Caring  for  children 

•  Co-evolution 

•  Collaboration 

•  Common  language 

•  Community 

•  Context 

•  Conversation 

•  Culture 

•  Democracy 

•  Evolution  of  mindsets 

•  Facilitator 

•  Ideal  vision 

•  Inclusiveness 

•  Participant  commitment 

•  Process  orientation 

•  Readiness 

•  Respect 

•  Responsibility 

•  Space 

•  Stakeholder  empowerment 

•  Systemic  thinking 

•  Time 

•  Wholeness 

Step-Up-to-Excellence 

Step-Up- to-Excellence  (SUTE)  (Duffy,  2002,  2003, 
2004, 2006)  is  a  process  methodology  designed  to  help 
change  leaders  in  school  districts  create  and  sustain 
whole-district  improvement.  This  methodology  com¬ 
bines  effective  tools  for  school  system  transformation. 
SUTE,  a  three-step  process  that  is  preceded  by  a  pre- 
launch  preparation  phase,  proceeds  as  follows. 

Pre-launch  preparation  phase 

Step  1:  Redesign  the  entire  school  system 

Step  2:  Create  strategic  alignment 

Step  3:  Evaluate  whole-system  performance 

Recycle  to  the  next  pre-launch  preparation  phase 

Pre-Launch  Preparation  Phase 

The  pre-launch  preparation  activities  are  carried  out 
by  the  superintendent  of  schools  and  a  couple  of  hand¬ 
picked  subordinates.  This  small  team  is  temporary,  and 
it  will  not  lead  the  transformation;  its  purpose  is  to 
prepare  the  system  to  engage  in  systemic  change.  If 
the  decision  is  made  that  the  transformation  effort  is 
to  be  launched,  then  the  activities  are  transferred  to  a 
strategic  leadership  team ,  which  is  composed  of  the 
superintendent,  teachers,  and  building  administrators. 


The  team  also  appoints  a  transformation  coordinator. 
The  process  then  proceeds  to  Step  1 . 

Step  1.  Redesign  the  Entire  School  System 

During  Step  1,  the  district’s  core  purpose,  mission, 
values,  and  goals  are  defined.  Educators  then  work  in 
small  teams  within  each  cluster  of  schools  (all  those 
schools  that  feed  into  a  single  high  school,  plus  that 
high  school)  to  redesign  their  entire  school  district  by 
improving  three  areas:  their  district’s  core  and  support¬ 
ing  work  processes,  its  internal  social  architecture,  and 
its  relationship  with  its  environment. 

Step  2.  Create  Strategic  Alignment 

After  the  redesign  process  of  a  district,  change  leaders 
invite  educators  to  align  their  individual  work  with  the 
goals  of  their  teams,  the  work  of  teams  with  the  goals 
of  their  schools ,  the  work  of  schools  with  the  goals  of 
their  clusters,  and  the  work  of  clusters  with  the  goals 
of  the  district.  Creation  of  strategic  alignment  ensures 
everyone  will  work  systemically  toward  the  same  goals 
and  vision  of  the  district.  It  also  determines  responsi¬ 
bilities  for  all  stakeholders  involved  in  a  child’s  edu¬ 
cational  experience  and  frees  the  district  from  unnec¬ 
essary  bureaucratic  hassles,  dysfunctional  policies,  and 
obstructionist  procedures  that  limit  effectiveness. 

Step  3.  Evaluate  Whole-System  Performance 

During  Step  3,  change  leaders  evaluate  the  perfor¬ 
mance  of  the  clusters,  schools,  and  teams  of  the  dis¬ 
trict.  The  purpose  of  this  evaluation  is  to  measure  the 
success  of  the  district’s  effort.  Evaluation  is  reported 
to  stakeholders  and  is  used  to  sustain  school  district 
improvement  by  managing  the  district’s  performance. 

Recycle  to  the  Next  Pre-Launch  Preparation  Phase 

After  a  predetermined  period,  the  district  steps  up 
again  by  cycling  back  to  the  pre-launch  preparation 
phase  for  further  improvement  and  transformation. 

Schlechty  Center’s  Guidelines 
for  Leadership  in  School  Reform 

The  Schlechty  Center  for  Leadership  in  School  Reform 
has  been  engaging  in  school  district  transformation  pro¬ 
cesses  from  a  comprehensive  and  systemic  approach  to 
school  reform  (Solomon,  2006).  Theories  and  frame¬ 
works  that  provide  guidelines  for  the  activities  of  the 
school  reform  processes  in  the  districts  and  schools  with 
which  the  Schlechty  Center  works  are  provided  below. 
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Two  Theories  Underlying  the 
Activities  of  the  Schlechty  Center 

The  Theory  of  Change 

The  theory  of  change  (Christensen,  1997)  focuses  on 
transforming  schools  from  organizations  based  on  the 
hypothesis  that  the  central  business  of  schools  has  to 
do  with  producing  conformity  and  attendance  to  orga¬ 
nizations  where  the  central  work  focuses  on  nurturing 
attention  and  commitment  towards  students.  The  the¬ 
ory  of  change  is  the  basis  for  the  Ten  District  Stan¬ 
dards ,  an  underlying  framework  of  the  Schlechty  Cen¬ 
ter  which  is  described  in  the  next  section. 

The  Theory  of  Engagement 

The  theory  of  engagement  focuses  on  student  motiva¬ 
tion  and  the  strategies  that  are  needed  to  increase  the 
perspective  that  schools  and  teachers  will  enlarge  the 
presence  of  engaging  tasks  and  activities  in  the  every¬ 
day  life  of  the  schools.  The  theory  of  engagement  is 
the  basis  of  the  Working  on  the  Work  framework 
(Schlechty,  2002),  described  below. 

Two  Frameworks  Underlying  the 
Activities  of  the  Schlechty  Center 

Ten  District  Standards 

The  Ten  District  Standards  framework  helps  leaders 
assess  and  build  system  capacity  so  the  entire  district 
is  aligned  and  focused  on  the  core  purpose  of  schools 
by: 

•  Developing  a  shared  understanding  of  the 
need  for  change 

•  Developing  shared  beliefs  and  vision 

•  Developing  a  focus  on  students  and  on  the 
quality  of  work  provided  to  students 

•  Developing  structures  for  participatory  lead¬ 
ership 

•  Developing  structures  for  results -oriented 
decision  making 

•  Developing  structures  for  continuity 

•  Providing  ongoing  support 

•  Fostering  innovation  and  flexibility 

•  Employing  technology 

•  Fostering  collaboration 

Working  on  the  Work 

The  Working  on  the  Work  framework  calls  on  every¬ 
one  to  provide  challenging  and  engaging  work  for 
students  that  results  in  students  learning  what  schools , 
parents,  and  the  community  want  them  to  learn: 


•  Work  that  is  challenging  to  students 

•  Work  with  which  students  persist  when  they 
experience  difficulty 

•  Work  from  which  students  gain  a  sense  of 
satisfaction 

Hammer  and  Champy’s  Business 
Process  Redesign/Reengineering 

Hammer  and  Champy  (1993,  2003)  defined  the  pro¬ 
cess  of  reengineering  a  corporation  as  the  “fundamen¬ 
tal  rethinking  and  radical  redesign  of  business  pro¬ 
cesses  to  achieve  dramatic  improvements  in  critical, 
contemporary  measures  of  performance,  such  as  cost, 
quality,  service  and  speed”  (p.  32).  They  provided  a 
systemic  change  design  model  for  organizational 
transformation  in  corporate  sectors  with  their  business 
process  redesign! reengineering  (BPR)  approach.  BPR 
is  a  management  approach  that  examines  aspects  of  a 
corporation  or  business  and  its  interactions  and 
attempts  to  advance  the  competence  of  the  underlying 
processes.  It  is  a  systemic  and  fundamental  approach 
that  redesigns  the  core  work  processes  and  either  mod¬ 
ifies  or  eliminates  activities  that  are  not  producing 
value  in  the  corporation.  Hammer  and  Champy  (1993, 
2003)  argued  that  far  too  much  time  is  wasted  by 
businesses  that  pass  on  tasks  from  one  department  to 
the  other,  and  they  claimed  that  it  is  much  more  impor¬ 
tant  to  build  a  team  that  can  perform  all  tasks  in  the 
process.  They  identified  four  main  themes  that  accom¬ 
pany  reengineering: 

•  Fundamentals—  Focusing  on  the  fundamen¬ 
tals  allows  a  look  at  tacit  principles  and 
assumptions;  once  these  rules  and  assump¬ 
tions  are  identified,  they  should  be  rede¬ 
signed  for  an  entirely  new  system. 

•  Radical— Organizational  change  must  not 
engage  in  piecemeal  changes;  systemic 
change  is  needed. 

•  Dramatic— There  should  be  big  leaps  in  per¬ 
formance;  reengineering  is  not  for  marginal 
changes. 

•  Processes— Organizational  changes  must  be 
process  oriented  and  must  not  be  focused  on 
individual  tasks. 

Common  Themes  in  Reengineering  Efforts 

Although  reengineering  processes  take  various  forms, 
similarities  exist.  Hammer  and  Champy  (1993,  2003) 
identified  nine  common  themes  to  reengineering 
efforts: 
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•  Steps  in  the  process  are  simplified  and  sev¬ 
eral  jobs  are  combined  into  one. 

•  Workers  make  decisions,  which  eliminates 
the  hierarchy  in  decision  making. 

•  Steps  or  processes  are  performed  in  a  natural 
order. 

•  Processes  are  not  standardized;  they  have 
multiple  versions. 

•  Work  occurs  where  it  makes  the  most  sense, 
sometimes  by  customers  or  suppliers. 

•  Checks  and  controls  are  reduced  to  reduce 
costs. 

•  Reconciliation  is  minimized  to  encourage 
consistency. 

•  A  case  manager  provides  a  single  point  of 
contact  with  customers. 

•  Centralization  and  decentralization  are  one 
process. 

The  Change  Process  in  Reengineering  Efforts 

Hammer  and  Champy  (1993,  2003)  also  provided 
change  process  guidelines  for  reengineering  efforts: 

•  Identify  and  map  the  processes  using  process 
mapping. 

•  Identify  the  process  requiring  reengineering. 

•  Achieve  high-level  understanding  of  the  cur¬ 
rent  process  from  a  customer  perspective. 

•  Look  outside  the  process  to  customer  needs 
by  observing  performance. 

•  Look  at  the  process  itself  by  observing  per¬ 
formance. 

•  Understand  what  is  critical  in  the  process. 

•  Consider  feasibility,  such  as  scope,  cost,  and 
commitment. 

•  Designate  the  process  owner  and  form  a  pro¬ 
cess  team. 

Ackoff’s  Idealized  Systems  Design 

Based  on  the  theoretical  foundation  of  systems  theory 
and  the  systems  view  of  the  world  and  organizations, 
Ackoff’s  (1999)  idealized  systems  design  approach 
identifies  design  strategies  and  implementation  plan¬ 
ning  processes  for  organizational  change  and  the  rec¬ 
reation  of  business  corporations.  The  process  of  creat¬ 
ing  the  idealized  systems  design  includes  the  following 
stages: 

•  Formulating  the  mess  (sensing  and  making 
sense  of  the  situation) 

•  Ends  planning  (where  to  go) 

•  Means  planning  (how  to  get  there) 


•  Resource  planning  (what  is  needed  to  get 
there) 

•  Implementation  and  control  (doing  it  and 
learning) 

The  design  comes  forth  with  situational  analysis, 
which  is  a  systemic  understanding  and  detailed  eval¬ 
uation  of  the  current  state  of  the  organization  and  its 
environment.  Ackoff  (1999)  suggested  that,  upon 
achieving  systemic  comprehensive  understanding  of 
the  current  system,  the  system  should  progress  to  the 
idealized  design  stage. 

The  ends  of  an  organization  consist  of  ideals, 
objectives,  and  goals.  Ends  planning  should  be 
directed  to  make  explicit  exactly  what  is  wanted  in  the 
organization.  Ackoff  (1999)  emphasized  that  the  vision 
of  the  ideal  system  of  an  organization  must  be  a  shared 
image  among  the  stakeholders  and  that  it  should  be 
created  by  all  stakeholders  in  the  system.  The  three 
characteristics  of  an  idealized  design  are  that  it  should 
be  technologically  feasible,  operationally  viable,  and 
capable  of  being  improved  continuously. 

Having  a  shared  ideal  vision  of  the  system,  system 
designers  then  engage  in  an  implementation  planning 
process.  The  means  that  the  planning  stage  selects 
creates  the  means  that  will  help  achieve  the  ends.  First, 
designers  engage  in  the  design  of  the  means  planning. 
Means  planning  determines  the  gaps  between  the  cur¬ 
rent  and  ideal  systems  and  constitutes  a  set  of  instruc¬ 
tions  that  enable  the  possible  realization  of  the  vision. 
These  means  include  acts,  practices,  processes, 
projects,  programs,  and  policies  of  the  system. 

The  means  planning  determines  what  kinds  of 
resources  are  required  for  the  implementation  and  fur¬ 
ther  requires  determining  the  allocation  of  these 
resources.  In  addition,  appointing  human  resources  is 
required  in  this  phase:  who  is  doing  what,  when,  how, 
and  where  in  the  system.  The  next  step  is  to  plan  the 
allocation  of  the  resources  that  the  means  require. 
Money,  capital  goods,  people,  consumables,  data, 
information,  and  knowledge  are  all  involved.  Finally, 
the  designers  formulate  the  design  of  a  management 
learning  and  adaptation  system  that  will  aid  in  the 
realization  of  these  requirements. 


CONCLUSION 

This  chapter  reviewed  the  current  research  on  systems 
design  for  change  in  education  and  training.  It  argued 
for  the  need  for  systemic  change  to  create  a  completely 
new  paradigm  suitable  for  today’s  information  age.  It 
presented  a  description  of  design  theory  and  systems 
theory  as  the  foundations  for  systems  design.  Principles 
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of  systems  design  were  offered,  as  well  as  several  sys¬ 
tems  design  models.  Despite  a  history  of  systems  design 
research  and  an  increasing  call  for  systemic  change 
efforts  over  the  last  several  decades,  most  change  efforts 
continue  to  be  piecemeal,  tinkering  with  or  revising  the 
currently  used  yet  horribly  outdated  paradigm  of  edu¬ 
cation  and  training.  This  chapter  has  summarized  a 
foundation  for  better  understanding  systems  design  and 
how  it  can  be  applied  to  training  and  education. 
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Methodological  Issues 


This  final  part  of  the  Handbook  was  led  by  Jeroen  J.G. 
van  Merrienboer  and  focuses  on  research  issues.  Rather 
than  have  separate  and  arbitrary  chapters  on  quantitative, 
qualitative,  and  other  research  approaches,  this  part  fol¬ 
lows  the  research  cycle  through  theory  development, 
experimental  design,  and  data  collection  and  analysis. 
Design  sections  adhere  to  the  main  parts  of  the  Hand¬ 
book  and  treat  research  on  strategies,  technologies,  mod¬ 
els,  and  design  and  development.  Analysis  methods 
include  the  analysis  of  learning  processes,  interactions, 


and  complex  performances.  This  part  ends  with  a  pro¬ 
vocative  discussion  that  is  intended  to  help  our  field  build 
a  strong  scientific  foundation  for  the  future  .  This  part  of 
the  Handbook  consists  of  four  chapters  covering  these 
topics:  (1)  theory  development,  (2)  research  design,  (3) 
data  collection  and  analysis,  and  (4)  foundations  for  the 
future.  Several  of  these  chapters  contain  multiple  sec¬ 
tions  by  different  authors.  As  before,  all  authors  are 
recognized  as  chapter  authors  and  individual  contribu¬ 
tions  to  specific  sections  are  noted  within  the  chapter. 
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ABSTRACT 

The  field  of  educational  technology  is  characterized  by 
an  abundance  of  sets  of  theoretical  statements.  These 
sets  can  be  described  from  at  least  four  perspectives.  A 
first  perspective  pertains  to  the  object  or  orientation  of 
the  (sets  of)  theoretical  statements.  Sets  may  address 
educational  principles,  features  of  interventions  or  tools, 
development  procedures,  or  the  nature  of  the  field  itself. 
The  origin  (theoretical  or  practical  context)  of  these  sets 
of  statements  is  a  second  perspective  from  which  they 
can  be  described.  Theoretical  statements  can  be  clearly 
anchored  in  specific  practical  experiences,  can  be  the 
result  of  a  deductive  effort  to  apply  fundamental  theo¬ 


retical  insights  or  theoretical  perspectives,  or  can  reflect 
a  thoughtful  interaction  between  a  practical  setting  and 
a  theoretical  perspective.  A  third  perspective  relates  to 
the  level  of  theoretical  sophistication  of  these  (sets  of) 
statements.  Sets  of  theoretical  statements  can  be  collec¬ 
tions  of  isolated  theoretical  expressions,  descriptive  or 
prescriptive  models  with  respect  to  educational  techno¬ 
logical  issues,  or  integrated,  internally  coherent  sets  of 
theoretical  principles.  The  degree  of  justification  of 
these  (sets  of)  statements  represents  a  fourth  perspec¬ 
tive.  Some  statements  are  purely  explorative,  whereas 
for  others  empirical  evidence  is  readily  available.  Tak¬ 
ing  these  differences  into  account,  this  chapter  aims  at 
discussing  theory  development.  First,  features  of 
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descriptive  and  prescriptive  theories  are  elaborated  by 
analyzing  their  commonalities  and  differences.  Second, 
the  development  of  theories  itself  is  addressed  by 
focusing  on  deductive,  inductive,  and  mixed 
approaches.  Third,  theoretical  levels  of  sophistication 
and  justification  are  dealt  with.  In  all  this,  the  elabora¬ 
tion  of  instructional  design  models  is  focused  upon. 


KEYWORDS 

Instructional  design  model:  A  coherent  set  of  mostly 
prescriptive  theoretical  statements  on  the  appropri¬ 
ateness  of  particular  instructional  approaches  or 
interventions. 

Theoretical  statement:  Formal  expression  about  the 
relationship  between  at  least  two  variables  or 
instantiations  of  variables. 

Theory:  An  integrated  and  internally  coherent  set  of 
theoretical  statements  that  provides  a  sufficient 
basis  for  empirical  research  in  which  these  state¬ 
ments  can  be  tested. 

Theory  development:  Systematic  effort  to  generate 
(coherent  sets  of)  theoretical  statements. 


INTRODUCTION 

Educational  technology  and  more  specifically  instruc¬ 
tional  design  are  characterized  by  an  abundance  of  sets 
of  theoretical  statements.  Repeatedly  overviews  of  the 
plethora  of  theoretical  perspectives  and  insights  have 
been  presented  (Dijkstra  et  al.,  1997;  Dills  and  Romis- 
zowski,  1997;  Jonassen,  1982,  1985,  1996a,  2004a; 
Reigeluth,  1983a,  1999;  Seel  and  Dijkstra,  2003;  Ten¬ 
nyson  et  al.,  1997).  Theoretical  statements  typically 
are  formal  expressions  about  the  relationships  between 
at  least  two  variables  or  instantiations  of  variables. 
Theoretical  statements  in  educational  technology  gen¬ 
erally  make  such  claims  that  are  regarded  to  be  relevant 
for  educational  technology  or  instructional  design.  A 
theory,  then,  is  an  integrated  and  internally  coherent 
set  of  theoretical  principles  that  provides  a  sufficient 
basis  for  empirical  research  in  which  these  statements 
can  be  tested.  For  instance,  a  theoretical  statement  may 
explain  how  a  learner-related  variable  in  combination 
with  an  instructional  goal  is  linked  to  an  instructional 
strategy  or  how  different  activities  are  sequenced  in  a 
development  procedure.  An  educational  technological 
theory  consists  of  a  series  of  such  statements  that  are 
mutually  consistent  and  refer  to  one  another. 

In  this  chapter,  differences  between  theoretical  state¬ 
ments  are  addressed  by  presenting  four  perspectives 
from  which  such  statements  can  be  described.  Taking 


Figure  55.1  Four  perspectives  to  describe  (sets  of)  theoretical 
statements. 

these  differences  into  account  with  respect  to  sets  of 
theoretical  statements,  the  remainder  of  this  chapter  dis¬ 
cusses  theory  development  with  a  focus  on  instructional 
design.  Features  of  descriptive  and  prescriptive  theories 
are  elaborated  by  analyzing  their  commonalities  and 
differences.  The  development  of  theories  is  addressed 
by  focusing  on  deductive,  inductive,  and  mixed 
approaches.  Finally,  levels  of  theoretical  sophistication 
and  empirical  justification  are  dealt  with. 

THEORETICAL  STATEMENTS: 

FOUR  PERSPECTIVES 

Theoretical  statements  and  sets  of  such  statements  dif¬ 
fer  in  substantial  ways.  Their  similarities  and  differ¬ 
ences  can  be  identified  and  described  from  at  least  four 
perspectives:  object,  origin,  sophistication,  and  justifi¬ 
cation  (see  Figure  55.1).  The  absence  of  a  clear  and 
consolidated  theoretical  base  in  the  field  of  educational 
technology  can  be  shown  by  describing  (sets  of)  the¬ 
oretical  statements  from  these  four  perspectives. 

The  Object  Perspective 

Theoretical  statements  differ  as  to  what  the  statements 
are  all  about.  There  are  large  variations  in  the  object 
of  (sets  of)  theoretical  statements.  A  first  series  of  sets 
addresses  instruction-related  principles.  These  state¬ 
ments  bring  forward  what  is  regarded  to  be  important 
or  even  essential  for  learning  to  occur  in  an  instruc¬ 
tional  setting.  A  clear  example  is  Ausubel’s  (1968,  p. 
vi)  statement:  “If  I  had  to  reduce  all  of  educational 
psychology  to  one  principle,  I  would  say  this:  the  most 
important  single  factor  influencing  learning  is  what  the 
learner  already  knows.  Ascertain  this  and  teach  him 
accordingly.”  A  more  recent  example  comes  from  Mer¬ 
rill,  who,  in  an  attempt  to  consolidate  the  variety  of 
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theoretical  statements  in  the  educational  technology 
held,  postulated  five  first  principles  of  instruction,  such 
as  “learning  is  promoted  when  learners  are  engaged  in 
solving  real-world  problem”  (Merrill,  2002,  p.  45). 

A  second  series  addresses  issues  with  respect  to 
(technological)  features  of  instructional  interventions, 
tools,  or  devices.  Traditionally,  such  statements  mostly 
pertain  to  links  between  the  format  and  outlook  of 
information,  on  the  one  hand,  and  issues  of  accessibil¬ 
ity  and  usability,  on  the  other  (Fleming  and  Levie, 
1993).  Examples  are  statements  with  respect  to  typo¬ 
graphical  cues  in  texts  (Glynn  et  al.,  1985),  headings 
(Hartley  and  Jonassen,  1985),  contents  lists  and 
indexes  (Wright,  1985),  or  various  features  of  Web- 
based  materials  (Brooks,  1997)  that  may  facilitate 
access  to  information  in  printed  or  digital  materials. 
More  recently,  and  especially  under  the  label  of  mul¬ 
timedia  design,  such  statements  also  pertain  to  links 
between  the  format  and  outlook  of  information  and 
cognitive  issues  directly  related  to  learning.  A  clear 
example  is  the  multimedia  principle  “students  learn 
better  from  words  and  pictures  than  from  words  alone” 
(Mayer,  2001,  p.  63).  The  Cambridge  Handbook  of 
Multimedia  Learning  (Mayer,  2005)  compiled  a  large 
number  of  such  principles.  To  this  series  also  belong 
the  large  number  of  statements  about  how  to  use  tech¬ 
nological  devices  such  as  different  types  of  informa¬ 
tion  and  communication  technologies  (Alessi  and  Trol- 
lip,  2001;  Jonassen,  1996b;  Morrison  et  al.,  1999)  to 
foster  learning.  Currently,  e-learning  (Clark  and 
Mayer,  2003;  Joe  hems  et  al.,  2004)  is  generating  a 
large  number  of  such  theoretical  statements. 

In  a  third  series,  the  object  of  the  statements  is  the 
design  or  development  process.  These  statements  dis¬ 
cuss  what  procedures  or  instruments  are  best  suited  for 
what  kind  of  design  problem  (task  analysis  procedures; 
see  Jonassen  et  al.,  1999)  or  specify  how  activities  in 
a  design  or  development  process  relate  to  one  another 
(Andrews  and  Goodson,  1980;  Dick  et  al.,  2001; 
Gustafson  and  Branch,  1997;  Kemp  et  al.,  1994;  Lee 
and  Owens,  2000;  Rothwell  and  Kazanas,  1998). 

Finally,  theoretical  reflections  also  pertain  to 
instructional  design  itself:  the  nature  of  educational 
technology  or  instructional  design  (Clark  and  Estes, 
1998;  Heinich,  1984;  Van  Patten,  1989;  Wilson,  1997), 
their  development  (Clark  and  Estes,  1999;  Reigeluth, 
1983b;  Schott  and  Driscoll,  1997),  and  the  appropri¬ 
ateness  of  particular  research  approaches  (Reeves, 
2005;  Ross  and  Morrison,  2004)  are  regularly  dis¬ 
cussed.  Clark  and  Estes  (1999),  for  example  ,  pointed 
to  the  technological  nature  of  instructional  design, 
Schott  and  Driscoll  (1997)  stressed  the  need  to  focus 
on  efficiency,  and  Reeves  (2005)  argued  for  the  appli¬ 
cation  of  more  design-based  research  approaches . 


The  Origin  Perspective 

Theoretical  statements  can  also  be  described  by  explic¬ 
itly  considering  their  origin.  Three  different  origins  can 
be  easily  distinguished.  A  first  position  is  practice  ori¬ 
ented  and  practice  anchored.  Theoretical  statements  are 
rooted  in  practice  and  result  from  a  reflection  on  design 
and  development  practices.  Examples  are  theoretical 
conclusions  that  are  immediately  linked  to  particular 
design  or  development  projects  (Elen  and  Clarebout, 
2001;  Hannafin  and  Land,  1997).  Regularly,  such  the¬ 
oretical  statements  are  prescriptive.  They  specify  how 
in  a  particular  context  an  outcome  can  be  realized  by 
the  application  of  a  particular  method.  A  second  posi¬ 
tion  is  theory  driven  and  theory  oriented.  Theoretical 
statements  result  from  deliberate  efforts  to  deduce 
design  or  development  principles  from  more  fundamen¬ 
tal  theories  (e.g.,  learning  theory,  systems  theory,  activ¬ 
ity  theory)  or  specific  theoretical  perspectives  (Gros, 
2002;  Jonassen  and  Rohrer-Murphy,  1999;  Richey, 
1986).  A  more  general  theory  is  applied  to  a  specific 
problem  or  context  to  generate  more  specific  theoretical 
statements.  Whereas  classical  examples  are  contribu¬ 
tions  of  Skinner  (1968)  and  Gagne  (1985),  a  more 
recent  example  is  van  Merrienboer’s  (1997)  four-com¬ 
ponent  instructional  design  (4C/ID)  model.  Most  typi¬ 
cal,  however,  is  a  third  position  in  which  a  local  instruc¬ 
tional  design  theory  is  developed  by  applying  a  specific 
theoretical  perspective  to  a  specific  instructional  design 
issue.  Using  a  specific  instructional  problem,  interven¬ 
tion,  or  medium  as  an  instructional  design  anchor  point 
(Elen,  2004),  an  attempt  is  made  to  construct  a  consis¬ 
tent  set  of  theoretical  statements  that  directly  apply  to 
the  selected  problem,  such  as  solving  problems  (Jonas¬ 
sen,  2004b).  The  statements  may  also  pertain  to  the  use 
of  instructional  interventions  (Elen  and  Clarebout, 
2006);  interventions,  such  as  problem-based  learning 
(Schmidt,  1993)  or  metacognitive  prompts  (Bannert, 
2006);  or  media,  such  as  e-leaming  (Jochems  et  al., 
2004)  or  learning  objects  (Wiley,  2002). 

The  Sophistication  Perspective 

Theoretical  statements  also  differ  with  respect  to  their 
theoretical  sophistication  and  empirical  justification. 
The  level  of  theoretical  sophistication  and  empirical 
justification  of  these  (sets  of)  statements  clearly  differs. 
In  a  number  of  cases  the  (sets  of)  theoretical  statements 
are  simple  collections  of  isolated  theoretical  expres¬ 
sions;  for  example,  in  the  conclusion  section  of  an 
empirical  study  a  number  of  implications  for  instruc¬ 
tional  design  are  mentioned.  Typical  for  instructional 
design,  however,  seems  to  be  the  abundance  of  descrip¬ 
tive  or  prescriptive  models  with  respect  to  educational 
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technology  issues  (Reigeluth,  1983a,  1999).  Addition¬ 
ally,  there  is  also  a  small  set  of  attempts  to  really  present 
an  educational  technological  theory  consisting  of  inte¬ 
grated,  internally  coherent  sets  of  theoretical  principles 
which  provide  a  sufficient  basis  for  empirical  research 
in  which  these  statements  can  be  tested.  The  need  for 
such  integrated,  internally  coherent  sets  becomes  very 
clear  when  attempts  are  made  to  automate  instructional 
design  (Spector  et  al.,  1993).  The  need  to  formulate 
precise  statements  that  are  also  mutually  consistent  and 
coherent  helps  them  to  become  theoretically  more 
explicit  and  sophisticated  (Elen,  1998). 

The  Justification  Perspective 

A  different  aspect  relates  to  the  empirical  validation 
of  the  statements.  Some  statements  are  purely  explor¬ 
ative  or  speculative,  whereas  for  others  empirical  evi¬ 
dence  is  readily  available.  This  empirical  evidence  may 
have  been  gathered  through  a  variety  of  both  quanti¬ 
tative  and  qualitative  methods  (e.g.,  experimental  stud¬ 
ies,  usability  and  feasibility  studies,  design  experi¬ 
ments,  case  studies,  development  research). 
Theoretical  sophistication  and  empirical  justification 
do  not  have  to  go  along;  for  example,  whereas  Merrill 
(2002)  claimed  the  theoretical  soundness  of  his  inter¬ 
related  set  of  first  principles  of  instruction,  he  also 
specified  the  need  to  test  the  empirical  validity  of  these 
statements  (Merrill,  2006). 


DESCRIPTIVE  AND 
PRESCRIPTIVE  THEORIES 

The  educational  technology  field  and  the  instructional 
design  field  are  areas  of  practice  as  well  as  areas  of 
scientific  inquiry.  Research  efforts  in  the  field  of 
instructional  design  are  twofold.  A  first  series  of  efforts 
and  approaches  aims  at  describing  instructional  phe¬ 
nomena.  Why  a  particular  intervention  is  suitable  for 
a  group  of  learners  or  how  design  processes  are  actu¬ 
ally  practiced  is  described  and  explained.  When  the 
object  relates  to  how  instructional  interventions  or 
approaches  affect  learning  processes  of  particular 
groups  of  learners  (for  collaborative  learning,  see  Lit¬ 
tleton  and  Hakkinen,  1999),  these  endeavors  may 
result  in  the  specification  of  an  instructional  theory 
(Reigeluth,  1983b).  When  the  process  is  focused  upon, 
the  descriptions  mainly  pertain  to  the  very  different 
context-specific  ways  in  which  design  and  develop¬ 
ment  procedures  and  processes  develop  (Richey  et  al., 
2004)  and  to  the  difficulties  designers  experience  to 
implement  well-defined  and  linear  design  procedures 
(Gibbons,  2003;  Rowland,  1993). 


A  second  series  of  research  efforts  aims  at  identi¬ 
fying  what  instructional  interventions  or  approaches 
should  look  like  in  view  of  reaching  an  educational 
outcome  within  a  specific  context  or  what  steps  (in 
what  order)  are  most  indicated  to  generate  a  suitable 
instruction  design.  These  theoretical  attempts  are  pre¬ 
scriptive  in  nature  (Reigeluth,  1983b).  Instructional 
design  (ID)  is  often  the  label  used  to  refer  to  a  pre¬ 
scriptive  discipline  that  aims  at  optimizing  decision¬ 
making  for  instruction  (Elen,  1995).  The  label  instruc¬ 
tional  systems  design  (ISD)  refers  to  similar  efforts 
that  also  include  the  actual  development  of  instruction. 
How  exactly  ID  and  ISD  relate  to  each  other  is  an 
object  of  much  discussion. 

Although  descriptive  and  prescriptive  theories  can 
be  distinguished,  they  are  highly  interrelated.  Descrip¬ 
tive  theories  evolve  into  more  prescriptive  indications 
(Clark  and  Mayer,  2003),  and  prescriptive  theories 
generate  research  questions  (Merrill,  2006),  which  in 
turn  result  in  the  elaboration  or  refinement  of  prescrip¬ 
tive  theories. 

Descriptive  Theories 

Descriptive  theories  in  the  field  of  educational  tech¬ 
nology  aim  at  modeling  and  explaining  either  the 
actual  instructional  processes  and  their  learning  effects 
or  the  actual  way  in  which  instruction  is  developed. 
By  modeling  these  instructional  and  development  pro¬ 
cesses,  the  theoretical  statements  identify  which  vari¬ 
ables  are  relevant  and  how  they  relate  to  one  another. 
Such  (sets  of)  statements  can  have  substantial  predic¬ 
tive  and  explanatory  power.  They  may  help  us  to  better 
understand  what  is  happening  in  a  concrete  setting  and 
to  formulate  expectations  about  probable  effects  and 
potential  difficulties. 

In  modeling  instructional  processes,  researchers 
are  deeply  influenced  by  the  theoretical  perspectives 
they  adopt.  The  theoretical  perspective  greatly  affects 
what  is  regarded  to  be  important,  what  is  considered, 
and  what  is  discarded.  With  respect  to  instructional 
processes,  for  example,  a  clear  evolution  from  behav- 
iorist  (Skinner,  1968),  information-processing  (Ander¬ 
son,  1983;  Andre,  1979;  Kintsch  and  van  Dijk,  1978), 
constructivist  (von  Glasersfeld,  1987;  Wilson,  1996), 
and  socioconstructivist  (Lave  and  Wenger,  1991)  can 
be  observed.  Each  of  these  theoretical  perspectives 
offers  another  set  of  glasses  to  look  at  instructional 
processes  and  hence  results  in  the  identification  of 
different  sets  of  variables  and  different  interrelations 
between  those  variables. 

After  a  period  in  which  the  constructivist  nature 
of  human  learning  was  stressed,  it  seems  that  most 
recently  a  number  of  theoretical  insights  are  more 
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realistic  with  regard  to  instructional  design.  First  of 
all,  more  and  more  studies  strongly  stress  the  need  to 
systematically  consider  the  limits  of  human  cognition 
and  more  specifically  the  limited  self-regulation  skills 
in  instructional  settings  (Kirschner  et  al.,  2006;  Mayer, 
2004).  Furthermore,  warnings  have  been  uttered  that 
learners  may  be  less  compliant  than  expected  (Good¬ 
year,  2000),  partly  because  their  instructional  concep¬ 
tions  do  not  match  those  of  the  designers  (Elen  and 
Lowyck,  2000). 

With  respect  to  the  development  of  instruction,  it 
seems  that  the  systems  approach  and  activity  theory 
remain  important.  Continuously  the  models  are  being 
revised  to  ensure  that  all  important  design  aspects  are 
systematically  considered  (Dick  et  al.,  2001;  Tenny¬ 
son,  2000).  At  the  same  time,  cognitive  research  has 
clearly  shown  that  designers,  even  expert  designers, 
do  not  follow  a  linear  procedure.  The  development  is 
therefore  better  described  as  an  iterative  process. 
Affected  by  development  approaches  in  informatics, 
rapid  prototyping  (Dorsey  et  al.,  1996)  has  attracted 
attention  as  a  valuable  alternative  to  more  linear  pro¬ 
cedures.  Jonassen  and  Rohrer-Murphy  (1999)  have 
argued  and  illustrated  that  activity  theory  may  provide 
a  suitable  framework  when  the  design  of  constructivist 
learning  environments  is  the  goal. 

Prescriptive  Theories 

In  contrast  to  descriptive  theories,  prescriptive  theories 
discuss  how  things  should  be  rather  than  how  things 
are.  Building  on  such  prescriptive  theories,  various 
attempts  have  been  made  to  automate  instructional 
design  by  actually  embedding  the  prescriptions  in  sys¬ 
tems  that  support  the  automated  development  of 
instruction,  such  as  the  Advanced  Instructional  Design 
Advisor  (Muraida  and  Spector,  1992)  and  ID  Expert 
(Merrill,  1998);  for  a  more  in-depth  discussion,  see 
Spector  and  Ohrazda  (2004). 

The  prescriptive  orientation,  as  clearly  outlined  by 
Reigeluth  (1983b),  reveals  the  normative  perspective 
and  brings  the  goal  issue  immediately  to  the  front. 
Prescriptive  theories  basically  specify  how  instruction 
(in  general  or  more  specific)  should  look  to  enable 
specific  groups  of  learners  to  reach  an  instructional 
goal  or  how  the  developmental  process  should  look  to 
increase  the  probability  that  adequate  instruction  is 
developed.  Not  surprisingly,  then,  changes  in  what  are 
regarded  to  be  important  instructional  outcomes  affect 
the  nature  of  prescriptive  theoretical  statements.  Sim¬ 
ilarly,  new  target  populations  (distance  education  stu¬ 
dents)  and,  more  specifically,  different  views  on  what 
are  relevant  learner  characteristics  result  in  differences 
between  instructional  design  models.  As  an  example. 


the  interest  in  motivational  issues  is  growing  and  may 
result  in  changes  to  instructional  design  models  (Clark 
et  al.,  2006).  Finally,  the  diversity  of  instructional 
interventions  considered  as  serious  alternatives  also 
affects  the  models.  Goals,  target  populations,  and 
instructional  alternatives  are  heavily  influenced  by  the 
context  in  which  the  instructional  design  model  is  elab¬ 
orated;  for  example,  some  specific  multimedia-based 
instructional  design  models  have  appeared  (Lee  and 
Owens,  2000)  that  differ  from  more  general  models  as 
they  consider  only  e-learning  environments  as  the  con¬ 
text  in  which  the  instructional  interventions  can  be 
embedded. 

The  issue  of  goals  has  been  especially  influential 
in  the  last  several  decades  with  respect  to  the  elabora¬ 
tion  of  instructional  design  models.  Two  interrelated 
observations  can  be  made.  First,  the  scope  of  instruc¬ 
tional  outcomes  has  been  broadened  (Reigeluth,  1999). 
Far  more  than  in  the  past,  attention  is  being  paid  to 
instructional  goals  such  as  the  development  of  attitudes 
and  the  acquisition  of  complex  problem-solving  and 
learning  skills.  This  reorientation  in  instructional  goals 
has  resulted  in  the  emergence  of  more  integrated  and 
task-based  instructional  design  models,  whereas  older 
models  can  be  said  to  be  more  analytical  and  informa¬ 
tion-based.  Immediately  related  to  the  broadening  of 
instructional  outcomes,  some  authors  have  pointed  to 
the  difficulties  of  specifying  instructional  outcomes  in 
operational  terms.  Not  surprisingly,  the  models  that 
pertain  to  such  more  vague  instructional  outcomes  are 
also  more  general  and  less  specific.  Typical  examples 
are  the  cognitive  apprenticeship  model  (Collins  et  al., 
1989)  and  the  notion  of  powerful  learning  environ¬ 
ments  (De  Corte,  2003).  In  these  cases,  no  direct  pre¬ 
scriptions  are  formulated  but  general  criteria  for 
instruction  are  presented.  Interest  in  models  on  auto¬ 
mated  skills  is  growing  (Clark,  2006),  and  it  is  well 
recognized  that  solving  complex,  ill-structured,  or 
wicked  problems  requires  the  use  of  a  rich  set  of  highly 
automated  cognitive  skills. 


DEVELOPING  THEORIES 

As  already  pointed  out,  theoretical  statements— both 
descriptive  and  prescriptive  ones  — in  the  held  of  edu¬ 
cational  technology  differ  as  to  their  origin.  Some  are 
generated  through  reflections  on  practical  experiences, 
others  are  the  outcome  of  deliberate  deductions,  and  a 
last  group  results  from  interactions  between  theory- 
driven  reflections  and  attempts  to  solve  specific  design 
issues.  This  section  focuses  on  the  development  of 
instructional  design  models,  as  they  constitute  the  bulk 
of  more  systematic  sets  of  theoretical  statements  in  the 
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field.  This  also  implies  a  focus  on  primarily  prescrip¬ 
tive  theoretical  statements. 

Given  the  applied  nature  of  instructional  design, 
the  classical  approach  is  a  deductive  one.  An  instruc¬ 
tional  design  model  is  the  result  of  a  deductive 
approach  when  a  theory,  primarily  a  learning  theory, 
is  taken  as  the  starting  point  for  its  elaboration.  In 
developing  the  model,  the  instructional  designer  ana¬ 
lyzes  a  learning  theory  and  is  especially  interested  in 
what  the  theory  specifies  to  be  important  learner  vari¬ 
ables  (internal  conditions)  and  environmental  features 
(external  conditions).  The  elaboration  of  the  instruc¬ 
tional  design  model  implies  the  application  of  the 
learning  theory  to  a  specific  instructional  context.  The 
model  specifies  the  instructional  implications  of  the 
learning  theory.  A  deductive  approach  necessarily 
implies  that  a  transition  from  description  to  prescrip¬ 
tion  is  made.  How  the  learning  theory  describes  learn¬ 
ing  is  used  as  the  starting  point  to  prescribe  what  the 
instruction  should  look  like.  Such  a  transition  is  intrin¬ 
sically  problematic,  as  what  is  may  drastically  differ 
from  what  should  be.  The  consecutive  models  pro¬ 
posed  by  Gagne  (1985)  are  typical  examples  of  a 
deductive  approach.  The  consecutive  versions  of  the 
Gagne  model  also  clearly  show  that  models  are 
adapted  along  with  changes  in  the  underlying  learning 
theory.  To  assess  the  validity  of  these  deductively  con¬ 
structed  design  models  evaluation  studies  and  experi¬ 
ments  are  regarded  as  being  adequate  research 
approaches  (Tennyson  and  Cocchiarella,  1986). 

When  instructional  design  models  emerge  from  a 
specific  practical  context,  they  are  developed  in  line 
with  an  inductive  approach.  Within  a  specific  context 
and  often  for  a  specific  target  group  a  number  of 
instructional  interventions,  media,  or  approaches  are 
explored.  What  outcomes  can  be  reached  by  means  of 
these  instructional  interventions,  media,  or  approaches 
are  investigated  (for  such  an  approach  with  regard  to 
video  games,  see  Gee,  2003;  for  WebQuests,  see 
Dodge,  1995).  In  most  situations,  these  endeavors 
begin  with  high  expectations  for  the  potential  of  the 
instructional  interventions,  media,  or  approaches  at 
hand.  The  inductive  elaboration  of  an  instructional 
design  model  entails  an  abstraction.  The  model  builder 
aims  at  generalizing  the  specific  context  and  at  broad¬ 
ening  the  application  context  of  the  instructional  inter¬ 
ventions,  media,  or  approaches.  In  the  absence  of  crit¬ 
ical  tests  for  these  generalizations,  there  is  an  apparent 
danger  of  overgeneralization.  Case  studies  and  usabil¬ 
ity  and  feasibility  studies  may  help  to  test  inductively 
generated  theoretical  statements. 

Although  inductive  and  deductive  approaches  can 
logically  be  identified  as  clear  alternatives,  most 
instructional  design  models  do  result  from  a  combina¬ 


tion  of  inductive  and  deductive  approaches.  Elen 
(1995)  suggested  that  differences  between  the  theoret¬ 
ical  statements  in  instructional  design  models  can  be 
explained  by  considering  at  the  same  time  their  knowl¬ 
edge  base  and  their  referent  system.  Whereas  the 
knowledge  base  refers  to  the  theoretical  basis  of  the 
model,  the  reference  system  refers  to  the  application 
context  of  the  model.  Any  attempt  therefore  to  build  a 
model  by  only  considering  the  knowledge  base  or  the 
referent  system  is  doomed  to  fail.  It  is  not  surprising 
that  even  when  not  explicitly  mentioned  and  even  when 
a  general  applicability  is  claimed,  most  models  are 
clearly  linked  to  a  particular  theoretical  perspective  and 
are  rooted  in  a  particular  instructional  context  consist¬ 
ing  of  a  set  of  highly  valued  instructional  outcomes, 
potential  target  groups,  or  the  instructional  media  being 
considered.  As  an  example,  programmed  instruction 
(Skinner,  1968)  and  the  notion  of  criss-crossing  the 
landscape  (Spiro  et  al.,  1991)  can  only  be  well  under¬ 
stood  and  compared  when  the  specific  outcomes,  target 
groups,  and  instructional  media  are  considered. 

Assessing  the  validity  of  models  based  on  a  mixed 
approach  remains  a  concern.  Design  experiments 
deeply  rooted  in  both  theory  and  practice  and  oriented 
toward  both  theory  development  and  solving  real  prob¬ 
lems  are  considered  adequate  (Reeves,  2005). 

THEORETICAL  SOPHISTICATION 
AND  JUSTIFICATION 

Gordon  (1968)  formulated  a  number  of  criteria  for 
theories  of  instruction.  He  argued  that  statements  of  an 
instructional  theory  should  include  a  set  of  postulates 
and  a  definition  of  the  terms  involved  in  these  postu¬ 
lates;  each  statement  of  an  instructional  theory  or  sub- 
theory  should  make  explicit  the  boundaries  of  its  con¬ 
cern  and  the  limitations  under  which  it  is  proposed;  any 
theoretical  construction  must  have  internal  consistency; 
it  must  be  a  logical  set  of  interrelationships.  Further¬ 
more,  an  instructional  theory  should  be  congruent  with 
empirical  data,  be  capable  of  generating  hypotheses, 
contain  generalizations  that  go  beyond  the  data,  be 
verifiable,  and  be  stated  in  such  a  way  that  it  is  possible 
to  collect  data  to  disprove  it;  also,  it  not  only  must 
explain  past  events  but  must  also  be  capable  of  predict¬ 
ing  future  events  and  may  be  expected  to  represent 
qualitative  synthesis.  These  general  criteria  still  apply 
today.  A  quick  review  of  current  (sets  of)  theoretical 
statements  rapidly  reveals  that  only  a  very  limited  num¬ 
ber  of  such  (sets  of)  theoretical  statements  actually 
meet  these  criteria.  Instead  of  being  theory  based,  the 
field  is  model  based.  The  field  is  rather  characterized 
by  a  plethora  of  models,  which  might  be  regarded  to 
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be  the  precursors  of  robust  theories.  Some  of  the  most 
prominent  problems  are  terminological  confusion,  an 
absence  of  specifications  of  the  limits  of  the  models, 
and  a  lack  of  empirical  evidence  to  back  up  the  claims 
embedded  in  the  models.  Although  these  models 
replace  simple  collections  of  implications  for  instruc¬ 
tion  at  the  end  of  research  articles,  it  can  also  be 
observed  that  every  time  new  media  or  technologies 
are  introduced  the  field  is  tempted  to  reinvent  the 
wheel.  This  suggests  a  need  for  (1)  a  consolidation  of 
the  theoretical  approaches,  and  (2)  a  kind  of  classifica¬ 
tion  of  the  various  models  so  their  relationships  can  be 
more  clearly  exemplified  and  more  robust  theories 
elaborated.  Some  (such  as  Duchastel,  1998)  have 
argued  for  the  need  of  such  a  consolidation,  but  not 
everybody  is  convinced  of  the  relevance  and  the  pos¬ 
sibility  of  such  an  effort  (see  the  discussion  on  ITFO- 
RUM,  http://itechl  .coe.uga.edu/itforum/paper27/index 
27  .html) . 


CONCLUSION 

Educational  technology,  in  general,  and  instructional 
design,  in  particular,  are  characterized  by  a  diversity 
of  efforts  with  respect  to  theory  development.  Despite 
these  efforts,  it  is  difficult  to  identify  a  generally 
accepted  set  of  robust  theories  or  even  a  widely 
accepted  approach  to  develop  such  theories.  From  the 
perspective  of  theory  development,  the  held  is  diverse 
and  disparate.  This  diversity  reflects  both  the  complex¬ 
ity  of  the  held  in  terms  of  the  number  of  aspects  that 
have  to  be  dealt  with  and  the  diversity  of  theoretical 
perspectives  that  are  brought  to  the  held.  More  stability 
would  require  a  greater  consensus  regarding  these  the¬ 
oretical  perspectives  and  the  recognition  of  the  need 
for  a  greater  abstraction.  Such  an  abstraction  would, 
for  example,  require  that  new  technological  devices 
are  not  taken  as  the  starting  point  for  the  elaboration 
of  new  models  but  as  a  challenge  to  apply  what  is 
known  to  that  new  device.  Such  attempts  to  apply  the 
available  models  may  in  turn  represent  nice  opportu¬ 
nities  to  actually  validate  the  models  and  highlight 
their  generalizability.  A  similar  abstraction  is  indicated 
with  regard  to  the  formulation  of  instructional  goals 
and  target  groups.  Instructional  goals  and  target  groups 
must  be  described  at  a  sufficient  level  of  abstraction 
using  the  most  relevant  variables.  As  long  as  the  held 
cannot  make  distinctions  between  relevant  and  super¬ 
ficial  variables,  theory  development  in  the  held  of  edu¬ 
cational  technology  will  remain  a  worthwhile  but 
impossible  endeavor.  From  this  perspective,  it  could 
be  argued  that  theory  development  is  best  helped  by 
systematic  efforts  to  automate  instructional  design. 


Even  when  the  actual  product  of  such  an  endeavor  may 
never  be  used  in  actual  design  work,  the  need  for 
clarity  with  regard  to  the  variables  and  the  need  for 
mutual  consistency  in  attempts  to  automate  may  very 
well  help  to  generate  more  coherent  sets  of  theoretical 
statements  and  to  develop  an  actual  instructional 
design  theory. 
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ABSTRACT 

This  chapter  examines  the  types  of  designs  and  meth¬ 
odological  approaches  used  by  educational  technology 
and  instructional  design  researchers  to  investigate:  (1) 
instructional  strategies,  (2)  educational  technologies, 
(3)  instructional  design  models,  and  (4)  design  and 
development  of  instruction.  These  four  approaches  cor¬ 
respond  to  the  four  main  parts  of  the  Handbook.  In  the 
opening  section,  Ross  and  Morrison  discuss  how  exper¬ 
imental  methods  have  been  predominantly  used  in 
determining  the  effectiveness  of  instructional  strategies 
used  to  teach  specific  content,  affect  the  design  of  a 
course,  or  improve  message  design.  In  the  second  chap¬ 
ter  section,  Hannafin  and  Young  examine  how  research 
on  instructional  technologies  has  been  influenced  by  the 
manner  in  which  technology  has  been  conceptualized 
as  a  treatment  or  intervention.  When  it  is  regarded  as 
simply  a  medium  for  learning  (e.g.,  computer-assisted 
instruction),  usage  of  highly  controlled  experimental 
designs  is  most  common;  when  conceived  more  broadly 
as  a  technology-enhanced  learning  environment 
(TELE),  use  of  design-based  research  is  favored.  In  the 
third  section,  van  den  Akker  and  Kuiper  examine  design 
and  development  research  focusing  on  models  for 
instructional  design.  Specific  attention  is  directed  to 
analyzing  the  trends  that  are  most  visible  in  model 
development  and  validation  and  exploring  how  research 
on  models  can  be  redirected  to  foster  blending  of  theory 
and  practice.  In  the  final  section,  Richey  and  Klein  focus 
on  methods  and  issues  related  to  conducting  research 
on  the  design  and  development  of  instruction.  Studies 
in  this  area,  although  highly  diverse,  most  frequently 
rely  on  qualitative  techniques,  including  case  studies, 
interviews,  document  reviews,  and  observations. 

KEYWORDS 

Component  investigation:  The  identification  or  confir¬ 
mation  through  research  of  steps  (procedural  mod¬ 
els)  and  factors  (conceptual  models)  critical  to  the 
instructional  design  process. 


Design  and  development  research:  Research  that  seeks 
to  create  knowledge  grounded  in  data  systemati¬ 
cally  derived  from  practice  and  based  upon  a  sys¬ 
tematic  analysis  of  specific  cases. 

Experiment:  A  design  that  involves  the  comparison  of 
one  treatment  to  another,  using  two  or  more  differ¬ 
ent  groups. 

External  validity:  The  degree  to  which  the  results  of 
an  experiment  can  be  generalized  to  other  settings. 

In-program  learner  behaviors:  Measures  of  learner 
activities  and  outcomes  while  engaged  in  an 
instructional  task. 

Instructional  strategies:  The  prescribed  sequences 
and  methods  of  instruction  to  achieve  a  learning 
objective. 

Internal  validity:  The  degree  to  which  the  results  of  a 
study  can  be  attributed  to  the  treatment  rather  than 
extraneous  or  confounding  variables. 

Log  files:  A  sequence  of  behavioral  data  stored  in  a 
permanent  hie,  offering  time-  or  location-stamped 
records  of  human  choices  made  through  computer 
interfaces  with  learning  environments. 

Mixed-methods  research:  Studies  that  combine  quali¬ 
tative  and  quantitative  data  collection  methods. 

Model  research:  Research  that  addresses  the  validity 
or  effectiveness  of  an  existing  or  newly  constructed 
development  model,  process,  or  technique. 

Model  validation:  Experiments,  quasi-experiments, 
interviews,  and  expert  reviews  that  are  used  for 
verification  and  evaluation  of  a  particular  design 
and  development  technique. 

Open-ended  learning  environment:  Instructional 
projects  based,  to  varying  degrees,  on  construc¬ 
tivist  principles  in  which  learners  self-discover 
concepts  and  principles  while  engaged  in  authen¬ 
tic  learning  tasks. 

Qualitative  research:  Research  that  emphasizes  verbal 
descriptions  or  narrative  accounts  of  observed  or 
recorded  events. 

Quasi-experiment:  An  experimental  design  in  which 
participants  are  not  randomly  assigned  to  treat¬ 
ments. 
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Technology -enhanced  learning  environments:  Tech¬ 
nology-based  learning  and  instructional  systems 
through  which  students  acquire  skills  or  knowl¬ 
edge,  usually  with  the  help  of  teachers  or  facilita¬ 
tors,  learning  support  tools,  and  technological 
resources. 

Tool  research:  The  research-based  design  and  devel¬ 
opment  of  products  and  tools  to  enhance  learning. 

INTRODUCTION 

Steven  M.  Ross 

Research  in  educational  technology  and  communi¬ 
cations  has  had  a  long  history,  extending  well  over 
half  a  century.  Throughout  most  of  the  1900s,  influ¬ 
enced  most  strongly  by  the  paradigms  of  behavior¬ 
ism  and  the  physical  sciences,  researchers  relied  pre¬ 
dominately  on  experimental  methods  both  to  derive 
principles  and  laws  and  empirically  identify  more 
effective  ways  of  designing  and  delivering  instruc¬ 
tion.  As  the  field  matured  and  conceptualizations  of 
instructional  design  and  technology  have  broadened, 
a  need  has  arisen  for  diverse  research  methodologies 
that  focus  not  only  on  the  culminating  outcomes  of 
technology-based  interventions  but  also  on  the  learn¬ 
ing  and  developmental  processes  that  interventions 
engender  through  interactions  with  learning  con¬ 
texts,  individual  differences,  and  instructional  design 
approaches . 

Based  on  this  rationale,  this  chapter  examines  the 
types  of  designs  and  methodological  approaches 
used  by  contemporary  instructional  design  and  edu¬ 
cational  technology  researchers  to  investigate:  (1) 
instructional  strategies,  (2)  educational  technologies, 
(3)  instructional  design  models,  and  (4)  design  and 
development  of  instruction.  In  the  opening  chapter 
section,  Ross  and  Morrison  describe  how  experimen¬ 
tal  methods  have  been  predominantly  used  in  deter¬ 
mining  the  effectiveness  of  instructional  strategies 
used  to  teach  specific  content,  affect  the  design  of  a 
course,  or  improve  message  design.  They  further 
examine  the  tensions  cultivated  by  current  national 
trends  that  emphasize  strong  rigor  in  research  meth¬ 
ods  (and  therefore  favor  randomized  experiments) 
with  needs  by  educational  practitioners  and  research¬ 
ers  to  learn  (through  qualitative  and  quantitative 
methods)  how  interventions  work  in  more  complex 
field-based  applied  contexts. 

In  the  second  chapter  section,  Hannafin  and 
Young  examine  how  research  on  instructional  tech¬ 


nologies  has  been  influenced  by  the  manner  in  which 
technology  (referred  to  as  IT)  has  been  conceptual¬ 
ized  as  a  treatment  or  intervention.  When  IT  is 
regarded  as  simply  a  medium  for  learning  (e.g.,  com¬ 
puter-assisted  instruction),  use  of  highly  controlled 
experimental  designs  is  most  common,  but,  when  IT 
is  conceived  more  broadly  as  a  technology-enhanced 
learning  environment  (TELE),  use  of  design-based 
research  is  favored.  Aside  from  identifying  the  vari¬ 
able  or  cluster  of  variables  that  constitute  effective 
application  in  instruction,  a  key  issue  and  challenge 
for  researchers  is  identifying  and  operationally 
defining  meaningful  dependent  measures  to  deter¬ 
mine  technology’s  complex  potential  impacts  on 
learners . 

In  the  third  section,  van  den  Akker  and  Kuiper 
examine  design  and  development  research  focusing 
on  models  for  instructional  design.  Specific  attention 
is  directed  to  analyzing  the  trends  that  are  most 
visible  in  model  development  and  validation  and 
exploring  how  research  on  models  can  be  redirected 
to  foster  blending  of  theory  and  practice .  Given  the 
complexity  and  conceptual  nature  of  design  models, 
studies  in  this  area  use  highly  diverse  methodologies 
but  predominantly  draw  on  qualitative  designs  (case 
studies,  document,  analyses,  and  surveys)  in  inves¬ 
tigating  model  uses  and  on  experimental-type  meth¬ 
ods  in  conducting  model  validations . 

In  the  fourth  and  final  section,  Richey  and  Klein 
focus  on  methods  and  issues  related  to  conducting 
research  on  the  design  and  development  of  instruc¬ 
tion.  Such  research  can  involve  studying  the  design 
and  development  work  of  others  or  studying  one’s 
own  development  and  design  activities  as  they  occur. 
Accordingly,  although  diverse  research  methodolo¬ 
gies  have  been  employed  (including  experimental- 
type  approaches),  design  and  development  studies 
most  frequently  rely  on  qualitative  techniques  con¬ 
ducted  in  actual  work  settings,  including  case  stud¬ 
ies,  interviews,  document  reviews,  and  observations. 
Methodological  issues  for  increasing  validity,  such 
as  avoiding  researcher  bias  and  over-relying  on  recall 
data,  are  also  discussed. 

These  sections  support  the  theme  that  the 
research  methods  in  the  field  of  instructional  design 
and  technology  are  continually  evolving  to  investi¬ 
gate  more  fully  and  meaningfully  the  impacts  of 
contemporary  technology  applications  and  technol¬ 
ogy-supported  learning  environments.  Researchers, 
therefore,  can  benefit  from  being  knowledgeable 
about  how  and  where  to  apply  diverse  methodologies 
ranging  from  case  studies  and  document  reviews  to 
randomized  experiments. 
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RESEARCH  ON 

INSTRUCTIONAL  STRATEGIES 

Steven  M.  Ross  and  Gary  R.  Morrison 

In  this  chapter  section,  we  examine  research  methods 
used  to  investigate  instructional  strategies  in  educa¬ 
tional  technology.  As  will  be  discussed,  such  strategies 
entail  methods  for  improving  instructional  delivery, 
design,  and  ultimately  learning  performance.  Although 
the  full  gamut  of  research  methodologies,  ranging 
from  formative  evaluation  and  design-based  research 
(see  Richey  and  Klein  in  this  chapter;  Wang  and 
Hannafin,  2005)  to  qualitative  and  case  studies  to  true 
(randomized)  experiments,  has  been  used  to  investi¬ 
gate  strategy  effects,  the  predominant  approach  in  the 
research  literature  has  been  experimental  designs  to 
increase  objectivity  and  rigor.  In  the  process,  a  frequent 
and  often  natural  trade-off  has  occurred  between  inter¬ 
nal  validity— the  ability  to  infer  causation  from  strat¬ 
egy  effects  — and  external  validity— the  ability  to  gen¬ 
eralize  results  to  real-world  educational  settings.  We 
will  revisit  these  issues  in  later  sections. 

That  research  on  instructional  strategies  in  educa¬ 
tional  technology  has  relied  so  extensively  on  experi¬ 
mental  methods  should  not  come  as  a  surprise.  Exper¬ 
imental  designs  have  had  a  long  tradition  in 
psychology  and  education.  When  psychology  emerged 
as  an  infant  science  during  the  1900s,  it  modeled  its 
research  methods  on  the  established  paradigms  of  the 
physical  sciences,  which  for  centuries  relied  on  exper¬ 
imentation  to  derive  principals  and  laws.  Subsequent 
reliance  on  experimental  approaches  was  strengthened 
by  behavioral  approaches  to  psychology  and  education 
that  predominated  during  the  first  half  of  the  20th 
century.  Thus,  usage  of  experimentation  in  educational 
technology  over  the  past  60  years  has  been  influenced 
by  developments  in  theory  and  research  practices 
within  its  parent  disciplines. 

In  the  following  pages,  we  examine  practices, 
issues,  and  trends  related  to  the  application  of  experi¬ 
mental  research  methods  in  educational  technology, 
with  particular  focus  on  instructional  strategies 
research.  Respecting  the  scope  and  desired  length  of 
this  chapter  section,  our  examination  of  experimental 
research  issues  and  methodology  necessarily  is  brief. 
Interested  readers  can  find  a  more  detailed  discussion 
in  Ross  et  al.  (2005)  and  in  any  mainstream  educational 
research  textbook  (such  as  Creswell,  2002).  In  subse¬ 
quent  sections,  we  examine  through  explanation  and 
concrete  examples  research  on  instructional  strategies 
usage  in  education,  including  micro- strategy,  macro¬ 
strategy,  and  message  design  research  methods.  Next, 


we  present  an  analysis  of  the  prevalence  over  time  of 
experimental  studies  in  published  research.  Factors 
affecting  the  rigor  of  experiments  on  instructional 
strategies,  such  as  internal  and  external  validity  con¬ 
cerns,  are  examined  in  a  final  major  section,  followed 
by  a  summary  of  key  ideas  emerging  from  the  preced¬ 
ing  topics. 

Research  on  Instructional  Strategies 
Usage  in  Educational  Technology 

The  Rise,  Fall,  and  Likely  Resurgence  of 
Experimentation  on  Instructional  Strategies 

Experiments  comprise  the  highest  proportion  of  stud¬ 
ies  on  instructional  strategies  published  in  the  research 
section  of  Educational  Technology  Research  and 
Development  (ETR&D)\  however,  a  noticeable  trend 
over  the  past  two  decades  both  in  ETR&D  and  in 
leading  educational  psychology  journals  (Hsieh  et  al., 
2005;  Levin,  2004)  has  been  a  steady  decline  in  the 
quantity  of  experimental  studies  performed  on  educa¬ 
tional  interventions.  The  rise,  however,  occurred  first. 

What  gave  experimental  designs  initial  impetus  for 
researching  instructional  strategies?  Two  decades  ago, 
Hannafin  (1986)  posited  four  reasons.  One  is  the  obvi¬ 
ous  influence  of  the  field’s  behavioral  roots.  A  second 
reason  is  that  experimentation  has  been  traditionally 
viewed  as  the  definition  of  acceptable  research  in  the 
field.  Positivist  views  have  reinforced  beliefs  about  the 
importance  of  scientific  rigor,  control,  statistical  veri¬ 
fication,  and  hypothesis  testing  and  their  being  the 
correct  approaches  to  research  in  the  field.  Qualitative 
researchers  have  challenged  this  way  of  thinking,  but 
until  recently  acceptance  of  alternative  paradigms  has 
been  reluctant  and  of  minimal  consequence  (Guba, 
1981;  Savenye  and  Robinson,  2004).  Third,  promotion 
and  tenure  criteria  at  colleges  and  universities  have 
been  strongly  biased  toward  experimental  studies. 
Fourth,  the  research  journals  have  published  propor¬ 
tionately  more  experimental  studies  than  alternative 
types.  As  a  result,  a  self-perpetuating  situation  is  cre¬ 
ated,  in  which  increased  exposure  to  experimental 
studies  increases  the  likelihood  that  beginning 
researchers  will  also  favor  the  experimental  method  in 
their  research. 

To  borrow  from  music  legend.  Bob  Dylan  (with 
apologies  to  our  past  English  teachers),  “the  times, 
they  have  been  a  changing”  and  appear  to  be  changing 
still.  In  the  recent  Hsieh  et  al.  (2005)  study,  articles 
published  in  four  empirical  educational  psychology 
journals  in  1983  and  from  1995  to  2004  were  reviewed. 
Findings  indicated  that  the  percentage  of  randomized 
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intervention  studies  published  in  these  journals 
declined  from  47%  in  1983  to  only  34%  in  1995  and 
26%  in  2004  (Hsieh  et  al.,  2005,  p.  524).  To  what 
factors  can  this  trend  be  attributed?  Ironically,  one 
potential  cause  might  be  the  very  promotion  of  such 
studies  as  the  gold  standard  of  scientifically  credible 
research.  Confronted  with  more  explicit  and  complex 
standards  for  rigor  (Shavelson  et  al.,  2002),  combined 
with  the  intrinsic  challenges  of  randomizing  interven¬ 
tions  in  actual  classrooms,  researchers  might  view  the 
randomized  experiment  as  exceeding  their  available 
resources  and  time.  A  second  factor,  as  noted  above, 
might  be  the  increased  interest  in  and  acceptance  of 
qualitative  studies  over  the  years. 

The  tide  seems  to  be  shifting  once  again,  however, 
but  too  late  for  the  new  wave  of  randomized  and  highly 
rigorous  quasi-experimental  studies  to  have  been 
detectable  in  the  reviews  by  Hsieh  et  al.  in  2005  or 
Levin  in  2004.  The  impetus  is  the  strong  advocacy  by 
the  No  Child  Left  Behind  (2001)  legislation  for  using 
rigorous  scientific  research  to  determine  what  works 
in  education.  Guidance  for  judging  the  quality  of 
research  has  recently  been  proposed  by  national  orga¬ 
nizations,  such  as  the  National  Research  Council 
(NRC)  (Eisenhart  and  Towne,  2003;  Feuer  et  al.,  2002; 
Shavelson  et  al.,  2002)  and  the  What  Works  Clearing¬ 
house  (http://w-w-c.org/standards.html).  In  the  hierar¬ 
chy  of  rigorous  studies,  randomized  experiments  hold 
the  highest  position,  followed  by  rigorous  regression 
discontinuity  designs  and  rigorous  quasi-experiments. 
Of  note,  the  2005  national  recompetition  for  Regional 
Educational  Laboratories  (RELs)  by  the  Institute  for 
Educational  Sciences  was  shifted  from  the  tradition¬ 
ally  emphasized  capacity  to  use  research  evidence  to 
provide  technical  assistance  to  practitioners  to  gener¬ 
ating  scientifically  based  research  evidence  on  impor¬ 
tant  educational  interventions  using  randomized  field 
trials.  Today,  in  2007,  many  of  the  federal  grant  com¬ 
petitions  specifically  encourage  or  require  randomized 
or  rigorous  quasi-experimental  designs  for  the  evalu¬ 
ation  component  of  the  project.  We  therefore  predict 
increased  publication  of  such  studies  in  the  research 
journals  within  the  next  few  years. 

Types  of  Experimental  Designs 

The  purpose  of  an  experiment  is  to  determine  the 
degree  to  which  outcomes  of  interest  are  caused  by  the 
treatment  (intervention)  as  opposed  to  extraneous  fac¬ 
tors.  The  true  or  randomized  experiment  generally  is 
considered  best  equipped  to  provide  such  evidence  due 
to  its  control  over  possible  sampling  bias.  An  advan¬ 
tage  over  less  rigorous  designs  is  the  elimination, 
through  random  assignment,  of  any  systematic  error 


that  might  be  associated  with  using  intact  groups;  thus, 
differences  in  learning  outcomes  should  be  solely 
attributable  to  treatment  effects. 

Oftentimes  in  educational  studies,  it  is  neither 
practical  nor  feasible  to  randomly  assign  participants 
to  treatments.  Such  constraints  are  likely  to  occur  in 
school-based  research,  due  to  principals  and  teachers 
understandably  being  resistant  to  changing  students’ 
class  assignments  or  subject  themselves  to  having  to 
use  different  programs  based  on  luck.  In  such 
instances,  rigorous  quasi-experimental  (matched  treat¬ 
ment-control  group)  designs  are  typically  far  more 
practical  to  implement  across  broad  and  diverse  appli¬ 
cation  sites  (Borman  et  al.,  2003).  A  common  appli¬ 
cation  in  educational  technology  would  be  to  expose 
to  similar  classes  or  groups  of  students  alternative 
instructional  strategies  and  compare  them  on  desig¬ 
nated  dependent  measures  (e.g.,  learning,  attitude, 
classroom  behavior)  during  the  year. 

An  important  component  of  the  quasi-experimen¬ 
tal  study  is  the  use  of  pretesting  or  analysis  of  prior 
achievement  to  establish  group  equivalence.  In  the  true 
experiment,  randomization  makes  it  improbable  that 
one  group  will  be  significantly  superior  in  ability  to 
another,  but  systematic  bias  can  easily  be  introduced 
in  the  quasi-experiment;  for  example,  although  the 
first-  and  third-period  algebra  classes  may  have  the 
same  teacher  and  identical  lessons,  it  may  be  the  case 
that  honors  English  is  offered  third  period  only,  thus 
restricting  those  honors  students  to  taking  first-period 
algebra. 

To  determine  the  effectiveness  of  instructional 
strategies,  educational  technology  researchers  have 
relied  extensively  on  randomized  and  rigorous  quasi¬ 
experiments  to  establish  high  levels  of  internal  valid¬ 
ity— the,  ability  to  isolate  treatment  effects  from  extra¬ 
neous  variables.  Because  instructional  strategies 
research  represents  our  focus,  we  next  establish  a 
clearer  definition  of  “instructional  strategies”  and  then 
examine  exemplary  studies  used  to  test  theories  and 
hypotheses  about  their  effectiveness  for  learning. 

Instructional  Strategies  and  Heuristics 

Instructional  strategies  can  be  defined  as  the  prescribed 
“sequences  and  methods  of  instruction  to  achieve  an 
objective”  (Morrison  et  al.,  2007,  p.  150).  The  partic¬ 
ular  prescriptions  are  not  only  determined  by  the  con¬ 
tent  and  performance  specified  in  the  learning  objec¬ 
tives,  but  also  by  underlying  learning  theory;  for 
example,  behavioral  learning  theory  promotes  strate¬ 
gies  that  use  reinforcement  and  active  responding 
(practice)  to  build  strong  connections  between  stimuli 
and  responses.  Cognitive  theory  shifts  the  emphasis  to 
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fostering  meaningful  learning  by  associating  new 
material  with  the  learner’s  prior  knowledge. 

Given  these  factors  — type  of  content,  desired  per¬ 
formance,  theoretical  paradigm— coupled  with  con¬ 
straints  imposed  by  learning  contexts  (e.g.,  student 
characteristics,  resources,  allotted  time),  decisions 
involving  which  instructional  strategies  to  choose  are 
usually  quite  complex.  As  described  by  Morrison  and 
colleagues  (2007), however,  practical  and  scientifically 
grounded  heuristics  can  be  derived  to  guide  the  iden¬ 
tification  of  strategies  for  teaching  certain  types  of 
content,  such  as  facts,  concepts,  principles  and  rules, 
procedures,  interpersonal  skills,  and  attitudes. 
Research-supported  heuristics  for  teaching  facts,  for 
example,  prescribe  such  strategies  as  rehearsal  prac¬ 
tice,  mnemonics,  and  usage  of  concrete  representa¬ 
tions. 

An  application  of  such  heuristics  in  a  more 
restricted  domain  is  illustrated  in  a  recent  paper  by 
Morrison  and  Anglin  (2005).  In  reviewing  a  collection 
of  experimental  studies  on  cognitive  load  theory,  pub¬ 
lished  in  a  special  issue  of  ETR&D ,  the  authors  created 
heuristics  for  e-learning  based  on  the  research  findings. 
To  illustrate,  their  initial  two  examples  are: 

•  For  teaching  learners  with  limited  technology 
skills,  a  strategy  requiring  initial  learning  of 
the  relevant  technology  skills,  then  the  par¬ 
ticular  content  area  concepts,  will  enhance 
learning  (Morrison  and  Anglin,  2005,  p.  95; 
based  on  Clarke  et  al.,  2005). 

•  Exploratory  practice  results  in  greater 
involvement  than  do  worked  examples  for 
experienced  students  (Morrison  and  Anglin, 
2005,  p.  96;  based  on  Paas  et  al.,  2005). 

The  above  examples  illustrate  how  the  instructional 
design  process  is  informed  directly  from  educational 
technology  research  on  instructional  strategies.  Spe¬ 
cifically,  research  findings  are  interpreted  into  heuris¬ 
tics  that  guide  strategy  identification,  development, 
and  usage  for  certain  types  of  educational  contexts  and 
learning  objectives.  Central  to  this  chapter  section  is 
the  historical  and  current  reliance  on  experimental 
research  to  provide  the  scientific  evidence  on  the 
effects  of  such  strategies. 

Research  and  Instructional  Methods 

In  this  section,  we  first  describe  research  methods  used 
to  investigate  micro-instructional  strategies  that  focus 
on  teaching  specific  content  information  such  as  facts 
and  concepts.  Second,  we  examine  methods  used  to 
investigate  macro-instructional  strategies  such  as 


cooperative  learning,  problem-based  learning,  and 
feedback  that  affect  the  design  of  a  course.  Third,  we 
discuss  research  methodologies  used  to  investigate 
message  design  strategies. 

Micro-Instructional  Strategy  Research  Methods 

Research  that  focuses  on  component  (or  micro )  strat¬ 
egies  to  help  students  learn  specific  content  such  as 
facts,  concepts,  and  principles  is  a  rich  area  of  explo¬ 
ration  and  experimentation  for  researchers  and  a  crit¬ 
ical  knowledge  base  for  the  development  of  heuristics 
by  practitioners.  The  most  common  research  approach 
is  a  true  experiment  that  contrasts  two  or  more  strate¬ 
gies.  The  following  examples  are  representative  of  the 
types  of  studies  used  to  investigate  micro-instructional 
strategies.  The  first  two  studies  focus  on  fact  learning 
and  the  third  focuses  on  concept  learning. 

Example  1 .  Rummel,  Levin,  and  Woodward  (2003) 

Rummel  et  al.  (2003)  compared  usage  of  a  visual  mne¬ 
monic  to  free  study  of  a  list  of  theorists’  names  and 
accomplishments.  Participants  were  randomly 
assigned  to  either  a  visual  mnemonic  or  a  free  study 
treatment.  The  visual  mnemonic  showed  an  image  of 
one  or  more  individuals  with  objects  to  provide  the 
mnemonic  device  to  aid  recall.  Dependent  variables 
included  an  immediate  and  delayed  post-test  and  a 
measure  of  attitude  toward  the  effectiveness  of  the 
method.  The  research  design  resulted  in  a  2  (study 
strategy)  x  2  (subtopic  order)  analysis  of  variance 
(ANOVA).  Results  showed  that  participants  in  the 
mnemonic  group  remembered  more  of  the  targeted 
information  (i.e.,  information  in  the  mnemonic)  than 
the  free  study  group.  For  designers,  this  suggests  that 
visual  mnemonics  are  beneficial  in  helping  learners 
recall  important  information. 

Example  2.  Woloshyn,  Paivio,  and  Pressley  (1994) 

Woloshyn  et  al.  (1994)  presented  participants  with  32 
factual  statements,  16  of  which  were  consistent  and  16 
of  which  were  inconsistent  with  their  beliefs.  The  con¬ 
trol  group  was  told  to  read  the  facts  for  understanding, 
whereas  the  elaboration  (experimental)  group  was  told 
to  explain  why  the  fact  was  true.  The  facts  were  written 
in  three  formats:  traditional  (“All  fire  trucks  are  not 
red.”),  refutational  (“Some  people  think  all  fire  trucks 
are  red;  they  are  not.”) ,  and  inverted  refutational,  where 
the  inaccurate  belief  was  presented  second  (“Not  all 
fire  trucks  are  red,  although  some  people  think  they 
are.”).  Participants  completed  an  immediate  and  14- 
day  delayed  test.  The  data  were  analyzed  via  a  2  (strat¬ 
egy)  x  2  (fact)  x  3  (sentence  format)  ANOVA.  Findings 
indicated  that  the  use  of  elaboration  enhanced  the 
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recall  of  factual  information  regardless  of  consistency 
with  prior  beliefs;  thus,  an  alternative  strategy  to 
rehearsal  of  factual  information  is  the  use  of  an  elab¬ 
oration  strategy. 

Example  3.  Park  and  Tennyson  (1986) 

This  experiment  was  designed  to  investigate  the  effec¬ 
tiveness  of  different  strategies  for  learning  concepts 
(Park  and  Tennyson,  1986).  The  four  treatments 
included  two  variations  in  presentation  form  (adaptive 
or  fixed  selection)  and  two  sequences  (generalization 
or  discrimination  rule).  The  result  was  a  2  x  2  factorial 
design  using  a  multiple  analysis  of  variance 
(MANOVA),  followed  by  separate  univariate  tests 
(ANOVAs).  This  design  allowed  the  researchers  to 
identify  significant  interactions  between  the  presenta¬ 
tion  format  and  sequence  that  provided  useful  heuris¬ 
tics  for  designers  based  on  the  intended  outcomes;  that 
is,  the  design  indicated  one  approach  for  developing 
conceptual  knowledge  and  another  for  developing  pro¬ 
cedural  knowledge. 

Macro-Instructional  Strategy  Research  Methods 

Macro-strategies  have  a  wider  focus  and  affect  a 
broader  aspect  of  a  course  than  do  micro  strategies.  As 
illustrations  of  macro  strategies,  we  will  describe 
below  experimental  research  methodologies  used  to 
investigate  broader  strategies  domains  such  as  person¬ 
alization,  feedback,  pedagogical  agents,  problem- 
based  learning,  cooperative  learning,  and  distance  edu¬ 
cation. 

Example  1 .  Kit  and  Sullivan  (2000) 

Ku  and  Sullivan  (2000)  described  a  study  in  which 
they  used  personal  information  about  each  participant 
to  personalize  word  problems  (Anand  and  Ross,  1987). 
Participants  were  grouped  by  ability  (high  or  low)  and 
assigned  to  either  a  personalized  or  nonpersonalized 
treatment.  Data  included  a  nonpersonalized  pretest,  a 
post-test  of  personalized  and  nonpersonalized  items, 
and  an  attitude  survey.  Two  different  analyses  were 
performed.  The  first  was  a  2  (treatment)  x  2  (ability) 
x  2  (test  occasion:  pretest  vs.  post-test)  ANOVA.  This 
analysis  used  treatment  and  ability  as  between-subject 
variables  and  test  occasion  as  a  within-subjects  vari¬ 
able.  The  second  analysis  was  a  2  (treatment)  x  2 
(ability)  x  2  (problem  type:  personalized  vs.  nonper¬ 
sonalized)  ANOVA  used  to  compare  post-test  perfor¬ 
mance  on  the  two  problem  types.  Similar  to  the  Park 
and  Tennyson  (1986)  study  described  above,  this 
design  allowed  the  investigators  to  identify  significant 
interactions  as  well  as  significant  main  effects.  The 
study  did  not  find  an  advantage  for  personalized  prob¬ 


lems  but  did  find  that  personalized  problems  were  pre¬ 
ferred  by  the  participants.  While  these  results  are 
inconsistent  with  other  studies  on  personalized  strate¬ 
gies,  the  results  support  the  idea  that  personalization 
may  enhance  motivation  when  solving  math  story 
problems. 

Example  2.  Kim,  Baylor,  and  Group  (2006) 

Another  example  of  a  macro  strategy  is  the  use  of 
pedagogical  agents  (Kim  et  al.,  2006).  Participants 
classified  as  low  or  high  competency  were  assigned  to 
a  proactive  (initiated  by  agent)  or  responsive  (initiated 
by  participant)  interaction,  resulting  in  four  treatments. 
Data  collected  included  participants’  self-efficiency 
rating,  attitude  toward  the  agent,  and  an  achievement 
test  that  assessed  recall  and  application.  The  resulting 
design  was  a  2  (competency)  x  2  (interaction) 
between-subjects  factorial  design  using  multiple  anal¬ 
ysis  of  covariance  (MANCOVA).  The  findings  suggest 
that  designers  create  highly  competent  pedagogical 
agents  as  learning  companions  (PALs)  when  the 
objectives  focus  on  knowledge  and  skill  acquisition; 
however,  if  the  learner  has  low  self-efficacy,  he  or  she 
might  be  threatened  by  this  type  of  expert  PAL.  The 
designer,  therefore,  might  consider  providing  a  less 
competent,  more  peer-like  PAL. 

Example  3.  Clariana  and  Lee  (2001 ) 

Feedback  in  instruction  is  an  example  of  a  well- 
researched  macro  strategy.  Clariana  and  Lee  (2001) 
examined  the  comparative  effectiveness  of  five  differ¬ 
ent  feedback  treatments.  Four  included  multiple- 
choice  questions  paired  with  (1)  a  single  try,  (2)  mul¬ 
tiple  try  and  feedback,  (3)  a  single  try  and  overt  typing 
of  correct  response,  and  (4)  multiple  try  with  overt 
typing  of  the  response.  The  fifth  treatment  was  con- 
structed-response  questions  with  feedback.  Data  col¬ 
lected  included  immediate  post-test,  time-on-task  data 
and  learner  responses  to  two  interview  questions.  A  5 
(treatment)  x  7  (post-test  question  level)  post-test-only 
design  with  repeated  measures,  using  a  mixed 
ANOVA,  was  employed.  A  second  5  (treatment)  x  2 
(time  in  lesson  and  time  in  test)  mixed  ANOVA  was 
also  used.  The  study  found  that  feedback  following  an 
incorrect  response  plus  entering  the  correct  answer  was 
the  most  effective  and  efficient  strategy  for  recogni¬ 
tion-type  learning. 

Example  4.  Cavalier,  Klein,  and  Cavalier  (1995) 

Research  on  classroom-oriented  strategies  such  as 
cooperative  learning  and  problem-based  learning  have 
commonly  employed  true  experimental  and  quasi- 
experimental  designs.  The  study  by  Cavalier  et  al. 
(1995)  of  cooperative  learning  in  a  technical  training 
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course  used  a  quasi-experimental  design.  Intact  groups 
were  assigned  to  either  a  cooperative  learning  treat¬ 
ment  or  control  (conventional  instruction)  treatment, 
both  of  which  were  taught  by  the  same  instructor. 
Assessments  included  a  post-test,  attitude  survey,  and 
group  interaction  behaviors.  ANOVA  was  used  to  com¬ 
pare  the  post-test  scores  of  the  two  groups,  and 
MANOVA  was  used  to  analyze  group  differences 
between  social  and  cognitive  interaction  behaviors. 
Findings  showed  that  practice  in  a  cooperative  group 
environment  produced  superior  achievement  as  well 
as  higher  levels  of  social  and  cognitive  interactions. 

Message  Design  Research  Methods 

Much  of  the  message  design  research  has  focused  on 
perception  issues  such  as  pictures,  colors,  and  legibil¬ 
ity  (Fleming  and  Levie,  1978).  More  recently,  the 
work  on  cognitive  load  theory  also  has  been  classified 
within  the  message  design  category  (Anglin  and  Mor¬ 
rison,  2001). 

Example  1 .  Lamberski  and  Dwyer  (1983) 

Lamberski  and  Dwyer  (1983)  investigated  the  effect  of 
color  vs.  black-and-white  illustrations  on  learning.  Par¬ 
ticipants  read  a  21 -page  instructional  unit  on  the  human 
heart  that  included  either  color  or  black-and-white  illus¬ 
trations  on  each  page.  An  80-item  criterion  test  consist¬ 
ing  of  terminology,  ability  to  draw  a  heart,  identification 
of  parts,  and  comprehension  of  heart  functions  was 
administered  immediately  after  completing  the  instruc¬ 
tion  and  6  weeks  later.  One  version  of  the  multiple- 
choice  items  had  the  distracters  printed  in  the  same 
color  as  they  appeared  in  the  drawings.  Participants 
were  randomly  assigned  to  one  of  four  treatment 
groups:  (1)  color-coded  instruction,  color-coded  test; 
(2)  color-coded  instruction,  black-and-white  test;  (3) 
black-and-white  instruction,  color-coded  test;  and  (4) 
black-and-white  instruction,  black-and-white  test.  The 
data  were  analyzed  using  ANOVA  with  repeated  mea¬ 
sures.  The  results  showed  that  the  color  instruction  was 
superior  to  black-and-white  instruction  for  this  task,  but 
the  color-coded  tests  were  of  no  benefit. 

Example  2.  Chandler  and  Sweller  (1992) 

A  more  recent  study  focused  on  the  split-attention 
effect  identified  by  Sweller  and  his  colleagues.  In  a 
traditional  textbook,  readers  must  split  their  attention 
between  the  narrative  explanation  in  the  text  and  a 
table  or  graphic.  In  the  current  study  (Chandler  and 
Sweller,  1992),  a  revised  version  of  the  text  that  inte¬ 
grated  the  explanatory  text  with  the  graphic  informa¬ 
tion  was  created.  A  t-test  was  used  to  assess  the  dif¬ 
ferences  between  the  split-attention  and  integrated 


groups’  scores  on  each  item  of  the  post-test.  The 
authors  concluded  from  the  findings  that  an  integrated 
design  used  to  reduce  split  attention  resulted  in  both 
improved  performance  and  more  efficient  learning. 

The  above  discussion  has  illustrated  the  types  of 
research  methods  used  to  investigate  the  effectiveness 
of  micro,  macro,  and  message  design  strategies.  All 
examples  were  of  true  or  rigorous  quasi-experiments, 
thus  reflecting  the  strong  historical  preference  in  the 
educational  technology  field  for  investigating  instruc¬ 
tional  strategies  via  experimentation.  But,  have  these 
preferences  of  educational  technology  researchers 
diminished  over  the  years?  In  the  following  section, 
we  examine  the  trends  through  an  analysis  of  pub¬ 
lished  articles  in  a  leading  educational  technology 
research  journal,  ETR&D. 

The  Prevalence  of  Experimentation  in 
Intervention  Research  in  Educational 
Technology:  A  Trend  Analysis  of  ETR&D 

As  described  in  an  earlier  section,  Hsieh  et  al.  (2005) 
recently  asked  if  educational  intervention  research  is 
on  the  decline.  They  defined  intervention  research  as 
strategies  for  affecting  cognitive,  affective,  and  behav¬ 
ioral  outcomes.  The  analysis  of  four  educational  psy¬ 
chology  journals  revealed  a  drop  in  the  percentage  of 
intervention  studies  published.  In  1983,  55%  of  the 
articles  were  intervention  studies, but  in  1995  and  2004 
the  percentages  fell  to  47%  and  35%,  respectively.  For 
this  chapter  section,  we  conducted  a  similar  analysis 
on  the  Research  section  of  ETR&D  for  the  same  years. 

Method 

Included  in  the  analyses  were  articles  published  in 
1983*  in  the  Educational  Communications  and  Tech¬ 
nology  Journal  (ECTJ:  the  predecessor  of  ETR&D )  and 
from  the  Research  section  of  ETR&D  (1994-2005). 
Excluded  from  the  analyses  were  studies,  whether 
experiments  or  other  designs ,  included  as  part  of  special 
issues  from  the  Development  section  of  the  journal. 
The  included  articles  were  classified  as  “true  experi¬ 
ments”  or  “other,”  with  the  latter  category  further  clas¬ 
sified  as  survey,  case  study,  qualitative,  theoretical,  lit¬ 
erature  review,  or  other.  For  the  true  experimental 
studies  only,  participants  were  classified  as  K-12  stu¬ 
dents,  college  students,  or  adults  (i.e.,  not  enrolled  in  a 
college-level  course).  The  type  of  assessments 
(achievement/performance  and  attitude)  were  noted  if 
there  were  multiple  forms  of  achievement;  for  example, 
a  post-test  that  assessed  both  recall  and  application  of 


*  1983  is  used  as  a  benchmark  as  in  the  study  by  Hsieh  et  al.  (2005). 
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Figure  54.1  Percentage  of  intervention  studies  published  in  ERT&D. 
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Figure  54.2  Percentage  of  true  experiments  published  in  ERT&D. 


the  content  was  considered  a  multiple  assessment.  Time 
of  testing  was  classified  as  immediate,  delayed,  or 
immediate  and  delayed.  The  length  of  the  intervention 
(e.g.,  treatment)  was  classified  as  less  than  3  hours,  1 
day  or  less,  1  week  or  less,  and  more  than  1  week.  Last, 
the  technology  of  interest  in  the  study  was  classified  as 
computer,  print,  television,  audio,  or  other. 

Results  and  Implications 

Consistent  with  Hsieh  et  al.  (2005),  the  data  show  (see 
Figure  54.1)  a  declining  trend  for  intervention  studies 
during  the  1995-2004  period.  In  1983,  75%  of  the 
articles  published  were  intervention  studies.  For  the 
1995-2004  time  period,  however,  intervention  studies 
accounted  for  only  44%  of  the  published  papers.  Fig¬ 
ure  54.2  shows  the  number  of  intervention  studies  that 
used  a  true  experimental  (e.g.,  random  assignment  of 
participants)  design.  Following  the  pattern  for  inter¬ 
vention  studies  in  general,  there  is  a  decreasing  trend 
in  the  usage  of  true  experiments  in  the  published 
ETR&D  research. 

An  analysis  of  the  non-intervention  studies  (see 
Figure  54.3)  provides  some  insights  into  the  type  of 
articles  published  by  the  Research  section.  In  1983, 


non-intervention  studies  accounted  for  only  25%  of 
published  articles.  The  non-intervention  studies  were 
equally  distributed  between  theoretical  and  literature 
reviews  and  qualitative  studies.  Theoretical  papers  are 
the  most  frequent  type  of  non-intervention  study  pub¬ 
lished,  and  qualitative  studies  account  for  the  second 
most  frequent  type  in  the  1994-2004  time  frame.  Our 
findings  regarding  the  drop  in  the  prevalence  of  inter¬ 
vention  studies  in  educational  technology  thus  mirror 
those  of  Hsieh  and  colleagues  (2005)  for  educational 
psychology  in  general. 

K-12  students  were  used  as  participants  in  49%  of 
the  studies  in  the  1995-2004  articles,  followed  by  col¬ 
lege  students  (43%).  The  majority  (64%)  of  the  inter¬ 
ventions  lasted  3  hours  or  less;  in  contrast,  20%  lasted 
more  than  a  week.  These  intervention  times  may  be 
the  result  of  the  number  of  studies  conducted  in  a  K-12 
classroom.  This  bimodal  distribution  suggests  that  the 
studies  are  either  a  one-time  investigation  of  a  micro- 
instructional  strategy  or  a  longer  examination  of  a 
macro  strategy  such  as  cooperative  learning.  Given  the 
large  number  of  instructional  designers  employed  in 
business,  military,  government,  and  healthcare  set¬ 
tings,  these  data  suggest  an  opportunity  for  collabora¬ 
tive  partnerships  between  practitioners  and  researchers 


724 


Research  Designs 


1983  1995-2004 


§a  Qualitative 

□  Theoretical/Reviews 

□  Other 


Figure  54.3  Other  studies  published  in  ETR&D. 
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Figure  54.4  Types  of  outcome  data. 


to  conduct  studies  in  workplace  settings  using  realistic 
materials  with  participants  who  have  a  need  and  pos¬ 
sible  motivation  for  learning  the  content.  Such  studies 
might  provide  the  field  with  an  additional  perspective 
and  data  on  micro-  and  macro- instructional  strategies. 

A  total  of  86%  of  the  studies  from  1995  to  2004 
used  only  an  immediate  achievement  or  performance 
measure.  Further,  14%  used  both  an  immediate  and 
delayed  assessment.  As  shown  in  Figure  54.4,  almost 
50%  of  the  studies  used  multiple  achievement/perfor¬ 
mance  measures;  however,  only  27%  of  the  studies 
employing  a  single  assessment  collected  attitude  data, 
and  only  12%  employing  multiple  forms  of  assessment 
collected  attitude  data.  Although  almost  half  of  the  stud¬ 
ies  used  multiple  achievement  assessments,  the  data 
suggest  that  newer  studies  should  consider  richer  data 
collections  that  add  not  only  instruments  to  assess 
higher  levels  of  learning  but  also  collection  of  data  on 
time  on  task  and  individual  use  of  the  materials  (e.g., 
instructional  path/selections),  as  well  as  attitudinal  data. 

Our  final  analysis  focuses  on  the  type  of  technol¬ 
ogy  used  in  the  intervention  studies.  Figure  54.5 
shows  the  primary  technology  of  interest  used.  The 


data  for  1983  illustrate  a  usage  of  varied  types  of 
technology,  whereas  in  the  1995-2004  articles  com¬ 
puter  technology  was  dominant  in  the  intervention 
studies  analyzed. 

Have  educational  technology  researchers  become 
less  active  in  using  experimental  designs?  Although 
our  data  do  show  an  increase  in  qualitative  studies, 
the  more  prominent  gain  actually  is  reflected  in  the 
writing  of  theoretical  articles  and  literature  reviews. 
The  latter  trend  may  be  indicative  of  Hsieh  and  col¬ 
leagues’  (2005)  concern  about  the  cost  and  complex¬ 
ity  for  the  typical  academician  of  creating  the  stim¬ 
ulus  materials  and  implementing  experiments  on 
meaningful  instructional  strategies.  In  1983,  it  was 
relatively  easy  to  create  television,  print,  audio,  and 
projected  materials,  such  as  35-mm  slides  for  stimu¬ 
lus  materials.  Similarly,  it  was  fairly  straightforward 
to  create  simple  computer  programs  on  a  microcom¬ 
puter  to  serve  as  the  instructional  presentation  for 
experiments.  The  trend  for  the  1995-2004  period, 
however,  has  been  to  focus  on  computer  technology 
for  delivery  of  the  intervention  treatment.  The  com¬ 
plexity  and  cost  of  producing  multimedia  materials 
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Figure  54.5  Types  of  technology  used  in  studies. 


may  have  significantly  limited  the  number  of  indi¬ 
vidual  researchers  with  either  the  expertise  or 
resources  required  to  develop  the  individual  treat¬ 
ments  .  The  complexity  and  cost  of  technology,  how¬ 
ever,  have  always  been  an  issue  in  the  field  of  edu¬ 
cational  technology;  for  example,  compare  the  cost 
and  complexity  of  Hoban’s  (1953)  study  that  used  an 
existing  U.S.  Army  16-mm  training  film  to  that  of 
Levonian’s  (1963)  study  in  which  a  specific  film  was 
created  for  the  study.  Although  the  use  of  technology 
to  deliver  a  treatment  has  costs,  additional  benefits 
are  derived,  such  as  the  automated  collection  and 
scoring  of  data.  If  the  cost  and  complexity  of  tech¬ 
nology  are  major  barriers  to  conducting  intervention 
research,  then  researchers  might  want  to  consider 
using  print  as  the  primary  technology  and  focusing 
on  the  instructional  strategy  as  suggested  by  Clark 
(1983,  1994).  Nonetheless,  the  current  era  of  scien¬ 
tifically  based  research  is  placing  more  emphasis  than 
ever  before  on  determining  the  effects  of  real-world 
interventions  (programs,  models,  strategies)  in  rais¬ 
ing  learner  achievement  (Shavelson  et  al.,  2002; 
Slavin,  2006).  From  an  external  validity  standpoint, 
receptivity  to  basic  research  performed  with  artificial, 
simplified  lesson  material  or  delivery  systems  is 
likely  to  be  much  lower  than  was  the  case  several 
decades  ago. 

Factors  Impacting  the  Rigor  of 
Experiments  on  Instructional  Strategies 

Thus  far,  we  have  examined  varied  experimental 
designs,  their  application  to  instructional  strategies 
research,  and  trends  in  educational  technology  strate¬ 
gies  research  over  time.  In  this  concluding  section,  we 
briefly  overview,  for  the  benefit  of  researchers  and 
consumers  of  research,  major  factors  that  impact  the 
rigor  of  experiments  on  instructional  strategies. 


Types  of  Validity  Threats 

Rigorous  experiments  are  valued  for  being  high  in 
internal  validity,  defined  as  the  degree  to  which  the 
design  controls  the  effects  extraneous  variables  (Borg 
et  al.,  1993).  Common  examples  of  the  latter  include 
the  control  and  experimental  groups  differing  in  abil¬ 
ities,  one  group  receiving  more  time  to  learn  or  expo¬ 
sure  to  a  better  teacher  than  the  other,  or  an  historical 
event  (e.g.,  an  after-school  program)  impacting  the 
fidelity  of  the  treatment-control  manipulation.  In  1963, 
Campbell  and  Stanley  identified  different  classes  of 
such  threats.  Researchers  interested  in  conducting 
experiments  on  instructional  design  and  technology 
topics  should  be  familiar  with  these  factors. 

Validity  threat  means  only  that  a  factor  has  the 
potential  to  bias  results.  Knowing  about  validity 
threats  gives  the  experimenter  a  framework  for  evalu¬ 
ating  the  particular  situation  and  making  a  judgment 
about  its  severity.  Such  knowledge  may  also  permit 
actions  to  be  taken  to  limit  the  influences  of  the  validity 
threat  in  question.  Examples  are  as  follows: 

•  Concern  that  a  pretest  may  bias  post-test 
results  leads  to  the  decision  by  the  experi¬ 
menter  not  to  use  a  pretest. 

•  Concern  that  two  intact  groups  to  be  used  for 
treatment  comparisons  (quasi-experimental 
design)  may  not  be  equal  in  ability  leads  to  the 
decision  to  pretest  participants  on  ability  and 
employ  a  statistical  adjustment  (analysis  of 
covariance)  if  the  groups  significantly  differ. 

•  Concern  that  participants  may  mature  or 
drop  out  during  the  experiment  leads  to  the 
decision  to  shorten  the  length  of  the  treat¬ 
ment  period,  use  different  types  of  partici¬ 
pants,  or  introduce  incentives  to  fulfill  all 
participation  requirements. 
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•  Concern  that  the  post-test  may  differ  in  diffi¬ 
culty  from  the  pretest  leads  to  the  decision  to 
use  each  test  form  as  the  pretest  for  half  the 
students  and  the  post-test  for  the  other  half. 

Even  after  all  reasonable  actions  have  been  taken  to 
eliminate  the  operation  of  one  or  more  validity 
threats,  the  experimenter  must  still  make  a  judgment 
about  the  internal  validity  of  the  experiment  overall. 
Where  internal  validity  is  seriously  compromised,  the 
experiment  needs  to  be  redone.  In  cases  less  severe, 
experimenters  have  the  obligation  to  note  the  validity 
threats  and  qualify  their  interpretations  of  results 
accordingly. 

Balancing  Internal  and  External  Validity 

The  quest  for  high  internal  validity  orients  researchers 
to  design  experiments  in  which  treatment  manipula¬ 
tions  can  be  tightly  controlled.  In  the  process,  using 
naturalistic  conditions  (e.g.,real  classrooms)  can  prove 
challenging,  given  the  many  extraneous  sources  of 
variance  that  are  likely  to  operate  in  those  contexts; 
for  example,  the  extensive  research  conducted  on  ver¬ 
bal  learning  in  the  1960s  and  1970s  largely  involved 
instructional  strategies  for  increasing  associative  learn¬ 
ing  of  simple  words  and  nonsense  syllables  (Under¬ 
wood,  1966).  With  simplicity  and  artificiality  come 
greater  opportunity  for  control. 

This  orientation  directly  supports  the  objectives  of 
the  basic  learning  or  educational  psychology  researcher 
whose  interests  lie  in  testing  the  generalizable  princi¬ 
ples  and  laws  associated  with  treatment  strategies,  inde¬ 
pendent  of  the  specific  delivery  methods  and  contextual 
conditions  (e.g.,  teacher  preparation  and  attitude,  class¬ 
room  structure  and  resources)  involved  in  their  imple¬ 
mentation  (Ross,  2003).  Educational  technology 
researchers,  however,  are  directly  interested  in  the 
interaction  of  medium  and  method  (Kozma,  1991, 
1994;  Ullmer,  1994).  To  learn  about  this  interaction, 
realistic  media  applications  rather  than  artificial  ones 
need  to  be  established.  In  other  words,  external  validity 
becomes  as  important  a  concern  as  internal  validity. 

Basic-Applied  Design  Replications 

Once  a  principle  has  been  thoroughly  tested  in  highly 
controlled  experiments,  the  natural  progression  is  to 
evaluate  its  use  in  a  real-world  application.  Making  the 
leap  to  a  real-life  application,  however,  runs  the  risk 
of  clouding  the  underlying  causes  of  obtained  treat¬ 
ment  effects  due  to  their  confounding  with  extraneous 
variables  (Ross,  2005). 


Levin’s  Stage  Model 

In  discussing  these  issues.  Levin  (2004)  proposed  a 
model  of  educational  intervention  research  (Levin  and 
O’Donnell,  1999).  Stages  1  and  2  consist  of  formulat¬ 
ing  ideas  and  hypotheses  and  conducting  initial  obser¬ 
vations  and  pilot  work.  Stage  2  encompasses  con¬ 
trolled  laboratory  experiments  (high  internal  validity) 
on  the  strategies  of  interest,  along  with  classroom- 
based  demonstrations  and  design  experiments.  Dem¬ 
onstration  experiments  involve  implementing  the 
instructional  strategy  in  a  single  classroom  or  outside 
a  classroom  with  a  particular  group  of  students. 
Although  realism  may  be  high,  internal  validity  will 
likely  be  quite  low  due  to  the  failure  to  control  the 
effects  of  extraneous  factors  that  are  intrinsically  part 
of  the  single  classroom,  and,  because  there  is  only  one 
classroom,  generalizability  to  other  settings,  and  thus 
external  validity,  likewise  may  be  low.  The  design 
experiment,  as  described  by  Levin  (2004),  is  compa¬ 
rable  to  popular  forms  of  research  in  instructional  tech¬ 
nology,  such  as  design-based  research,  developmental 
research,  design  research,  and  formative  research 
(Wang  and  Hannafin,  2005;  also  see  Richey  and  Klein 
in  this  chapter).  Such  studies  are  also  conducted  in 
authentic  learning  environments,  but  unlike  traditional 
experiments  the  implementation  of  the  instructional 
strategies  is  not  fixed  but  modified  over  time  based  on 
formative  evaluation  data  and  associated  instructional 
design  decisions.  Stage  3  consists  of  randomized  field 
trials  (RFTs)  whereby  the  intervention  is  implemented 
under  realistic  conditions  with  randomly  selected  par¬ 
ticipants.  Outcomes  are  then  compared  to  those  for 
participants  the  randomly  selected  control  group. 

Units  of  Random  Assignment 

In  conducting  RFTs,  what  options  exist  for  the  units 
of  randomization?  As  the  first  author  has  described 
(Ross,  2005),  the  most  powerful  approach  is  to  ran¬ 
domize  at  the  student  level  within  classes.  Half  of  the 
students  in  each  participating  classroom  are  assigned 
to  the  experimental  program  and  the  other  half  to  the 
control  program.  Classroom  teachers  and  conditions 
are  thus  identical  for  both  groups,  thereby  eliminating 
major  validity  threats.  A  further  benefit  is  increasing 
the  potential  for  large  sample  sizes  (and  greater  statis¬ 
tical  power)  because  the  units  of  analysis  will  be  indi¬ 
vidual  students  rather  than  whole  classes  or  schools. 
Unfortunately,  unless  the  control  and  experimental 
interventions  can  be  administered  individually  and 
unobtrusively  by  computers  or  self-paced  manuals, 
they  may  be  mutually  interfering  or  create  unreason¬ 
able  delivery  demands  on  teachers  (Ross,  2005). 
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If  student-level  assignment  is  not  feasible, 
researchers  may  next  consider  randomizing  at  the 
classroom  level',  for  example,  half  of  the  fourth-grade 
classes  at  participating  schools  would  be  designated  to 
use  Program  A  and  the  other  half  to  use  Program  B . 
The  major  advantage  is  that  each  classroom  (and 
teacher)  is  exposed  to  only  one  program.  A  downside, 
however,  is  reducing  sample  size  and  statistical  power 
by  treating  whole  classrooms  rather  than  students  as 
the  randomized  observation  units.  Researchers  might 
also  encounter  resistance  from  principals  and  teachers 
concerned  about  creating  a  schism  in  the  school’s  aca¬ 
demic  program.  Actualizing  what  researchers  some¬ 
times  call  the  John  Henry  effect,  control  group  teachers 
might  attempt  to  outperform  the  experimental  group 
by  emulating  their  strategies  or  simply  trying  harder. 

A  third  approach  is  to  randomize  at  the  school 
level',  for  example,  half  of  the  schools  eligible  to  use 
the  experimental  program  in  a  district  would  be  ran¬ 
domly  selected  to  implement  it  and  the  other  half  to 
serve  as  control  schools.  Although  this  design  keeps 
instructional  activities  consistent  within  each  school, 
its  major  disadvantage  is  drastically  reduced  sample 
size  due  to  schools  being  the  units  of  randomization. 
Much  happens  in  a  school  separate  from  program 
effects.  Only  by  comparing  the  experimental  and  con¬ 
trol  programs  at  a  sufficiently  large  number  of  schools 
(say,  20-30  for  each)  can  we  gain  confidence  that 
program  effects  (not  school  variables)  are  the  primary 
cause  of  achievement  differences. 

Replication  Experiments 

Levin’s  model  just  reviewed  encompasses  a  progres¬ 
sion  from  nonexperimental,  demonstration,  and  design 
studies  to  RFTs.  A  complementary  model  proposed  by 
the  present  authors  (Ross  and  Morrison,  1989)  pro¬ 
motes  a  replication  of  a  highly  controlled,  experimen¬ 
tal  design  with  an  applied  design  examining  a  compa¬ 
rable  instructional  strategy.  Specifically,  Experiment  1 , 
the  basic  research  part,  would  examine  the  instruc¬ 
tional  strategies  of  interest  by  establishing  a  relatively 
high  degree  of  control  and  high  internal  validity. 
Experiment  2,  the  applied  component,  would  then 
reexamine  the  same  learning  variables  by  establishing 
more  realistic  conditions  and  high  external  validity. 
Consistency  of  findings  across  experiments  would  pro¬ 
vide  strong  convergent  evidence  supporting  the 
obtained  effects  and  underlying  theoretical  principles. 
Inconsistent  findings,  however,  would  suggest  influ¬ 
ences  of  intervening  variables  that  alter  the  effects  of 
the  variables  of  interest  when  converted  from  their  pure 
form  to  realistic  applications.  Such  contamination  may 
often  represent  media  effects ,  as  might  occur,  for  exam¬ 
ple,  when  strategies  used  with  print  material  are  natu¬ 


rally  made  more  powerful  and  effectual  via  interactive 
computer-mediated  communications  (Lou  et  al.,2006) 
and  adaptive  computer-based  instruction  (CBI) .  In  the 
latter  case,  for  example,  a  learner  who  confuses,  say, 
discovery  learning  with  inquiry  learning  in  response 
to  an  inserted  lesson  question  may  be  branched  imme¬ 
diately  to  a  remedial  CBI  frame  that  differentiates 
between  the  two  approaches,  whereas  his  or  her  coun¬ 
terpart  in  a  parallel  print  lesson  might  experience  the 
same  type  of  feedback  by  having  to  reference  the 
response  selected  on  an  answer  page  and  manually 
locate  the  appropriate  response-sensitive  feedback  in 
another  section  of  the  lesson.  The  next  implied  step  of 
a  replication  design  would  be  further  experimentation 
on  the  nature  and  locus  of  any  unique  effects  associated 
with  the  applied  context. 

As  an  example  of  basic-applied  replication  studies, 
Morrison  and  colleagues  (1995)  examined  uses  of  dif¬ 
ferent  feedback  strategies  in  learning  from  CBI.  Built 
into  the  experimental  design  was  a  factor  representing 
the  conditions  under  which  college  students  partici¬ 
pated  in  the  experiment— simulated  or  realistic.  Inter¬ 
estingly,  the  results  showed  similar  relative  effects  of 
the  different  feedback  conditions.  Examination  of 
learning  process  variables,  however,  further  revealed 
that  students  in  the  realistic  conditions  performed  bet¬ 
ter,  while  making  greater  and  more  appropriate  use  of 
instructional  support  options  provided  in  association 
with  the  feedback.  Although  the  experimentation  on  the 
simulated  condition  was  valuable  as  a  basic  and  purer 
test  of  theoretical  assumptions,  the  field  experiment 
provided  more  valid  insights  into  how  the  different 
forms  of  feedback  likely  would  be  used  in  combination 
with  other  learning  resources  on  an  actual  learning  task. 

Are  Media  Instructional  Strategies? 

A  popular  focus  of  past  research  on  instructional  strat¬ 
egies  in  educational  technology  was  comparing  differ¬ 
ent  types  of  media-based  instruction  to  one  another  or 
to  teacher-based  instruction  to  determine  which 
approach  was  “best.”  The  fallacy  or,  at  least,  unrea¬ 
sonableness  of  this  orientation,  now  known  as  media 
comparison  studies,  was  forcibly  explicated  by  Clark 
(1983)  in  his  now  classic  article  (see  also  Hagler  and 
Knowlton,  1987;  Petkovich  and  Tennyson,  1984;  Ross 
and  Morrison,  1989;  Salomon  and  Clark,  1977).  Clark 
argued  that  media  were  analogous  to  grocery  trucks 
that  carry  food  but  do  not  in  themselves  provide  nour¬ 
ishment  (i.e.,  instruction). 

For  present  purposes,  these  considerations  present 
a  strong  case  against  experimentation  that  simply  com¬ 
pares  media.  This  traditional  type  of  media  comparison 
study  still  remains  prevalent  today  and,  in  fact,  may 
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have  been  rejuvenated  from  the  current  emphasis, 
spurred  by  No  Child  Left  Behind,  on  conducting  rigor¬ 
ous  intervention  research  to  determine  what  works  in 
education  (Berliner,  2002;  Slavin,  2002).  As  an  exam¬ 
ple,  a  study  currently  completing  its  third  and  final  year 
is  using  a  large-scale  randomized  field  trial  to  determine 
the  efficacy  of  technology  applications  designed  to 
improve  student  learning  in  math  and  reading  (http:// 
edtech.mathematica-mpr.com/).  The  basic  design 
involves  random  assignment  of  teachers  within  schools 
to  treatments  using  commercial  tutorial  software  pro¬ 
grams  in  these  subjects  and  to  control  (no  software) 
treatments.  Students  taught  by  the  two  groups  of  teach¬ 
ers  are  then  compared  according  to  how  they  perform 
on  standardized  tests.  As  of  this  writing,  the  results  have 
not  been  released,  but  it  seems  fair  to  speculate  that,  if 
the  control  teachers  used  equivalent  time  to  provide 
quality  supplementary  instruction  (using  workbooks  or 
cooperative  learning),  then  the  two  comparison  groups 
might  perform  the  same.  Is  the  effectiveness  of  technol¬ 
ogy  really  being  tested  here  or  is  it  the  efficacy  of  the 
types  of  instructional  strategies  the  software  developers 
chose  to  incorporate  in  their  programs?  A  more  mean¬ 
ingful  focus,  it  would  seem,  is  to  analyze  how  unique 
attributes  of  the  technology  enable  specialized  or 
improved  usage  of  specific  instructional  strategies  (e.g., 
remedial  branching)  that  work  effectively  for  certain 
types  of  learners  and  contexts. 

A  second  type  of  inappropriate  media  comparison 
experiment  is  to  create  artificially  comparable  alterna¬ 
tive  media  presentations,  such  that  both  variations  con¬ 
tain  identical  attributes  but  use  different  modes  of 
delivery.  So,  for  example,  if  a  researcher  wants  to 
explore  the  effects  of  the  computer  as  a  medium  for 
distance  education  (DE)  learning  in  algebra  (Lou  et 
al.,  2006),  it  makes  little  sense  to  develop  a  course  that 
is  an  exact  replica  (90-minute  lectures  and  all)  of  the 
traditional  algebra  course  used  at  the  same  schools.  In 
fact,  the  first  author  is  currently  part  of  a  team  con¬ 
ducting  a  randomized  field  trial  of  a  hybrid  algebra 
course  that  combines  DE  with  in-class  teacher  coach¬ 
ing.  For  both  educational  and  scientific  purposes,  the 
primary  interest  is  making  the  hybrid  course  as  pow¬ 
erful  as  possible  given  the  unique  capabilities  of  the 
combined  DE  and  teacher-based  interventions.  The 
instructional  strategies  embedded  in  the  hybrid  and 
traditional  courses  will  necessarily  differ  but  represent 
realistic  and  ecologically  valid  interventions. 

One  recommended  alternative  to  media  compari¬ 
sons  is  to  test  a  priori  hypotheses  of  differences 
between  the  two  media  presentations  based  directly  on 
analyses  of  their  different  attributes  (Kozma,  1991, 
1994).  It  might  be  hypothesized,  for  example,  that  for 
teaching  an  instructional  unit  on  a  cardiac  surgery  pro¬ 


cedure,  a  conventional  lecture  presentation  might  be 
superior  to  an  interactive  video  presentation  for  facil¬ 
itating  retention  of  factual  information,  whereas  the 
converse  would  be  true  for  facilitating  meaningful 
understanding  of  concepts.  The  rationale  for  these 
hypotheses  would  be  based  on  analyses  of  the  special 
capabilities  (embedded  instructional  strategies)  of  each 
medium  in  relation  to  type  of  material  taught.  Findings 
would  be  used  to  support  or  refute  these  assumptions. 

A  second  recommended  study,  which  we  have 
called  media  replications  (Ross  and  Morrison,  1989), 
examines  the  consistency  of  effects  of  given  instruc¬ 
tional  strategies  delivered  by  alternative  media.  Consis¬ 
tent  findings,  if  obtained,  are  treated  as  corroborative 
evidence  to  strengthen  theoretical  understanding  of  the 
instructional  variables  in  question  as  well  as  claims 
concerning  the  associated  strategy’s  effectiveness  for 
learning.  If  inconsistent  outcomes  are  obtained,  meth¬ 
ods  and  theoretical  assumptions  are  reexamined  and  the 
target  strategy  subjected  to  further  empirical  tests  using 
diverse  learners  and  conditions.  Thus,  media  replica¬ 
tions  provide  valuable  tests  of  the  generalizability  of 
the  effects  of  particular  instructional  strategies. 

Summary 

The  focus  of  this  chapter  section  is  research  on  instruc¬ 
tional  strategies  involving  applications  of  educational 
technology.  Major  concepts  and  points  addressed 
included  the  following: 

•  Experimental  studies  have  been  the  pre¬ 
ferred  choice  of  researchers  for  investigating 
instructional  strategies. 

•  Usage  of  experimental  designs  in  educational 
technology  and  in  educational  psychology 
research  in  general  has  declined  over  the  past 
several  decades.  A  primary  reason  seems  to 
be  the  increased  logistical  demands  of  con¬ 
ducting  experiments  on  meaningful  interven¬ 
tions;  another  reason  has  been  the  growing 
popularity  of  qualitative  and  correlational 
research.  The  current  emphasis  on  scientifi¬ 
cally  based  research,  however,  spurred  by  the 
No  Child  Left  Behind  legislation,  is  creating 
a  resurgence  in  the  popularity  and  status  of 
experimental  studies. 

•  True  or  randomized  experiments  involve  ran¬ 
domly  assigning  participants  to  treatment 
conditions.  Quasi-experiments,  however,  use 
preexisting  groups  or  convenience  samples 
that  are  not  randomly  composed.  By  reducing 
sampling  bias  as  a  validity  threat,  true  experi¬ 
ments  typically  have  greater  internal  validity— 
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the  ability  to  attribute  outcomes  to  the  treat¬ 
ment  alone— than  do  quasi-experiments. 

•  Instructional  strategies  are  the  prescribed 
sequences  and  methods  of  instruction  to 
achieve  an  objective.  The  particular  prescrip¬ 
tions  are  not  only  determined  by  the  content 
and  performance  specified  in  the  learning 
objectives  but  also  by  underlying  learning 
theory.  Practical  and  scientifically  grounded 
heuristics  can  be  derived  to  guide  the  iden¬ 
tification  of  strategies  for  teaching  certain 
types  of  content,  such  as  facts,  concepts, 
principles  and  rules,  procedures,  interper¬ 
sonal  skills,  and  attitudes. 

•  Micro-instructional  strategies  focus  on 
teaching  specific  content  information  such 
as  facts  and  concepts.  Macro-instructional 
strategies  such  as  cooperative  learning,  prob¬ 
lem-based  learning,  and  feedback  represent 
broader,  more  complex  instructional  strate¬ 
gies  that  affect  the  design  of  a  course.  Mes¬ 
sage  design  research  has  focused  on 
perception  issues  such  as  pictures,  colors, 
and  legibility.  More  recently,  the  work  on 
cognitive  load  theory  also  has  been  classified 
within  the  message  design  category. 

•  In  examining  research  design  trends  in  arti¬ 
cles  published  over  several  decades  in  Edu¬ 
cational  Technology  Research  and 
Development  ( ETR&D ),  we  found  a  decline 
in  experiments  on  instructional  strategies 
and  an  increase  in  qualitative  studies.  The 
complexity  and  cost  of  producing  multime¬ 
dia  materials  may  have  significantly  limited 
the  number  of  individual  researchers  with 
either  the  expertise  or  resources  needed  to 
develop  the  individual  treatments. 

•  Researchers  interested  in  conducting  rigor¬ 
ous  experiments  on  instructional  strategies 
need  to  be  aware  of  potential  validity  threats 
such  as  history,  maturation,  testing,  instru¬ 
mentation,  statistical  regression,  selection, 
mortality,  and  diffusion  of  treatments. 
Achieving  balance  between  internal  validity 
(control)  and  external  validity,  and  between 
basic  vs.  applied  research  (generalizability) 
is  also  important. 

•  Comparing  media  is  less  important  scientif¬ 
ically  and  practically  than  well-designed 
experiments  investigating:  (1)  which  instruc¬ 
tional  strategies  are  most  effectual  and  prac¬ 
tical  using  different  media,  and  (2)  how  the 
effects  of  instructional  strategies  are  repli¬ 
cated  or  mitigated  by  media  characteristics. 


RESEARCH  ON 

EDUCATIONAL  TECHNOLOGIES 

Robert  D.  Hannafin  and  Michael  Young 

Technology  . . .  Does  it  work?  There  is  probably  not  a 
more  frequently  asked  question  of,  and  by,  educational 
technology  researchers.  It  is  asked  by  teachers,  now 
expected  to  wisely  integrate  new  tools;  by  parents 
shopping  for  a  new  home  computer  that  will  help  their 
children  in  school;  by  school  boards,  who  need  to 
justify  expensive  network  improvements;  and  by  the 
taxpayers,  who  suffer  a  higher  mil  rate  to  pay  for  them. 
It  is  a  fair,  and  on  the  face  of  it,  simple  question,  one 
that  so-called  experts  should  be  able  to  answer;  how¬ 
ever,  despite  a  substantial  research  effort,  the  answer 
has  not  come  and  if  anything  has  led  to  more  questions . 
In  this  section,  we  use  the  question  of  Does  it  work ? 
to  frame  the  discussion  of  research  in  technology. 

Germane  to  answering  this  simple  question  are 
issues  of  research  methodology,  contemporary  under¬ 
standing  of  how  people  think  and  learn,  and  consensus 
among  educational  researchers  as  to  what  constitutes 
evidence  of  improved  learning  and  enhanced  thinking 
abilities;  for  example,  some  would  argue  that  learning 
occurs  only  in  the  heads  of  the  users  and  thus  would 
accept  a  test  of  facts  and  procedures  retrieved  from 
memory  as  evidence  of  learning.  Such  constructivists 
contend  that  knowledge  is  accumulated  and  stored  by 
the  brain,  much  as  a  computer  stores  and  retrieves 
information.  Others,  however,  would  argue  that  knowl¬ 
edge  and  practice  are  deeply  intertwined  in  the  situa¬ 
tions  and  context  in  which  they  occur.  They  would 
argue  that  we  do  not  want  students  who  know  disem¬ 
bodied  facts  and  cannot  apply  them  to  solve  problems, 
nor  do  we  want  robots  who  act  without  reflection  or 
understanding  of  the  procedures  they  are  doing.  So, 
from  this  perspective,  knowing  must  be  enacted  and  is 
referred  to,  variously,  as  situated,  embedded,  and 
embodied  cognition.  For  this  group,  test  performance 
is  not  enough.  They  seek  evidence  that  students  can 
use  their  knowledge  in  context,  in  the  lived-in  world, 
to  address  problems  that  matter.  Such  differences  in 
perspective  lead  to  very  different  conceptions  for  “it” 
and  “works”  when  asking  Does  it  workl 

In  this  section,  we  discuss  what  it  is  and  how  the 
definition  has  evolved  over  the  past  30  years .  We  briefly 
examine  a  sample  of  studies  over  this  period  to  provide 
examples  of  that  evolution.  The  blurring  of  the  bound¬ 
aries  between  design  and  research  is  then  discussed  in 
the  context  of  several  recent  technology-enhanced 
learning  environments  (Wang  and  Hannafin,  2005). 
Finally,  we  describe  several  of  the  data  analysis  and 
research  design  challenges  presented  by  these  new 
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environments  and  the  reconception  of  it.  In  this  section, 
we  do  not  hope  to  cover  all  types  of  research  relevant 
to  this  discussion;  rather  we  hope  to  stimulate  discus¬ 
sion  and  to  provide  a  framework  for  that  discussion. 

When  “It”  Was  Simple 

What  Is  “It”? 

It  can  be  defined  for  our  purposes  as  any  treatment  that 
uses  technology  to  affect  a  predetermined  outcome.  The 
question  asked  over  the  years  by  stakeholders  and 
researchers  alike  has  often  taken  the  form:  Do  comput¬ 
ers  improve  learning ?  Hopes  were  high  in  the  1970s 
and  1980s  that  the  computer  alone  would  positively 
impact  learning,  this  despite  the  fact  that  previous  media 
innovations  (e.g.,  film,  radio,  television)  had  consis¬ 
tently  failed  to  have  the  expected  effect  (Cuban,  1986). 
The  computer  generation  of  media  comparison  studies 
was  exclusively  experimental  and  quasi-experimental 
and  examined  whether  one  delivery  medium  (computer) 
was  favored  over  another  (book  or  worksheet).  In  a 
typical  media  comparison  study,  all  other  variables  were 
controlled  in  an  attempt  to  prove  that  instruction  deliv¬ 
ered  by  computers  was  more  effective  than  the  same 
instruction  delivered  via  traditional  means  (for  a  sum¬ 
mary,  see  Clark,  1983).  To  prove  causality,  it  was  nec¬ 
essary  to  keep  constant  all  other  instructional  noise  like 
instructional  strategies  and  methods;  thus,  these  studies, 
when  well  designed,  enabled  researchers  to  answer  the 
question  Does  it  work? ,  where  it  was  simply  the  com¬ 
puter,  and  work  was  performance  on  a  post-test  or  some 
other  learning  outcome. 

The  value  of  this  line  of  inquiry  was  questioned  by 
several  researchers,  most  notably  Clark  (1983)  in  his 
seminal  work,  “Reconsidering  Research  on  Learning 
from  Media.”  Clark  argued  that  efforts  to  privilege  the 
role  of  any  medium  in  learning  should  stop  altogether. 
He  pointed  out  that  many  of  the  studies  attributing  learn¬ 
ing  gains  to  media  either  had  unexplored  alternative 
explanations  or  had  unintentionally  attributed  the  impact 
of  the  instructional  methods  to  the  delivery  medium, 
with  methods  defined  as  “the  conditions  which  can  be 
implemented  to  foster  the  acquisition  of  competence” 
(Glaser,  1976,  p.  1,  as  cited  in  Clark,  1983). 

Technology  as  Conveyances:  The  CAI  Era 

The  majority  of  the  empirical  technology  research 
from  the  late  1970s  through  the  1990s  would  fall  under 
what  Salomon  and  Clark  (1977)  and  later  Clark  (1983) 
termed  learning  with  media.  These  studies  used  tech¬ 
nology  (usually  computers)  as  conveyances  of  the 
treatment  (e.g.,  computer-assisted  instruction,  or  CAI) 


to  examine  and  isolate  other  variables  of  interest 
(Clark,  1983),  but,  although  computers  were  not  the 
variable  of  study,  the  research  goal  of  both  categories 
was  the  same— that  is,  to  experimentally  isolate  vari¬ 
ables  to  prove  a  causal  relationship  with  learning.  For 
educational  technology  research  to  be  taken  seriously 
and  to  empirically  prove  causality,  studies  must  have 
treatment  fidelity.  This  need  can  be  traced,  according 
to  Ross  and  Morrison  (1989;  this  chapter),  to  the  field’s 
roots  in  educational  psychology  and  the  inherent  value 
placed  on  studies  with  high  internal  validity.  We  iden¬ 
tified  three  types  of  studies  that  used  technology 
(mainly  computers)  as  conveyances:  aptitude-treat¬ 
ment  interaction,  learner  control,  and  in-program 
learner  behaviors  studies.  We  discuss  each  in  turn. 

Aptitude-Treatment  Interactions 

Some  of  the  work  where  computers  were  conveyances 
of  the  treatment  include  aptitude-treatment  interaction 
(ATI)  research  (Cronbach  and  Snow,  1981;  DiVesta, 
1975)  where  aptitude  is  a  learner’s  incoming  knowl¬ 
edge,  skills,  and  personality  traits  and  treatment  is  the 
condition  or  environment  that  supports  learning  (Shute, 
1993).  ATI  research  began  before  computers  became 
popular,  but  many  researchers  took  advantage  of  the 
computer’s  ability  to  customize  instructional  treatments 
(e.g.,  adapting  questions  to  prior  performance  or  skip¬ 
ping  presentations  based  on  pretest  performance)  to 
renew  the  effort  to  find  elusive  interactions  with  learner 
aptitudes  (Cronbach  and  Snow,  1981).  Unfortunately, 
despite  the  design  flexibility  the  computers  afforded 
researchers,  identifying  stable  ATIs  that  could  prescribe 
instructional  programs  remained  elusive. 

Learner  Control 

Learner  control  studies  represent  another  line  of  inquiry 
in  the  computer-as-conveyance  category.  In  these  stud¬ 
ies,  computers  were  typically  used  to  allow  learners 
some  control  over  elements  of  their  instruction  and  to 
track  learner  choices  and  options  use  during  an  instruc¬ 
tional  program.  Kinzie  et  al.  (1988),  for  example,  found 
that  students  given  control  over  content  review  scored 
higher  on  a  post-test  than  students  without  that  control. 
Gray  (1987)  reported  that  students  controlling  the 
sequence  of  their  instruction  scored  higher  on  one  of 
two  achievement  measures  than  learners  under  program 
control.  Ross  et  al.  (1989)  found  that  college  students 
granted  choice  of  presentation  medium  (print  vs.  com¬ 
puter)  for  their  instruction  achieved  more  than  students 
with  no  choice.  Tennyson  and  associates  noted  that 
college  students  benefited  from  controlling  elements  of 
their  own  instruction,  but  only  when  advised  on  their 
performance  relative  to  program  criteria  (Tennyson  and 
Buttrey,  1980;  Tennyson  and  Rothen,  1979).  On  the 
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other  hand,  students  did  not  always  make  wise  instruc¬ 
tional  choices.  Pollock  and  Sullivan  (1990)  reported 
that  learners  receiving  required  practice  scored  higher 
on  a  post-test  than  learners  who  could  bypass  the  prac¬ 
tice.  Tennyson  and  Rothen  (1979)  cautioned  that  stu¬ 
dents  tend  to  make  poor  instructional  choices  more 
frequently  in  complex  instructional  tasks,  and  Carrier 
(1984)  reported  that  this  was  also  true  when  they  lack 
sufficient  prior  knowledge  .  Researchers  have  also  noted 
that  learners  given  guidance  by  their  instructional  pro¬ 
gram  seem  to  make  better  choices  and  achieve  more 
than  learners  given  no  guidance  (Ross  and  Rakow, 
1981;  Tennyson,  1980). 

A  great  many  studies  also  examined  student  ability, 
with  and  without  learner  control.  The  hope  here  was 
that  because  the  computer  was  infinitely  patient  and 
because  of  its  ability  to  deliver  large  amounts  of  prac¬ 
tice  and  feedback  and  to  adapt  instruction  dynamically 
to  adjust  to  learner  needs,  low-ability  learners  in  par¬ 
ticular  would  benefit.  On  balance,  there  is  some  evi¬ 
dence  that  lower  ability  students  benefit  more  from 
program  control  than  when  they  are  afforded  some 
control  (Goetzfried  and  Hannafin,  1985;  Ross  and 
Rakow,  1981).  Hativa  (1988)  found  that  high-ability 
students  in  a  learner-control  treatment  spent  more  time 
on  task  than  low-ability  students.  Others  have  con¬ 
cluded  that  low-ability  learners  lack  the  metacognitive 
skills  to  manage  their  own  instruction  (Chung  and 
Reigeluth,  1992;  Hannafin,  1984).  Thus,  ceding  con¬ 
trol  to  learners  is  effective  in  certain  circumstances  but 
not  in  others  (for  a  summary,  see  Hannafin,  1984). 

In-Program  Learner  Behaviors 

Many  studies  have  used  the  computer  to  track  learner 
en  route  behaviors  such  as  option-selection  patterns 
and  time  on  task  during  CAI  programs  (Hannafin  and 
Sullivan,  1995;  Hicken  et  al.,  1992).  In  studies  by 
Carrier  and  her  associates,  sixth  graders  were  found  to 
select  more  options  earlier  in  a  program  on  advertising 
than  they  did  toward  the  end  of  the  program  (Carrier 
et  al.,  1984, 1986),  a  pattern  also  detected  by  Hannafin 
and  Sullivan  (1996)  among  high-school  students  work¬ 
ing  in  a  geometry  CAI  program.  Kinzie  and  associates 
(Kinzie  and  Sullivan,  1989;  Kinzie  et  al.,  1988) 
reported  that  students  in  a  learner-control  group, 
despite  bypassing  instances  of  content  review,  spent  a 
similar  amount  of  time  overall  on  the  instructional 
program  as  did  learners  who  were  required  to  see  the 
review.  Tennyson  (1980)  found  that  college  students 
given  control  over  the  amount  and  sequence  of  ele¬ 
ments  of  an  instructional  program  spent  less  time  on 
the  program  and  chose  fewer  options  than  students 
who  were  advised  by  the  program.  According  to  Car¬ 
rier  and  colleagues  (1986),  tracking  on-task  behaviors 


is  important  to  understanding  why  some  learners  make 
more  prudent  choices  than  others. 

These  experimental  and  quasi-experimental  studies 
sought  to  control  conditions  and  isolate  the  independent 
variable  (it)  to  prove  an  impact  on  learning.  It  was  sim¬ 
ple,  or  pure.  We  do  not  mean  to  suggest,  however,  that 
the  research  was  simple,  or  simplistic.  It  is  arguably 
more  difficult  to  design  and  conduct  an  experimental 
study  with  high  internal  validity  than  any  other  research 
design  (see  Ross  and  Morrison  in  this  chapter),  but  the 
value  of  tightly  controlled  laboratory  studies  and  the 
separation  of  media  and  methods  began  to  be  questioned 
by  educational  technology  researchers  in  the  late  1980s. 

Then  “It”  Got  Complicated 

“It”  and  the  Great  Media  Debate 

Kozma  (1991,  1994)  agreed  with  most  of  Clark’s 
(1983 , 1994)  arguments  but  took  exception  to  the  some¬ 
what  circular  nature  of  part  of  Clark’s  logic.  Kozma 
argued  that  if,  as  he  believed,  powerful  learning  oppor¬ 
tunities  reside  in  the  interactions  between  method  and 
medium  and  if  researchers  were  not  allowed  to  look  at 
those  in  the  name  of  design  integrity,  then  how  could 
they  ever  be  detected?  Thus,  in  the  context  of  this 
discussion,  Kozma  was  calling  for  a  broader,  more 
inclusive  definition  of  it,  conceding  the  inability  to 
know  exactly  what  part  of  a  treatment  caused  learning 
to  occur;  whereas,  Clark  argued  for  a  definition  of  it  as 
media  alone,  insisting  that  was  the  only  way  we  can 
prove  causality.  In  many  ways,  this  great  media  debate 
exemplifies  the  tension  described  by  Ross  and  Morrison 
between  the  need  for  rigorous  research  design  (internal 
validity)  and  the  need  to  build  realistic,  ecologically 
valid  instructional  environments  (external  validity). 

The  impact  of  Clark’s  work  was  profound.  It 
caused  many  researchers  to  rethink  their  designs,  be 
more  cautious  about  causality  claims,  and  to  acknowl¬ 
edge  limitations  at  work  when  confounding  factors  are 
suspected,  but  his  most  lasting  contribution  is  perhaps 
prompting  a  lively  discussion  that  eventually  led  to 
many  clarifications.  Together,  Clark  and  Kozma 
sparked  a  conversation  that  eventually  led  to  more 
accurate  definitions  and  clearer  vocabulary  for  a  gen¬ 
eration  of  researchers.  One  merely  needs  to  cite  the 
phrase  “grocery  truck  analogy”  to  be  universally 
understood  as  referring  to  the  separation  of  media  and 
method.  The  great  media  debate  in  a  sense  was  all 
about  the  question  Does  it  workl  Though  the  great 
debate  focused  on  just  the  delivery  medium,  it  raised 
issues  that  related  quite  directly  to  how  the  field  per¬ 
ceived  it,  what  was  considered  good  research,  and  what 
was  worth  investigating. 
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Before  Kozma’s  (1991)  response  to  Clark,  Ross 
and  Morrison  (1989)  argued  that,  because  educational 
technology  researchers  tend  to  have  a  genuine  interest 
in  the  technology  aspect  of  the  treatments  they  inves¬ 
tigate,  it  was  reasonable  to  value  the  interaction  of 
medium  and  method,  rather  than  viewing  it  as  an 
unwanted  source  of  error  variance.  Thus,  they  argued, 
external  validity  — or  the  extent  to  which  results  can 
be  applied  to  real-life  settings  — should  be  an  important 
design  consideration.  They  also  said  that  separating 
media  from  methods  may  not  always  be  desirable,  in 
a  sense  giving  the  held  permission  to  consider  a 
broader  definition  of  it. 

OLEs  and  TELEs 

About  the  same  time  Ross  and  Morrison  (1989)  were 
making  this  case  to  education  technology  researchers 
from  a  methodological  point  of  view,  a  powerful 
instructional  movement  emerged,  with  competing  the¬ 
oretical  assumptions  that  unapologetically  broke  many 
conventions  that  caused  earlier  debate.  The  phrase 
“open  learning  environments”  (OLEs)  has  been  used 
to  describe  a  wide  array  of  instructional  projects  based, 
to  varying  degrees,  on  constructivist  principles.  Con¬ 
structivism  as  a  theory  of  how  people  learn  had  been 
around  for  a  number  of  years,  but  little  attempt  was 
made  to  instantiate  any  of  its  tenets  into  an  instructional 
model  or  product  that  would  enable  researchers  to  test 
the  theory.  In  other  words,  constructivist  learning 
remained  a  theoretical  proposition— one  could  experi¬ 
ence  it  in  real-life  situations,  but  no  one  knew  how 
constructivist  learning  could  be  proactively  applied 
toward  the  design  of  instruction  in  schools.  Advances 
in  technology  changed  that  by  allowing  problem  situ¬ 
ations  to  be  enacted  in  realistic  and  engaging  contexts. 
In  this  section,  we  use  the  term  technology-enhanced 
learning  environments  (TELEs),  which  was  used  by 
Wang  and  Hannafin*  (2005,  p.  5)  to  describe  “technol¬ 
ogy  based  learning  and  instructional  systems  through 
which  students  acquire  skills  or  knowledge,  usually 
with  the  help  of  teachers  or  facilitators,  learning  sup¬ 
port  tools,  and  technological  resources.” 

Then  “It”  Became  a  “Cloud 
of  Correlated  Variables” 

Another  pivot  point  in  the  evolution  of  it  was  the 
Salomon  et  al.  (1991)  essay  in  Educational  Researcher 
where  they  argued  that  learning  effects  were  possible 
for  learners  who  mindfully  engaged  with  intelligent 


*  Wang  and  Hannafin  based  this  description  on  work  by  Aleven  et  al. 
(2003),  Land  (2000),  and  Shapiro  et  al.  (1995). 


technologies  such  as  TELEs.  One  example  the  authors 
cited  was  where  students  who  used  the  computer  pro¬ 
gram  STELLA  (Richmond,  1985)  to  construct  math¬ 
ematical  models  of,  say,  ecological  systems  were  able 
to  manipulate  variables,  values,  and  generate  and  test 
hypotheses  without  needing  to  first  memorize  discrete 
facts.  The  tool,  they  argued,  allowed  learners  to  orga¬ 
nize  ideas  according  to  deep  rather  than  surface  crite¬ 
ria,  an  ability  more  typical  of  the  expert  than  novice 
learner.  They  asserted  that  such  mindful  partnerships 
in  TELEs  support  the  notion  that  computers  can  be 
used  as  what  Pea  (1985)  called  cognitive  tools.  Further, 
they  argued  that  the  tools  (Salomon  et  al.,  1991,  p.  4): 

. . .  potentially  allow  a  learner  to  function  at  a  level  that 
transcends  the  limitations  of  his  cognitive  systems. 
...Indeed  it  can  be  argued  that  work  with  specific  com¬ 
puter  tools  might  accomplish  more  than  just  enabling  the 
beginner  to  do  the  same  thing,  faster  and  with  less  effort. 
These  tools  might  redefine  and  restructure  the  learning 
or  performance  task  much  as  the  pencil  has  qualitatively 
restructured  the  act  of  remembering.  . .  .In  sum,  the  intel¬ 
lectual  partnership  with  such  tools  can  change  the  ratio 
between  accessing  prior  knowledge  and  construction  of 
new  knowledge  in  favor  of  the  latter.  ...A  partnership 
between  a  human  and  technology  could  be  far  more 
“intelligent”  than  the  performance  of  the  human  alone. 

They  then  made  the  case  that  the  impact  of  technology 
is  cultural,  and  cannot  be  measured  in  isolation 
(Salomon  et  al.,  1991,  p.  8): 

...  profound  effects  of  intelligent  technology  on  minds 
can  take  place  only  when  major  changes  in  the  culture 
take  place  as  well.  No  important  impact  can  be  expected 
when  the  same  old  activity  is  carried  out  with  a  technol¬ 
ogy  that  makes  it  a  bit  faster  or  easier;  the  activity  itself 
has  to  change,  and  such  a  change  cannot  take  place  in  a 
cultural  vacuum.  . .  .But  this  means  that  it  is  not  the  tech¬ 
nology  alone  affecting  minds  but  the  whole  “cloud  of 
correlated  variables”** — technology,  activity,  goal,  set¬ 
ting,  teacher’s  role,  culture  — exerting  their  combined 
effects. 

Jasper  Woodbury  Series 

One  of  the  earliest  and  best  known  examples  of  a  TELE 
was  the  Jasper  Woodbury  series  developed  by  the  Cog¬ 
nition  and  Technology  Group  at  Vanderbilt  (CTGV, 
1990,  1993).  The  work  by  the  CTGV  took  seriously 
the  idea  that  learning,  in  this  case  middle-school  math¬ 
ematics  concepts  such  as  distance-rate-time,  are  “sit¬ 
uated”  and  that  the  best  situations  are  those  that  have 
many  attributes  of  realism,  those  that  “ring  true”  for 


**  The  authors  credit  Scarr  (1985)  with  this  analogy. 
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everyday  cognition  (Rogoff  and  Lave,  1999)  and 
engage  one’s  “cognition  in  the  wild”  (Hutchins,  1995). 
This  perspective  led  to  very  different  research  ques¬ 
tions,  specifically  different  conceptions  of  what  it 
should  be.  And,  by  instantiating  a  theory  of  situated 
learning  that  takes  as  a  working  premise  the  nonlinear 
multidimensional  interactive  nature  of  learning,  simple 
static  (pre-post)  assessments  of  its  success  were  not 
enough.  To  detect  if  the  Jasper  Woodbury  series 
worked  required  as  a  starting  premise  that  assessment 
itself  also  had  to  be  situated,  inherently  the  result  of 
student-environment  interactions,  in  the  service  of 
both  student  learning  and  effective  teaching,  and  dif¬ 
ferent  in  each  new  circumstance. 

Design-Based  Research 

Struggling  with  similar  issues.  Brown  ( 1992)  proposed 
a  hybrid  cycle  of  prototyping,  classroom  field  testing, 
and  laboratory  study  that  drew  from  the  work  of  engi¬ 
neering  research,  and  she  called  this  approach  design 
research.  Design  research  combines  qualitative  and 
quantitative  analysis  and  rejects  the  call  for  educa¬ 
tional  research  to  make  scientific,  controlled  studies 
that  prove  causality  the  sine  qua  non  of  scholarly  work 
in  education.  Using  mixed  methods  optimizes  both 
internal  and  external  validity  simultaneously  while 
acknowledging  other  forms  of  test  validity  such  as 
ecological  and  systemic  validity  (Messick,  1995). 
Design-based  research  tends  to  theoretically  align  with 
the  situated  cognition  tradition,  which  suggests  that 
assessments  should  be  seamless  because  they  them¬ 
selves  are  the  result  of  embedded  and  embodied  cog¬ 
nition.  In  sum,  the  techniques  of  design  research  seek 
a  dialectic  between  theory  and  practice  that  contributes 
in  a  scholarly  way  to  both  activities. 

This  approach  continues  to  be  controversial  (see 
the  special  issue  of  Educational  Researcher ,  32(1), 
2003)  but  has  ignited  a  dialog  among  researchers  as 
to  the  value  of  seeking  only  causal  results  that  rarely 
generalize  to  school  setting,  or  descriptions  of  settings 
that  fail  to  inform  theoretical  model-building.  Iterative 
techniques  that  use  both  qualitative  and  quantitative 
analyses  for  multilevel  descriptions  of  learning  provide 
quite  a  challenge  to  most  existing  research  frame¬ 
works,  because  they  reject  the  standard  linear  model 
and  instead  propose  nonlinear  multidimensional  com¬ 
plex  systems  that  are  at  the  very  least  emergent  from 
student  and  environment  interactions  across  time.  Such 
effects  are  not  available  for  averaging  across  partici¬ 
pants  and  require  the  unit  of  analysis  focus  on  the 
agent-environment  interactions  rather  than  either  the 
student’s  performance  or  the  environment  alone 
(Kulikowich  and  Young,  2001). 


In  the  spring  2006,*  we  had  the  privilege  to  inter¬ 
view  developers  of  seven  renowned  TELEs  who  have 
been  trying  to  answer  Does  it  work?  by  conducting 
design  research  (in  some  form)  as  advocated  by  Brown 
(1992).  We  next  discuss  three  of  those:  Model-lt ,  Alien 
Rescue ,  and  River  City.  Each  TELE  is  different,  to  be 
sure,  with  distinct  learning  assumptions  and  radically 
different  design  considerations;  however,  like  the 
CTGV,  they  share  the  premise  that  learning  is  situated, 
inextricably  linked  to  the  environment,  including  the 
medium  (CTGV,  1990;  Kozma,  1994).  Also  driving 
the  development  of  these  robust  and  student-centered 
TELEs  was  the  reconception  of  what  constitutes  learn¬ 
ing,  or,  again  in  the  context  of  our  super  ordinate 
question,  what  is  meant  by  work  when  asking  Does  it 
work 1  These  programs  sought  to  support  deep  under¬ 
standing  and  elicit  performances  as  evidence  of  that 
understanding. 

Model-lt 

The  designers  of  Model-lt  sought  to  create  a  cognitive 
tool  that  was  flexible  and  adaptive,  that  would  allow 
for  the  building  and  testing  of  systems  models  with 
varying  degrees  of  complexity,  and  that  would  facili¬ 
tate  a  learner’s  transition  to  more  expert-like  modeling 
practices  and  reasoning  (Metcalf  et  al.,  2000).  The 
designers  believed  that  the  computer  could  amplify  the 
range  of  activities  possible  to  learners,  and  they  argued 
that  computer-based  learning  environments  in  which 
the  learner  actively  constructs  an  artifact  have  strong 
potential  for  promoting  cognitive  growth  (Jackson  et 
al.,  1995).  Citing  Salomon  (1990),  Jackson  and  col¬ 
leagues  argued  that  cognitive  capacities  are  extended 
while  using  visualization  technology;  not  only  do 
learners  accomplish  more  with  the  computer  than 
would  be  possible  without,  but  they  also  engage  in 
much  more  mindful  learning  with  computers  than 
without  computers.  During  the  first  implementation 
phase  of  Model-lt,  eighth- grade  students  built  a  model 
of  a  pond’s  ecosystem  after  systematically  taking  mea¬ 
surements  of  the  different  chemicals  found  in  the  pond 
over  time.  The  researchers  found  that,  when  students 
were  able  to  collect  and  analyze  authentic  data  and 
represent  those  data  in  a  working  model  that  accurately 
portrayed  the  interrelationships  among  the  elements  in 
the  ecosystem,  they  were  also  able  to  test  and  refine 
their  mental  representations  and  understandings  of  that 
system  (Metcalf  et  al„  2000). 


*  As  part  of  a  graduate  course  in  interactive  learning  environments,  the 
authors  interviewed  Dr.  Elliot  Soloway  (Model-lt),  Dr.  Janet  Kolodner 
(Learning  by  Design),  Dr.  James  Slotta  (WISE),  Dr.  Chris  Dede  (River 
City),  Dr.  Sasha  Barab  (Quest  Atlantis),  Drs.  Min  Liu  and  Susan  Ped¬ 
erson  (Alien  Rescue),  and  Drs.  Thomas  Brush  and  John  Saye  (Persistent 
Issues  in  History). 
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Alien  Rescue 

Alien  Rescue  is  a  simulation  that  opens  with  a  video 
news  report  that  six  species  of  alien  life  forms  are  on 
spaceships  orbiting  Earth.  Students  learn  that,  because 
the  home  worlds  of  the  aliens  have  been  destroyed,  the 
aliens  have  come  to  Earth  to  ask  for  help  in  finding 
new  homes  in  our  solar  system.  To  add  a  sense  of 
urgency,  before  they  can  establish  contact  with  Earth 
their  spaceship  is  damaged,  forcing  them  to  enter  a 
suspended  state  of  animation.  The  student  scientists 
are  sent  to  the  newly  operational  international  space 
station  orbiting  Earth  to  participate  in  a  multinational 
rescue  operation  to  save  the  aliens.  Their  task  is  to  use 
a  variety  of  tools  at  their  disposal  to  learn  about  the 
aliens  and  to  explore  the  solar  system  to  find  a  suitable 
habitat  to  match  the  unique  characteristic  of  each  alien 
species.  One  of  the  challenging  research  questions  in 
this  environment  involves  finding  which  of  the  tools 
are  most  helpful  in  solving  the  problem  (Liu  and  Bera, 
2005;  Liu  et  al.,  2002).  Alien  Rescue  contains  an  amal¬ 
gam  of  strategies,  scaffolds,  and  technologies  that 
make  it  difficult,  if  not  impossible,  to  separate  medium 
from  methods. 

River  City 

Three-dimensional  multi-user  virtual  environments 
(MUVEs)  such  as  River  City  illustrate  the  complexity 
of  understanding  and  analyzing  the  interactions 
between  learner  and  environment.  River  City  is  a  class- 
room-based  situated-learning  experience  to  support 
problem  solving  in  biology  and  ecology  (Dede,  2003). 
Students  explore  a  three-dimensional  virtual  19th-cen¬ 
tury  town  to  discover  why  its  citizens  (represented  as 
non-player  agents)  are  getting  sick.  Three  strands  of 
illness  (waterborne,  airborne,  and  insect-borne)  are 
integrated  with  historical,  social,  and  geographical 
content  to  allow  students  to  tackle  multicausal  prob¬ 
lems  embedded  within  a  complex  environment.  A  river 
runs  through  the  city,  and  the  landscape  has  different 
terrains  that  influence  water  runoff.  The  students  also 
inhabit  the  city  as  avatars  and  interact  with  other  com¬ 
puter-based  characters  in  the  city.  Students  acquire 
clues  by  interacting  with  the  computer-based  agents 
and  by  overhearing  residents’  conversations  with  one 
another  where  they  disclose  indirect  clues  about  what 
is  going  on  in  River  City. 

River  City  generates  a  personal  history  of  each 
student  in  the  form  of  extensive  log  files.  The  logs 
show  exactly  where  each  student  went,  with  whom 
they  communicated,  what  they  said,  what  artifacts  they 
activated,  and  how  long  each  of  these  activities  took. 
According  to  Dede  (2006,  personal  communication), 
the  challenge  for  researchers  is  to  relate  the  rich  log 


file  data  with  the  more  familiar  quantitative  perfor¬ 
mance  measures  (test  scores  on  content  knowledge  and 
the  quality  of  letters  to  the  mayor)  and  qualitative 
measures  (in-class  observations  interviews).  Dede  con¬ 
ceded  in  the  2006  interview  that  the  sheer  volume  of 
data  in  the  log  files  made  it  very  challenging  to  make 
sense  of  them: 

It  is  not  like  you  can  form  a  query  that  can  be  answered 
by  an  automated  process.  It  depends  on  the  type  of  ques¬ 
tions  we  are  asking.  The  log  files  cannot  explain  why 
something  happened.  Log  files  need  human  inspection 
or  interviews  to  be  fully  interpretable.  Currently,  we  are 
in  the  process  of  exploring  the  range  of  questions  the  log 
files  can  answer. 

Parts  of  “It”  Look  Strangely  Familiar 

We  believe  that  a  large  body  of  literature  from  earlier 
CAI  studies  (noted  earlier)  with  a  rich  library  of  find¬ 
ings  about  learners  and  their  behaviors  has  not  been 
adequately  exploited  by  designers  of  TELEs.  Some  of 
the  problems  encountered  by  TELE  developers  (e.g., 
persistent  underuse  of  program  scaffolds)  might  have 
been  anticipated;  for  example,  the  work  by  Tennyson 
and  his  associates  on  program  advisement  in  a  CAI 
program  seems  to  have  implications  for  the  design  of 
effective  scaffolds,  but  this  work  is  not  frequently  cited 
in  the  scaffolding  literature  (Tennyson,  1980;  Tenny¬ 
son  and  Buttrey,  1980;  Tennyson  and  Rothen,  1979). 
Also,  the  work  by  Carrier  and  associates  on  option 
usage  patterns  would  seem  to  be  helpful  in  designing 
ways  to  sustain  leaner  interest  and  curiosity  (Carrier 
et  al.,  1984,  1986).  There  seems  to  be  an  unnecessary 
schism  between  the  new  guard  (expanded  view  of  it) 
and  the  old  (experimental  purists),  which  perhaps 
results  in  a  tendency  to  focus  on  the  differences  while 
overlooking  valuable  similarities  that  could  create  syn¬ 
ergy  between  both  views. 

Where  Is  “It”  Going? 

As  noted  by  Ross  and  Morrison  earlier  in  this  chapter, 
the  2006  trend  in  research  design  is  toward  an  increase 
in  empirical  studies,  with  fewer  qualitative  investiga¬ 
tions.  They  suggest  that  this  is  due,  in  part,  to  political 
forces  that  establish  research  priorities  and  the  types 
of  research  that  will  be  supported  by  government  grant 
programs.  This  trend  was  also  recognized  by  the  TELE 
designers  we  interviewed.  In  response  to  the  question 
"How  much  attention  did  you  pay  to  the  content  stan¬ 
dards  in  your  design?”  most  answered  that  they  simply 
had  no  choice  but  to  align  content  with  relevant  state 
and  national  standards.  Even  if  they  disagreed  philo¬ 
sophically  with  imposed  standards,  however,  they  did 
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not  believe  this  detracted  from  the  effectiveness  of 
their  programs.  Some  were  quite  forthcoming  that  they 
had  to  play  by  the  rules  if  they  wanted  to  find  support, 
but  here,  too,  they  saw  it  as  not  seriously  compromis¬ 
ing  the  overall  goals  of  the  product.  Most  thought  they 
could  do  both,  but  if  Ross  and  Morrison  are  right  then 
what  are  the  long-term  implications  for  the  develop¬ 
ment  of  such  rich  TELEs? 

The  future  of  efforts  to  define  educational  technol¬ 
ogy  may  go  beyond  hardware  and  software,  or  even 
pedagogy,  to  include  individual  and  group  human- 
computer  interactions,  as  well  as  rich  three-dimen¬ 
sional  avatar-based  virtual  worlds  and  multiverse  envi¬ 
ronments  that  create  a  sociocultural  context  for  think¬ 
ing  and  learning.  We  might  extend  Friedman’s  (2005) 
view  that  "the  world  is  flat”  to  suggest  that  learning 
technology  might  enable  any  learner,  anywhere  in  the 
world,  to  collaborate  and  learn  with  any  other  learner 
who  has  shared  interests  by  meeting  in  virtual  space. 
The  added  advantage  for  educational  researchers  is 
that  server  log  files  can  record  every  action  taken  by 
learners,  including  time  and  location,  so  learners’ 
behaviors  can  be  calibrated  to  when,  where,  and  how 
they  interact  with  each  other  and  reproducible  contex¬ 
tual  factors  within  the  learning  environment  to  produce 
knowledge  and  understanding. 

There  is  also  a  trend  to  extend  our  understanding 
of  technology  and  learning  into  the  realm  of  literacy 
(Leu  et  al.,  2004).  As  technology  becomes  ubiquitous 
in  schools,  instead  of  writing  a  book  report,  students 
may  create  a  multimedia  production,  complete  with 
movies,  photos,  animations,  simulations,  music,  voice 
recordings,  and,  of  course,  prudently  used  text.  Also, 
the  format  for  communication  may  expand  beyond  the 
book  report  format  to  include  presentation  slides,  dig¬ 
ital  movies,  Web  pages,  and  hyperlinked  documents, 
both  individual  (blogs)  and  collaborative  (wikis).  It 
seems  clear  to  many  that  reading  and  writing  in  these 
formats  require  new  literacies  beyond  those  required 
to  craft  and  comprehend  a  book  report;  for  example, 
the  results  of  a  Google  Internet  search  do  not  present 
the  same  challenges  to  comprehension  as  those  of  the 
chapters  of  a  novel  or  a  Shakespeare  play.  Similarly, 
one  would  not  expect  students  to  apply  the  same  orga¬ 
nizational  principles  they  learned  for  writing  a  good 
essay  (opening  sentence,  elaboration,  closing  argu¬ 
ment),  for  example,  when  they  are  composing  a  menu 
bar  for  a  website  .  Future  research  on  educational  tech¬ 
nology  may  involve  its  impact  on  curriculum  and  the 
broader  view  of  what  it  means  to  be  literate,  a  good 
reader  and  writer.  So,  in  addition  to  operationally 
defining  a  particular  technology,  as  it  is  used  in  a 
particular  classroom,  toward  particular  pedagogical 
goals,  researchers  in  the  near  future  may  have  to 


address  the  systemwide  impact  of  the  use  of  a  tech¬ 
nology  and  the  extent  to  which  its  associated  new 
literacies  become  ingrained  into  the  teaching  and 
learning  environment. 

If,  as  we  have  suggested  here,  research  on  the 
effects  of  technology  requires  a  complex  understand¬ 
ing  of  learner-,  technology-,  physical-,  and 
social-environment  interactions,  then  simple  statistical 
analyses  that  work  for  highly  controlled  experimental 
laboratory  environments  may  prove  inadequate.  Such 
an  understanding  requires  huge  datasets  from  log  files 
of  massively  multiplayer  environment  interactions 
through  time,  in  which  case  new  experimental  designs 
and  statistical  methods,  including  data  visualization, 
complex  systems  modeling,  and  simulations,  may 
prove  useful  in  educational  technology  research,  as 
they  have  in  physics,  climatology,  meteorology,  and 
other  fields  dealing  with  applied  and  messy  phenom¬ 
ena  and  large  dynamic  datasets. 

These  assumptions  suggest  that  the  randomized 
designs  of  true  experiments  may  fatally  trade  internal 
validity  for  all  applied  value,  yielding  no  generaliza¬ 
tions,  and  they  may  be  antithetical  to  assumptions  that 
learning  is  highly  contextual,  individual,  social,  and 
interactive.  That  is,  although  the  results  of  a  true  exper¬ 
iment  may  be  true,  they  may  only  be  so  for  the  com¬ 
pletely  artificial  controlled  conditions  of  the  laboratory 
which  eliminate  the  interactions  of  interest  to  ensure 
that  only  one  variable  is  causing  changes.  If  the  more 
situated,  embedded,  and  embodied  views  of  cognition 
are  correct,  there  may  never  be  a  real-world  occasion 
on  which  only  a  single  variable  is  affecting  thinking, 
learning,  perceiving,  and  acting  in  context— learners 
may  never  step  in  the  same  river  twice.  Thus,  scholarly 
methods  for  addressing  the  nature  of  complex  interac¬ 
tions  must  be  developed  and  applied,  including  those 
successfully  used  in  other  complex  fields,  such  as  mod¬ 
eling,  simulating,  and  data  visualization  of  what  is 
occurring  in  context.  Techniques  such  as  these  may  be 
useful  in  interpreting  and  making  sense  of  log  files  in 
the  spirit  of  design  research  to  simultaneously  advance 
theory  and  the  practice  of  instructional  design. 

Log  files,  such  as  those  generated  in  River  City, 
are  a  sequence  of  behavioral  data  stored  in  a  permanent 
file  (Hulshof,  2004).  They  offer  time-stamped  (some¬ 
times  location-stamped)  records  of  human  choices 
made  through  computer  interfaces  with  learning  envi¬ 
ronments.  Figure  54.6  is  an  example  of  a  log  file, 
showing  a  text  summary  of  user  actions.  Figure  54.7 
and  Figure  54.8  are  visualizations  of  log  files  that 
graphically  present  the  log  data  across  time  and  space 
in  a  TELE.  Such  files  result  from  online  Internet  nav¬ 
igation  (stored  at  the  server  side  or  as  cookies  at  the 
client  side),  online  and  console -based  game  play,  the 
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Student:  Young  Mike 

Starting  9/1 8/98 
Time  in:  11:43:59  AM 
Startup  1 1 :43:59  AM 
Instructions  11:44:01  AM 

Facts  3:33:38  PM 
Add  Fact:  CC  at  1  56.6  3:33:51  PM 
Add  Fact:  home  at  1 32.6  3:33:59  PM 
Calculating  3:34:02  PM 
Calc:  156.6  -  132.6  -  24 

ANSWER:  How  far  is  it  from  Cedar  Creek  to  home?  =  24  miles 
Facts  3:38:39  PM 

Add  Fact:  has  12  gallons  3:38:53  PM 
Add  Fact:  burns  5  gal/hr  3:39:08  PM 
Calculating  3:39:10  PM 
Calc:  5  *  3  =  15 

ANSWER:  How  much  fuel  is  needed?  =  15  gals. 

Survey  Data: 

P  Q  F  C  V,  So,  Ma,  Sc,  St,  ga,  Cm  Comp,  R,  H,  Sol 
1 01 , 78,  1 01 2,  1 70,  27,  0,  0,  0,  0,  0,  87,  99,  46,  98,  0, 

Time  out:  3:56:47  PM 

Figure  54.6  Sample  log  file. 


Subject  # 


Figure  54.7  Visualization  of  log  file  data.  (Based  on  Borner,  K.  et  al.,  in  Proceedings  of  the  Sixth  International  Conference  on 
Information  Visualization  (IV’02),  July  10-12,  2002,  London,  pp.  25-31;  http://vw.indiana.edu/cive02/004_borner_VisSpatialTem- 
poral.pdf.) 

use  of  computer-based  modeling  and  writing  tools,  and  panoply  of  variables  available  through  data  visualiza- 

avatar-based  communication  and  interactions.  Log  files  tion  in  the  log  files.  See  particularly  Shavelson  et  al. 

retain  a  record  of  process  and  thus  afford  researchers  (2003)  in  the  special  issue  of  Educational  Researcher 

a  rich  source  for  assessment  and  design  research  deci-  on  the  role  of  design  in  educational  research, 

sion  making.  It  bears  repeating  that  advocates  would  Log  files  are  used  for  market  research  (as  when 

argue  that  in  design-based  research,  unlike  tightly  con-  Amazon.com  selects  other  book  choices  to  present  to 

trolled  experimental  studies,  a  priori  hypotheses  are  users  based  on  their  log  files  of  book-buying  choices) 

not  possible  or  even  particularly  useful.  Warrants  based  and  for  customization  of  advertising  in  many  e-com- 

on  theory  can  be  made  for  both  design  changes  and  merce  settings.  Educational  research  is  just  starting  to 

classroom  activities  in  concert  with  observation  of  a  tap  into  this  potential.  Log  file  data  can  include  both 
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Figure  54.8  Another  visualization  of  log  file  data. 


time  and  spatial  data  associated  with  each  online 
behavior.  As  shown  in  Figure  54.8,  both  files  can  dis¬ 
play  where  information  is  found  or  where  each  type 
of  chat  occurs  within  an  avatar-based  virtual  environ¬ 
ment. 

Parallel  to  these  new  data  analysis  challenges, 
hybrid  methodologies  that  combine  the  causal  ele¬ 
ments  of  true  experiments  with  correlational  and 
observational  techniques  of  qualitative  research  must 
be  sought.  Iterative  cycles  of  design,  analysis,  theory, 
and  redesign  may  yield  scholarly  and  scientific  infor¬ 
mation  that  addresses  causality  while  honoring  the 
complex  interactions  that  constitute  learning. 

Wrapping  “It”  Up 

Educational  research  is  naturally  driven  by  funding 
and  the  political  agenda  that  influences  what  schools 
do  and  the  metrics  to  which  students,  teachers,  schools, 
and  districts  are  held  accountable.  As  a  final  observa¬ 
tion  about  the  nature  and  future  of  educational  tech¬ 
nology  research,  it  is  important  to  point  out  that  poli¬ 
cymakers  set  the  agenda  for  what  is  investigated  and 


what  counts  as  a  convincing  argument  for  what  works 
concerning  the  wise  use  of  technology  in  instruction. 
This  suggests  that,  like  all  of  science,  we  must 
acknowledge  that  educational  technology  research  is 
a  human  endeavor  and,  as  such,  draws  upon  an  ever- 
emerging  social  interaction  that  defines  how  technol¬ 
ogy  should  work  and  what  is  accepted  as  evidence  of 
its  working.  To  the  extent  that  this  is  true,  we  must 
accept  that  if  we  find  that  technology  is  working  today 
that  it  might  not  be  working  tomorrow,  and  vice  versa, 
as  new  expectations  for  education  and  new  testing 
criteria  are  adopted.  For  our  purposes,  we  just  want  to 
go  on  record  as  stating  that,  to  be  true  to  contemporary 
learning  theory,  our  description  of  educational  tech¬ 
nology  research  must  itself  be  seen  as  situated  in  the 
context  of  today’s  educational  practices  and  learning 
environments  that  are  currently  hot  among  researchers 
and  educators. 

If  this  section  had  been  written  by  researchers  in 
the  teacher  education  community,  or  the  learning  sci¬ 
ences,  or  the  corporate  or  military  training  community, 
the  emphasis  would  look  and  feel  different.  We  believe, 
however,  that  the  central  arguments  proposed 
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here— namely,  the  proposition  that  all  research  in  tech¬ 
nology  can  be  boiled  down  to  the  central  question  of 
Does  it  work?  —  still  apply.  The  tools  have  gotten  more 
powerful,  the  environments  more  complex  and  realistic, 
the  data  collection  more  multivariate  and  dynamic,  but 
researchers  and  stakeholders  still  want  to  know  if  the 
intervention  is  having  its  intended  effect,  any  ill  effects, 
or  any  effect  at  all  for  that  matter,  with  the  ultimate 
purpose  of  safe,  effective,  and  efficient  learning. 

Summary 

We  discussed  research  on  technology  presented  in  the 
context  of  the  question  Does  it  work!  Major  concepts 
and  points  addressed  include  the  following: 

•  Technology  has  been  used  in  educational 
research  as  the  variable  under  study  (e.g., 
media  comparison  studies),  as  the  conveyor 
of  instructional  treatments,  and  to  support 
complex  learning  environments. 

•  The  research  community  valued  experimen¬ 
tal  research  with  high  internal  validity  that 
could  identify  causal  relationships,  which 
gave  rise  to  a  necessarily  narrow  conception 
of  what  was  studied  (it). 

•  As  human  learning  was  increasingly  viewed 
as  embedded  and  embodied,  TELEs 
emerged  to  create  complex  and  realistic 
environments. 

•  Research  on  TELEs  is  largely  design  based, 
with  an  emphasis  on  external  validity,  and  it 
conceives  of  it  as  a  “cloud  of  correlated  vari¬ 
ables,”  with  the  unit  of  analysis  being  the 
learner-environment  interaction. 

•  Some  of  the  findings  from  design-based 
research  might  have  been  predicted  by  a  bet¬ 
ter  understanding  of  previous  research  on 
technology. 

•  Vast  amounts  of  user  data  recorded  in  the 
log  hies  of  TELEs  hold  great  potential  to 
advance  the  understanding  of  how  students 
learn  in  open  environments,  but  effective 
analysis  techniques  are  complex,  multidi¬ 
mensional,  and  multivariate. 

We  addressed  two  key  issues:  (1)  what  variables  con¬ 
stitute  the  wise  application  of  technology  to  instruc¬ 
tion,  and  (2)  how  do  we  operationally  define  the  depen¬ 
dent  measures  used  to  determine  how  technology- 
pedagogy-environment  interactions  affect  knowledge, 
attitudes,  and  behaviors.  The  variables  we  select  will 
emerge  from  the  dialectic  among  theories  such  as  sit¬ 
uated  cognition,  environments  that  can  generate  large 


log  hies  concerning  the  dynamics  of  student-computer 
interaction,  and  learners  who  perceive  and  act  accord¬ 
ing  to  their  own  goals  and  constructed  understandings. 
In  the  introduction,  we  cited  the  question  Does  it 
work?  as  the  organizer  for  our  discussion.  As  we  look 
forward,  we  see  promising  trends  that  can  advance  our 
understanding  of  what  it  is  and  help  us  reliably  mea¬ 
sure  more  fully  how  it  works. 

RESEARCH  ON  MODELS  FOR 
INSTRUCTIONAL  DESIGN 

Jan  van  den  Akker  and  Wilmad  Kuiper 

This  chapter  section  deals  with  research  on  models  for 
instructional  design.  The  literature  on  instructional 
design  is  abundantly  supplied  with  models.  Many  of 
these  models  claim  to  be  unique  and  deserving  of 
attention,  but  most  have  the  limitation  of  being  too 
abstract  to  inspire  and  support  designers  in  their  pro¬ 
fessional  practice.  With  the  ultimate  aim  to  try  to 
strengthen  the  weak  link  between  theory  and  practice 
in  instructional  design  models,  we  raise  two  main 
questions.  First,  which  trends  are  visible  in  research 
on  model  development,  use,  and  validation  so  far? 
Second,  how  can  research  on  instructional  design  be 
redirected  to  foster  blending  of  theory  and  practice? 
In  answering  the  second  question,  we  take  the  position 
that  instructional  design  can  become  more  productive 
if  its  perspective  is  broadened  to  educational  design. 
This  conception  includes  incorporating  multiple  levels 
and  components  of  learning  and  instruction  (for  exam¬ 
ple,  not  only  the  individual  student  or  a  group  of  stu¬ 
dents  but  also  the  teacher  who  operates  within  a  cur¬ 
ricular  framework)  as  well  as  embedding  design  work 
in  systemic  educational  development  approaches.  A 
design  research  approach  is  therefore  implied. 

In  using  the  term  research  on  models  we  build  on 
Richey  and  Klein  (in  this  chapter) ,  who  divide  studies 
of  instructional  design  and  development  into  two  gen¬ 
eral  types  of  design  and  developmental  research:  prod¬ 
uct  and  tool  research  and  model  research.  The  two 
types  have  been  previously  called,  respectively.  Type 
1  and  Type  2  (Richey  et  al.,  2004)  or  formative 
research  and  reconstructive  studies  (van  den  Akker, 
1999).  The  first  type  of  research  pertains  primarily  to 
research-based  design  and  development  of  products 
and  tools.  A  typical  example  of  product  and  tool 
research  is  McKenney’s  (2002)  report  on  CASCADE- 
SEA,  a  computer-based  tool  to  support  curriculum 
materials  development  in  the  context  of  secondary  sci¬ 
ence  and  mathematics  education  in  southern  Africa. 
The  second  type  of  research— as  stated,  the  focus  of 
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this  chapter  section— deals  with  the  development,  val¬ 
idation,  and  use  of  design  and  development  models, 
such  as  Jones  and  Richey’s  (2000)  in-depth  examina¬ 
tion  of  the  use  of  rapid  prototyping  methods  in  two 
instructional  design  projects  in  natural  work  settings. 
These  studies  focus  on  the  models  and  processes  them¬ 
selves,  rather  than  their  demonstration.  Although  it  is 
possible  to  conduct  model  research  in  conjunction  with 
the  development  of  a  product  or  program,  most  model 
studies  concentrate  on  previously  developed  instruc¬ 
tion  and  consequently  are  not  project  specific.  Model 
research  may  address  the  validity  or  effectiveness  of 
an  existing  or  newly  constructed  development  model, 
process,  or  technique.  In  addition,  these  studies  often 
seek  to  identify  and  describe  the  conditions  that  facil¬ 
itate  successful  design  and  development  (see  Richey 
and  Klein  in  this  chapter). 

Research  on  Instructional  Design  Models: 
Trends  and  Reflections 

Instructional  Design  Models  as  Conceptual, 
Communication,  and  Procedural  Tools 

Over  the  years,  there  have  been  many  attempts,  includ¬ 
ing  efforts  by  the  Association  for  Educational  Commu¬ 
nications  and  Technology  (AECT),  to  label  and  define 
the  field  of  instructional  design  or  technology.  A  rep¬ 
resentative  AECT  definition  is  the  one  published  in 
Instructional  Technology :  The  Definitions  and  Domains 
of  the  Field  (Seels  and  Richey,  1994).  In  that  volume, 
instructional  technology  is  concisely  defined  as  the 
"theory  and  practice  of  design,  development,  utiliza¬ 
tion,  management,  and  evaluation  of  processes  and 
resources  for  learning”  (p.  1).  In  this  chapter  section, 
we  take  this  definition  and  the  term  instructional  design 
as  a  starting  point.  Later  on,  the  instructional  design 
perspective  will  be  broadened  to  systematic  educa¬ 
tional  engineering.  Rather  than  adopting  the  terms  pro¬ 
cesses  and  resources  from  the  Seels  and  Richey  (1994) 
definition,  we  prefer  to  use  the  more  generic  term  inter¬ 
ventions,  which  serves  as  a  common  denominator  for 
products,  programs,  materials,  procedures,  scenarios, 
processes,  and  the  like  (van  den  Akker,  1999). 

The  literature  is  well  supplied  with  definitions 
(Reiser  and  Dempsey,  2002)  and  with  alternative 
instructional  design  models.  These  models  serve  as 
conceptual,  communication,  and  procedural  tools  for 
analyzing,  designing,  creating,  and  evaluating  broad 
educational  environments  as  well  as  narrow  training 
applications  (Gustafson  and  Branch,  2002b).  Most 
models  assume  that  the  use  of  systematic  procedures 
can  make  interventions  more  effective.  Design  and 
development  may  be  conceived  as  a  single  linear  pro¬ 


cess  or— more  realistically  and  preferably— as  a  set  of 
concurrent  iterative  or  recursive  engineering  proce¬ 
dures  with  integrated  research  activities  to  feed  the 
process  forward  and  backward.  Most  model  descrip¬ 
tions  include  the  core  elements  of  analysis,  design, 
development,  implementation,  and  evaluation 
(ADDIE) .  The  five  core  elements  each  inform  the  other 
as  development  takes  place,  and  revision  continues  at 
least  until  the  instruction  is  implemented  (Gustafson 
and  Branch,  2002a,b).  Figure  54.9  depicts  the  relation¬ 
ship  among  these  so-called  ADDIE  elements. 

Characterization  of  Research 
on  Instructional  Design  Models 

In  reviewing  model  research  studies,  Richey  and  col¬ 
leagues  (2004;  see  also  Richey  and  Klein  in  this  chap¬ 
ter)  analyzed  58  examples  published  between  1984  and 
2002  in  terms  of,  among  other  things,  range  of  research 
methodologies  used  and  nature  of  research  conclusions. 
From  their  analysis,  it  appears  that  quite  a  diversity  of 
research  methods  is  employed  in  model  research.  As 
regards  model  development,  common  research  methods 
are  literature  reviews,  surveys  (based  on  interviews), 
case  studies  (employed,  but  far  less  prominently  than 
in  product  and  tool  research),  Delphi  techniques  (aimed 
at  reaching  consensus  of  experts’  opinions),  and  think- 
aloud  protocols.  For  research  on  model  use,  surveys 
(based  on  interviews),  case  studies  (interviews  and  field 
observations),  document  analyses,  and  think-aloud 
methods  are  employed.  Research  methods  commonly 
applied  in  the  context  of  model  validation  are  experi¬ 
ments  and  quasi-experiments  (critical  to  the  verification 
and  evaluation  of  a  particular  design  and  development 
technique),  interviews,  and  expert  reviews.  Model 
research  studies  are  unique  in  that  they  are  ultimately 
directed  toward  general  principles,  which  are  applica¬ 
ble  in  a  wide  range  of  design  and  development 
projects.  The  issues  addressed  in  the  conclusions  are 


740 


Research  Designs 


the  following  (Richey  et  al.,  2004):  evidence  of  validity 
or  effectiveness  of  a  particular  technique  or  model, 
conditions  and  procedures  that  facilitate  the  successful 
use  of  a  particular  technique  or  model,  explanations 
of  the  successes  or  failures  encountered  in  using  a 
particular  technique  or  model,  a  synthesis  of  events  or 
opinions  related  to  the  use  of  a  particular  technique  or 
model,  and  a  new  or  enhanced  model. 

We  next  illustrate  some  typical  model  research 
studies  in  more  detail  and  begin  with  four  examples 
of  examples  of  research  on  model  development .  From 
the  beginning  of  the  1970s  to  the  present,  several  lit¬ 
erature  reviews  of  instructional  design  models  have 
been  published,  including  the  review  by  Andrews  and 
Goodson  (1980)  of  40  models  in  the  Journal  for 
Instructional  Development.  Of  more  recent  date  are 
the  reviews  by  Gustafson  (1981,  1991)  and  Gustafson 
and  Branch  (1997,  2002a)  of  instructional  develop¬ 
ment  models.  To  help  clarify  the  underlying  assump¬ 
tions  of  each  model  and  identify  the  conditions  under 
which  each  might  be  appropriately  applied,  Gustafson 
and  Branch  (2002a)  classified  up  to  15  ADDIE  models 
according  to  the  primary  type  of  instruction  they  were 
designed  to  produce.  Their  taxonomy  of  instructional 
development  models  has  three  categories,  indicating 
whether  a  given  model  is  best  applied  for  designing 
and  developing:  (1)  instruction  to  be  delivered  in  a 
classroom  by  a  teacher,  with  little  time  and  resources 
for  the  teacher  to  devote  to  the  comprehensive  devel¬ 
opment  of  instructional  materials  (four  models);  (2) 
instructional  products  for  implementation  by  users  oth¬ 
ers  than  the  designers,  such  as  computer-based  mod¬ 
ules  designed  for  wide  distribution,  with  considerable 
emphasis  on  tryout  and  revision  (five  models);  and  (3) 
large-scale  instructional  systems  directed  at  an  organi¬ 
zation’s  problems  or  goals,  such  as  a  distance  learning 
course  or  a  curriculum  encompassing  an  entire  degree 
program  (six  models).  Gustafson  and  Branch  catego¬ 
rized  the  ADDIE  models  according  to  the  following 
nine  characteristics:  (1)  typical  output  in  terms  of 
amount  of  instruction  prepared,  (2)  resources  commit¬ 
ted  to  the  development  effort,  (3)  whether  it  is  a  team 
or  individual  effort,  (4)  expected  instructional  design 
skill  and  experience  of  the  individual  or  team,  (5) 
whether  instructional  materials  will  be  selected  from 
existing  courses  or  represent  original  design  and  devel¬ 
opment,  (6)  amount  of  preliminary  analysis  conducted, 
(7)  anticipated  technological  complexity  of  the  devel¬ 
opment  and  delivery  environments,  (8)  amount  of  try¬ 
out  and  revision  conducted,  and  (9)  amount  of  dissem¬ 
ination  and  follow-up  occurring  after  development.  For 
a  characterization  of  the  models  selected  as  well  as 
their  underlying  assumptions,  we  refer  to  Gustafson 
and  Branch  (2002b). 


Other  authors  have  created  different  classification 
schemes  for  instructional  design  models  and  pro¬ 
cesses.  Richey  (2005)  distinguished  two  major  types 
of  instructional  design  models.  One  type,  labeled  con¬ 
ceptual  models ,  identifies  variables  that  impact  the 
design  process  and  shows  their  interrelationships;  the 
second  type, procedural  models,  represents  the  recom¬ 
mended  steps  to  follow  in  a  design  process.  Most 
instructional  design  models  can  be  visualized  in  dia¬ 
grams  and  are  procedural  in  nature.  Most  procedural 
models  pertain  to  comprehensive  design  projects.  Oth¬ 
ers  address  more  specific  aspects  of  the  design,  devel¬ 
opment,  and  evaluation  processes. 

Another  example  of  model  development  research 
is  the  already-mentioned  study  by  Jones  and  Richey 
(2000),  a  study  that  also  has  been  referred  to  and 
summarized  by  Richey  and  colleagues  (2004)  and  by 
Richey  and  Klein  (this  chapter).  Jones  and  Richey 
conducted  an  in-depth  qualitative  examination  of  the 
use  of  rapid  prototyping  methods  in  two  instructional 
design  projects  in  natural  settings.  Data  were  collected 
from  structured  personal  interviews,  activity  logs,  and 
a  review  of  extant  data.  Participants  included  two  expe¬ 
rienced  instructional  designers  who  employed  rapid 
prototyping  in  the  design  and  development  of  two 
projects  in  the  automotive  industry  and  in  the  health¬ 
care  industry.  In  addition,  one  client  was  interviewed 
to  obtain  perceptions  of  product  quality  and  usability, 
cycle-time  reduction,  and  customer  satisfaction.  After 
discussing  how  rapid  prototyping  was  successfully 
used  in  the  context  of  the  study,  the  authors  provided 
general  conclusions  and  suggested  a  revised  model  of 
instructional  design  that  includes  rapid  prototyping. 

An  example  of  research  on  model  use  is  an  inves¬ 
tigation  by  Rowland  (1992)  of  what  expert  instruc¬ 
tional  designers  actually  do.  Four  expert  and  four  nov¬ 
ice  designers  were  asked  to  think  aloud  as  they  solved 
a  design  problem.  The  resulting  protocols  were  ana¬ 
lyzed  and  compared.  Important  differences  were  iden¬ 
tified  between  experts  and  novices,  including  the  fol¬ 
lowing: 

•  Experts  interpreted  the  problem  they  were 
faced  with  as  poorly  defined  by  the  given 
information.  They  challenged  givens  and 
added  much  information  through  inference. 
Novices  interpreted  the  problem  as  well 
defined  by  the  given  information  and  did 
little  elaboration. 

•  Experts  thought  of  possible  solutions  imme¬ 
diately  but  delayed  commitment  pending  a 
deeper  understanding  of  the  problem.  Nov¬ 
ices  moved  quickly  to  solution  generation 
after  a  brief  examination  of  the  materials. 
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•  Experts  represented  the  problem  as  a  deep 
and  rich  causal  network  of  many  links.  Nov¬ 
ices  split  the  problem  (and  therefore  the 
solution)  into  parts  based  on  surface  differ¬ 
ences. 

•  Experts  considered  many  global  (system- 
wide)  and  local  (immediate)  factors  in  mak¬ 
ing  decisions.  Novices  considered  few 
factors,  only  local  factors,  and  only  one  fac¬ 
tor  at  a  time. 

Rowland  also  discussed  the  implications  of  these  differ¬ 
ences  for  assisting  and  training  instructional  designers. 

Another  example  of  research  on  model  use  is  Viss- 
cher-Voerman’s  (1999)  reconstructive  study  of  how 
instructional  designers  actually  conduct  projects  in 
various  training  and  education  contexts  (design  of  text¬ 
books,  curriculum  development,  multimedia,  human 
resource  development  programs,  and  distance  educa¬ 
tion).  This  researcher  collected  case  study  data  from 
24  professional  designers  by  means  of  interviews  and 
document  analysis.  Results  showed  that  scientific 
design  models  differed  from  design  approaches 
applied  in  practice  by  experienced  designers  in  the 
sense  that  experienced  educational  designers  worked 
much  more  heterogeneously  and  diversely  than  models 
suggest.  Specific  practices  of  experienced  educational 
designers  included:  (1)  conducting  only  a  restricted 
problem  and  context  analysis  at  the  start,  generally 
resulting  in  a  further  specification  of  a  potential  solu¬ 
tion  rather  than  a  specification  of  the  problem;  (2) 
minimally  generating  and  weighing  alternative  solu¬ 
tions,  while  instead  exploring  alternatives  within  a 
given  solution;  (3)  not  distinguishing  between  design 
and  development;  and  (4)  generally  interweaving  for¬ 
mative  evaluation  activities  with  design  activities 
rather  than  conducting  formative  evaluation  activities 
in  a  distinct  phase.  Based  on  the  extensive  research 
findings,  Visscher-Voerman  created  a  classification 
framework  consisting  of  four  design  paradigms  (Viss¬ 
cher-Voerman  and  Gustafson,  2004;  Visscher-Voerman 
et  al.,  1999):  (1)  instrumental  (planning  by  objectives), 
(2)  communicative  (communication  to  reach  consen¬ 
sus),  (3)  pragmatic  (interactive  and  repeated  tryout  and 
revision),  and  (4)  artistic  (creation  of  products  based 
on  educational  connoisseurship).  Rather  than  focusing 
on  the  context  of  the  development  and  use  of  instruc¬ 
tion  as  done  by  Gustafson  and  Branch,  Visscher-Voer¬ 
man  characterized  the  underlying  philosophy  and  val¬ 
ues  of  each  design  paradigm. 

A  third  example  of  research  on  model  use  is  a  study 
conducted  by  Kirschner  et  al.  (2002).  First,  they 
reviewed  the  following  eight  studies  describing  actual 
instructional  design  practices:  Kerr  (1983),  Le  Maistre 


(1998),  Perez  and  Emery  (1995),  Pieters  and  Bergman 
(1995),  Rowland  (1992),  Visscher-Voerman  (1999), 
Wedman  and  Tessmer  (1993),  and  Winer  and  Vasquez- 
Abad  (1995).  From  their  review  they  concluded  that 
instructional  designers,  among  other  things,  should 
thoroughly  explore  and  interpret  the  problem;  consider 
a  wide  range  of  possible  solutions  and  a  wide  range  of 
factors,  combining  them  and  using  context  knowledge; 
take  more  time  for  prototyping  and  evaluation;  and  use 
a  highly  interactive  and  collaborative  design  approach. 
Second,  they  carried  out  two  experiments  with  expert 
educational  designers  from  a  university  setting  and  a 
business  setting  to  find  out  what  the  designers  actually 
did  and  which  priorities  they  employed  when  designing 
competence-based  learning  environments.  From  the 
two  experiments  it  appeared  that  designers  in  a  univer¬ 
sity  context  and  in  a  business  context  agreed  almost 
completely  on  what  design  principles  are  important,  the 
most  important  being  that  one  should  start  a  design 
enterprise  from  the  needs  of  the  learners,  instead  of  the 
content  structure  of  the  learning  domain.  The  main 
difference  between  the  two  groups  is  that  university 
designers  find  it  extremely  important  to  consider  alter¬ 
native  solutions  during  the  entire  design  process,  some¬ 
thing  that  is  considerably  less  important  by  business 
designers.  University  designers  also  tend  to  focus  on 
the  project  plan  and  the  desired  characteristics  of  the 
instructional  blueprint  whereas  business  designers  were 
much  more  client  oriented  and  stressed  the  importance 
of  gaining  client  buy-in  early  in  the  process. 

An  example  of  research  on  model  validation  is 
Tracey’s  (2002;  also  see  Richey  and  Klein  in  this  chap¬ 
ter)  comparison  between  the  use  of  the  Dick  and  Carey 
(1996)  model  and  an  instructional  design  model 
enhanced  with  a  consideration  of  multiple  intelli¬ 
gences  (MI).  Tracey  established  two  design  teams, 
each  with  two  novice  designers.  One  team  worked  with 
the  Dick  and  Carey  model,  and  the  second  used  the 
MI  model.  Both  teams  were  instructed  to  design  a  2- 
hour,  instructor-led,  classroom-based,  team-building 
course  for  a  nonprofit  organization.  The  resulting  pro¬ 
grams  were  implemented  and  their  impact  compared 
in  terms  of  learner  knowledge  and  attitudes.  Instru¬ 
mentation  used  during  the  study  included  designer  logs 
giving  reactions  to  their  tasks  and  the  models  used, 
learner  knowledge  post-tests,  and  surveys  addressing 
participant  reactions  to  the  instruction. 

Reflections  on  Research  on 
Instructional  Design  Models 

Many  of  the  instructional  design  models  described  in 
the  literature  claim  to  be  unique  and  deserving  of 
attention;  however,  in  almost  all  instances,  the 
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authors  have  assumed  that  their  models  are  worth¬ 
while  or  rely  on  user  testimonials  as  proof  of  their 
effectiveness  but  present  no  evidence  to  substantiate 
their  positions .  There  is  a  disturbingly  small  volume 
of  literature  describing  any  testing  of  the  models. 
Many  have  never  been  systematically  applied,  not  to 
mention  rigorously  evaluated  and  empirically  vali¬ 
dated  (Gustafson  and  Branch,  2002a;  Richey,  2005). 
That  there  is  a  need  for  doing  so  can,  among  other 
things,  be  derived  from  Visscher-Voerman’s  (1999) 
retrospective  study  (see  also  Visscher-Voerman  and 
Gustafson,  2004)  from  which  it  appeared  that  design 
approaches  applied  by  experienced  educational 
designers  differ  from  design  approaches  reflected  in 
scientific  design  models.  Also,  Kirschner  and  col¬ 
leagues  (2002)  found  a  clear  gap  between  instruc¬ 
tional  design  processes  as  described  in  prescriptive 
instructional  design  models  and  the  design  process 
as  it  is  performed  in  the  real  world. 

As  already  stated,  research  also  shows  that 
design  models  are  interpreted  differently  by  expert 
designers  and  novice  designers  (Perez  and  Emery, 
1995;  Rowland,  1992, 1993).  Although  the  key  tasks 
are  still  completed  by  experts,  experts  tend  to  treat 
design  problems  as  ill  defined.  They  consider  a  wide 
variety  of  situational  factors  in  combination,  as  well 
as  both  instructional  and  non-instructional  solutions 
but  delay  making  design  decisions  as  long  as  possi¬ 
ble  (Richey,  2005).  Experts  interpret  design  prob¬ 
lems,  whereas  novices  identify  them  (Perez  and 
Emery,  1995).  The  developmental  study  of  rapid  pro¬ 
totyping  in  action  (Jones  and  Richey,  2000)  showed 
that  some  approaches  to  design,  especially  interac¬ 
tive,  cyclic,  and  spiral  procedures  such  as  rapid  or 
evolutionary  prototyping,  also  appeal  to  a  greater 
ability  of  judgment  and  larger  repertoire  of  design 
skills.  Expert  designers  basically  use  a  general 
instructional  design  model,  but  the  design  process 
tends  to  be  more  iterative,  and  the  sequencing  of 
design  steps  varies  to  meet  the  demands  of  the  indi¬ 
vidual  design  project.  Design  tasks  are  often  per¬ 
formed  concurrently,  a  finding  that  has  also  been 
confirmed  by  Visscher-Voerman. 

Richey  (2005)  noted  the  paucity  of  model  valida¬ 
tion  studies  and  attributed  such  to  time  constraints  and 
ill-defined  model  validation  procedures  rather  than  to 
a  lack  of  the  fundamental  need  for  validation.  She 
viewed  instructional  design  model  validation  as  a  care¬ 
fully  planned  process  of  collecting  and  analyzing 
empirical  data  to  demonstrate  the  effectiveness  of  a 
model’s  use  in  the  workplace  (external  validation)  or 
to  provide  support  for  the  various  components  of  the 
model  itself  (internal  validation).  Richey  distinguished 
the  following  five  approaches  to  validation: 


•  Internal  validation  procedures  (emphasizing 
formative  evaluation  of  the  model): 

Expert  review,  a  process  whereby  instruction¬ 
al  design  experts  (often  representing  both 
design  practitioners  and  design  theorists) 
critique  a  given  model  in  terms  of  its  com¬ 
ponents,  overall  structure,  and  future  use 

Usability  documentation,  the  systematic 
documentation  of  designers  using  a  par¬ 
ticular  model 

Component  investigation ,  the  identification 
or  confirmation  through  research  of  steps 
(procedural  models)  and  factors  (concep¬ 
tual  models)  critical  to  the  instructional 
design  process 

•  External  validation  procedures  (emphasizing 
summative  or  confirmative  evaluation  of  the 
model): 

Field  evaluation,  involving  the  actual  use  of 
the  model  to  produce  instruction 
Controlled  testing,  establishing  experiments 
that  isolate  the  effects  of  the  given  in¬ 
structional  design  model  as  compared  to 
the  use  of  another  model  or  approach 

Major  Challenges  to  Research 
on  Instructional  Design  Models 

It  is  difficult  to  find  fault  with  Richey’s  underlying 
message  that  “as  a  field  we  should  be  validating  our 
many  instructional  design  models  as  well  as  develop¬ 
ing  them,  and  that  validation  should  become  a  natural 
part  of  the  model  development  process”  (2005,  p.  183). 
Nevertheless,  some  major  challenges  arise  when 
addressing  this  issue. 

As  can  be  inferred  from  the  retrospective  study 
conducted  by  Visscher-Voerman  (1999),  there  is  a 
strong  need  for  a  better  correspondence  between,  on 
the  one  hand,  the  rather  abstract  instructional  design 
models  and,  on  the  other  hand,  the  real  and  messy 
world  of  professional  design  and  development  prac¬ 
tices.  This  issue  has  two  sides.  Acknowledging  that 
most  of  the  deviations  of  design  practice  from  design 
theory  (as  reflected  from  the  Visscher-Voerman  study) 
seem  to  be  realistic,  the  models  need  to  be  improved 
to  better  match  actual  processes.  At  the  same  time, 
there  seems  to  be  ample  room  for  improvement  of 
existing,  often  ineffective  practices. 

How  can  we  conceptualize  models  that  fit  better 
to  the  real-world  challenges  of  practitioners,  while  also 
stimulating  and  supporting  approaches  that  seem  more 
valid  from  a  theoretical  perspective?  In  other  words, 
how  can  we  strengthen  the  links  between  theory  and 
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TABLE  54.1 

Curriculum  Components 


Rationale: 

Aims  and  objectives: 
Content: 

Learning  activities: 
Teacher  role: 

Materials  and  resources: 
Grouping: 

Location: 

Time: 

Assessment: 


Why  are  they  learning? 

Toward  which  goals  are  they  learning? 

What  are  they  learning? 

How  are  they  learning? 

How  is  the  teacher  facilitating  learning? 

With  what  are  they  learning? 

With  whom  are  they  learning? 

Where  are  they  learning? 

When  and  for  how  long  are  they  learning? 

How  should  their  learning  progress  be  assessed? 


practice  in  instructional  design  models,  making  them 
more  relevant  for  the  improvement  of  learning?  For 
that  purpose,  in  the  authors’  opinion,  more  productive 
instructional  design  models  should: 

•  Be  highly  relevant  to  realistic  circumstances . 

•  Take  into  account  the  wider  environment  of 
design  and  development  challenges,  with 
more  deliberate  anticipation  on  implementa¬ 
tion  processes  in  user  contexts. 

•  Be  comprehensive,  considering  a  more  sys¬ 
temic  view  on  educational  improvement. 

•  Address  learning  processes  and  outcomes 
from  multiple  perspectives  and  include  a 
wide  range  of  influential  variables. 

We  elaborate  on  these  issues  in  the  next  section. 

Broadening  the  Instructional 
Design  Perspective 

Addressing  Multiple  Levels 
of  Learning  and  Instruction 

The  improvement  of  learning  should,  of  course,  always 
be  the  ultimate  aim  of  instructional  design,  but  suc¬ 
cessful  approaches  require  a  broadening  to  educational 
design.  An  essential  insight  is  that  individual  learning 
occurs  in  educational  contexts  that  influence  the  aims, 
processes,  and  outcomes  of  learning  in  distinctive 
ways.  In  most  instructional  situations,  teachers  (or  edu¬ 
cators  or  trainers)  still  have  a  dominant  role  in  arrang¬ 
ing  the  learning  trajectories  and  environments  of  their 
students.  Also,  teachers  usually  do  not  operate  in  iso¬ 
lation  or  complete  autonomy,  but  within  curriculum 
frameworks.  Those  curricula  (in  essence,  plans  for 
learning)  can  be  found  at  various  levels,  ranging  from 
the  macro  level  (system,  society,  nation,  state)  via  the 
me  so  level  of  the  school  or  institution,  to  the  micro 


level  of  the  classroom  or  group,  and  ultimately  to  the 
nano  level  of  the  individual  person  (van  den  Akker, 
2003).  Student  learning  is  usually  influenced  by  cur¬ 
ricular  representations  at  higher  levels;  for  example, 
schools  are  usually  not  neutral  institutions  with  regard 
to  what  and  how  students  learn  within  their  environ¬ 
ment.  Despite  this,  the  repertoire  of  teachers  within 
their  respective  curricular  framework  at  the  micro  level 
(textbooks,  instructional  materials)  is  most  influential 
on  the  daily  learning  activities  of  students.  Thus,  the 
design  of  individual  student  learning  is  almost  always 
embedded  in  educational  planning  and  implementation 
at  other  levels,  and  that  perspective  deserves  to  be 
incorporated  in  design  approaches  (McKenney  et  al., 
2006). 

Incorporating  Multiple  Components 
of  Learning  and  Instruction 

The  relevance  and  meaningfulness  of  instructional 
design  would  also  be  improved  if  the  scope  of  design 
would  be  broadened  and  elaborated.  Educational 
design  is  not  only  about  learning  activities.  Many  other 
related  questions  about  learning  design  are  usually  at 
stake.  From  a  broader  curriculum  perspective,  it  makes 
sense  to  pay  attention  to  a  coherent  set  of  ten  compo¬ 
nents  that  address  ten  specific  questions  about  the  plan¬ 
ning  of  student  learning  (Table  54.1)  (McKenney  et 
al.,  2006;  van  den  Akker,  2003). 

Our  preferential  visualization  of  the  ten  compo¬ 
nents  is  to  arrange  them  as  a  spider  web  (Figure  54.10), 
which  not  only  illustrates  their  many  interconnections 
but  also  underlines  the  vulnerability  of  the  curriculum 
as  a  whole,  especially  during  implementation.  At  the 
hub  of  the  model  is  the  rationale,  which  connects  all 
the  other  components:  aims  and  objectives,  content, 
learning  activities,  teacher  role,  materials  and 
resources,  grouping,  location,  time,  and  assessment. 
The  spider  web  metaphor  emphasizes  that,  within  one 
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Figure  54.10  Curricular  spider  web. 


curriculum,  component  accents  may  vary  over  time  but 
any  dramatic  shift  in  balance  will  pull  the  entirety  out 
of  alignment.  Though  the  system  may  endure  the  stress 
for  a  while,  prolonged  imbalance  will  cause  it  to  break. 
Efforts  directed  toward  curriculum  reform,  (re)design, 
development,  and  implementation  must  therefore 
devote  attention  to  the  balance  and  linkages  among  the 
ten  components.  A  striking  example  is  the  trend  toward 
integration  of  ICT  in  the  curriculum,  usually  with  ini¬ 
tial  attention  to  changes  in  materials  and  resources. 
Many  implementation  studies  have  revealed  the  need 
for  a  more  comprehensive  approach  and  systematic 
attention  to  the  other  components  before  one  can 
expect  robust  changes.  The  spider  web  also  illustrates 
a  familiar  expression:  Every  chain  is  as  strong  as  its 
weakest  link.  That  seems  another  very  appropriate 
metaphor  for  a  curriculum,  pointing  to  the  complexity 
of  efforts  to  improve  the  curriculum  in  a  balanced, 
consistent,  and  sustainable  manner. 

Strengthening  the  Relevance 
and  Scope  of  Design  Approaches 

In  addition  to  addressing  more  levels  and  components 
of  learning,  instructional  design  also  can  become  more 
productive  by  linking  its  efforts  to  other  educational 
development  activities.  If  instructional  design  focuses 
on  exemplary  trajectories  for  learning  that  cover  many 
of  the  aforementioned  components  of  the  spider  web, 
then  it  has  already  strengthened  its  curricular  rele¬ 


vance.  To  increase  its  scope  and  potential  impact  at  a 
larger  scale,  however,  it  is  also  recommended  that  the 
design  work  be  connected  to  other  systemic  activities 
in  the  domains  of  teacher  education,  school  develop¬ 
ment,  and  examinations/assessment  (McKenney  et  al., 
2006).  Such  interconnections  definitely  contribute  to 
system  coherence,  which  appears  to  be  a  vital  condi¬ 
tion  to  sustainability  of  innovations.  As  expressed  by 
Hargreaves  and  Fink  (2006,  p.  1):  “Change  in  educa¬ 
tion  is  easy  to  propose,  hard  to  implement,  and  extraor¬ 
dinarily  difficult  to  sustain.”  Integrating  instructional 
design  in  systemic  (and  usually  long-term)  approaches 
of  educational  development  can  further  the  sustainabil¬ 
ity  of  its  results. 

Implications  for  Research  Approaches 

Design  Research 

Although  the  validation  studies  on  instructional  design 
models,  as  suggested  by  Richey  (2005),  are  certainly 
worthwhile  to  apply,  the  previous  section  has  already 
underlined  various  characteristics  of  (more)  productive 
research  approaches.  They  all  refer  to  broadening  the 
scope  of  instructional  design,  embedding  it  within  inte¬ 
grated  educational  development.  Moreover,  there  is  a 
need  for  more  interactive  and  developmental 
approaches  to  research  on  instructional  design  models. 
Although  other  labels  also  are  used,  such  research 
approaches  are  often  labeled  as  design-based  research 
(Design-Based  Research  Collective,  2003)  or  as  design 
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research  (van  den  Akker  et  al.,  2006a).  The  following 
generic  definition,  which  encompasses  most  variations 
of  educational  design  research,  is  offered  by  Barab  and 
Squire  (2004,  p.  2):  “a  series  of  approaches  with  the 
intent  of  producing  new  theories,  artifacts,  and  prac¬ 
tices  that  account  for  and  potentially  impact  learning 
and  teaching  in  naturalistic  settings.”  According  to  van 
den  Akker  et  al .  (2006a) ,  various  interests  in  conducting 
such  design  research  include:  (1)  increasing  the  rele¬ 
vance  of  research  for  educational  policy  and  practice, 
(2)  developing  empirically  grounded  theories,  and  (3) 
increasing  the  robustness  of  design  practice.  The  edited 
book  Educational  Design  Research  (van  den  Akker  et 
al.,  2006b)  contains  a  series  of  chapters  providing 
examples  of  and  reflections  on  design  research  from  a 
variety  of  perspectives  (e.g.,  learning  design,  technol¬ 
ogy,  curriculum,  philosophy,  engineering).  Some  com¬ 
mon  characteristics  of  design  research  appear  to  be 
(Cobb  et  al.,  2003;  Design-Based  Research  Collective, 
2003;  Kelly,  2003;  Reeves  et  al.,  2005;  van  den  Akker, 
1999;  van  den  Akker  et  al.,  2006a): 

•  Interventionist—  The  research  aims  at  devel¬ 
oping  an  intervention  in  the  real  world. 

•  Iterative— The  research  incorporates  a 
cyclic  approach  of  design,  evaluation,  and 
revision. 

•  Process  oriented— A  black  box  model  of 
input-output  measurement  is  avoided;  the 
focus  is  on  understanding  and  improving 
interventions. 

•  Utility  oriented— The  merit  of  the  design  is 
measured,  in  part, by  its  practicality  for  users 
in  real  contexts. 

•  Theory  oriented— The  design  is  (at  least 
partly)  based  on  theoretical  propositions, 
and  field  testing  of  the  design  contributes  to 
theory  building. 

Moreover,  it  should  be  noted  that  design  research 
does  not  emphasize  isolated  variables.  Although 
design  researchers  do  focus  on  specific  objects  and 
processes  in  specific  contexts,  they  try  to  study  those 
as  integral  and  meaningful  phenomena.  The  context- 
bound  nature  of  much  design  research  (conducted  with 
often  rather  small  and  purposive  samples)  also  explains 
why  researchers  usually  do  not  strive  for  context-free 
generalizations  from  sample  to  population,  based  on 
statistical  techniques.  Investments  are  needed  in  ana¬ 
lytical  forms  of  generalization;  readers  need  to  be  sup¬ 
ported  in  their  own  attempts  to  explore  the  potential 
transfer  of  research  findings  to  theoretical  propositions 
in  relation  to  their  own  contexts.  Reports  on  design 
research  can  facilitate  that  task  of  analogy  reasoning 


by  a  clear  articulation  of  the  design  principles  applied 
and  by  a  careful  description  of  both  the  evaluation 
procedures  and  the  implementation  contexts  (van  den 
Akker,  1999). 

Research-Based  Educational  Engineering 

Within  design  research,  one  may  want  to  distinguish 
between  validation  studies  (primarily  aiming  to  prove 
or  disprove  learning  theories)  and  development  studies 
(aiming  to  solve  an  educational  problem  using  relevant 
theoretical  knowledge).  The  latter  typically  includes 
the  following  stages  (Nieveen  et  al.,  2006;  van  den 
Akker,  1999): 

•  Preliminary  research  includes  thorough  con¬ 
text  and  problem  analysis  along  with  devel¬ 
opment  of  a  conceptual  framework  based  on 
literature  review.  Some  typical  activities 
include  literature  review,  consultation  of 
experts,  analysis  of  available  promising 
examples  for  related  purposes  and  contexts, 
and  case  studies  of  existing  practices  to  spec¬ 
ify  and  better  understand  needs  and  prob¬ 
lems  in  intended  user  contexts.  This  stage 
should  result  in  the  formulation  of  design 
principles  for  subsequent  design  activities. 

•  Prototyping  stage  sets  out  substantive  design 
specifications  and  procedural  guidelines  and 
optimizes  prototypes  through  iterative  cycles 
of  design,  formative  evaluation,  and  revision 
(Nieveen,  1999). 

•  Summative  evaluation  of  the  effectiveness 
preferably  also  includes  an  exploration  of 
transferability  and  scaling  (Burkhardt,  2006; 
Burkhardt  and  Schoenfeld,  2003). 

•  Systematic  reflection  and  documentation 
portrays  the  entire  study  to  support  retro¬ 
spective  analysis,  followed  by  specification 
of  design  principles  and  articulation  of  their 
links  to  the  conceptual  framework. 

The  whole  cycle  of  research-based  educational 
engineering  departs  from  a  sound  theoretical  base  by 
validation  studies,  develops  through  practical  under¬ 
standing  from  development  studies,  and  is  eventually 
tested  by  (large-scale)  effectiveness  research.  The 
cycle  also  shows  a  gradual  shift  from  exploratory  to 
confirmatory  approaches  (Nieveen  et  al.,  2006).  The 
increasing  call  for  more  evidence-based  learning  and 
teaching  approaches  also  implies  that  educational 
design  and  development  activities  themselves  have  a 
stronger  theoretical  and  empirical  base  that  underpins 
the  claims  for  evidence.  Research-based  educational 
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engineering  as  described  above  can  contribute  to  this 
challenge. 

Obviously,  these  more  comprehensive  tasks  cannot 
be  addressed  by  instructional  designers  or  educational 
researchers  only.  Such  challenges  require  thorough 
interaction  and  collaboration  between  designers, 
researchers,  practitioners,  and  policymakers  alike. 
Moreover,  these  interactions  preferably  go  beyond  the 
short-term  time  horizon  that  is  often  available  for 
design  and  research  activities.  They  would  be  sup¬ 
ported  by  a  programmatic  framework  that  can  connect 
and  sustain  a  series  of  separate  studies. 

Summary 

The  focus  of  this  chapter  section  is  research  on  instruc¬ 
tional  design  model  development,  use,  and  validation. 
Major  concepts  and  point  addressed  included  the  fol¬ 
lowing: 

•  Common  research  methods  as  regards  model 
development  are  literature  reviews,  surveys 
(based  on  interviews),  case  studies,  Delphi 
techniques,  and  think- aloud  protocols.  For 
research  on  model  use,  surveys  (based  on 
interviews),  case  studies  (interviews  and 
field  observations),  document  analyses,  and 
think-aloud  methods  are  employed. 
Research  methods  commonly  applied  in  the 
context  of  model  validation  are  experiments 
and  quasi-experiments,  interviews,  and 
expert  reviews.  Examples  of  studies  in  these 
three  categories  of  model  research  have  been 
described. 

•  Apparent  from  several  studies  is  the  strong 
need  for  a  better  correspondence  between, 
on  the  one  hand,  the  rather  abstract  instruc¬ 
tional  design  models  and,  on  the  other  hand, 
the  real  and  messy  world  of  professional 
design  and  development  practices. 
Acknowledging  that  most  of  the  deviations 
of  design  practice  from  design  theory  seem 
to  be  realistic,  the  models  need  to  be 
improved  to  better  match  actual  processes. 

At  the  same  time,  there  seems  to  be  ample 
room  for  improvement  of  existing,  often 
ineffective  practices. 

•  To  foster  the  blending  of  theory  and  practice 
and  to  make  the  models  more  relevant  for 
the  improvement  of  learning  (the  ultimate 
aim  of  instructional  design),  more  produc¬ 
tive  instructional  design  models  should 
address  learning  processes  and  outcomes 
from  multiple  perspectives  (not  only  a  focus 


on  students  but  also  on  teachers  who  operate 
within  curriculum  frameworks  at  various 
levels)  and  include  a  wide  range  of  influen¬ 
tial  variables.  Basically,  this  requires  a 
broadening  to  educational  design. 

•  The  relevance  and  meaningfulness  of 
instructional  design  would  also  be  improved 
if  the  scope  of  design  would  be  broadened 
and  elaborated.  Educational  design  is  not 
only  about  learning  activities.  Many  other 
related  questions  about  learning  design  are 
usually  at  stake.  From  a  broader  curriculum 
perspective,  it  makes  sense  to  pay  attention 
to  a  coherent  set  of  ten  components  that 
address  ten  specific  questions  about  the  plan¬ 
ning  of  student  learning:  rationale,  aims  and 
objectives,  content,  learning  activities, 
teacher  role,  materials  and  resources,  group¬ 
ing,  location,  time,  and  assessment. 

•  Besides  addressing  more  levels  and  compo¬ 
nents  of  learning,  instructional  design  also 
can  become  more  productive  by  linking  its 
efforts  to  other  educational  development 
activities.  To  increase  its  scope  and  potential 
impact  at  a  larger  scale,  it  is  recommended 
that  the  design  work  also  be  connected  to 
other  systemic  activities  in  the  domains  of 
teacher  education,  school  development,  and 
examinations  and  assessment. 

•  Moreover,  more  interactive  and  develop¬ 
mental  approaches  to  research  on  instruc¬ 
tional  design  models  are  needed.  Such 
research  approaches  are  often  labeled  as 
design-based  research  or  design  research. 
Educational  design  research  is  defined  as  “a 
series  of  approaches  with  the  intent  of  pro¬ 
ducing  new  theories,  artifacts,  and  practices 
that  account  for  and  potentially  impact  learn¬ 
ing  and  teaching  in  naturalistic  settings.” 
Interests  in  conducting  design  research 
include:  (1)  increasing  the  relevance  of 
research  for  educational  policy  and  practice, 
(2)  developing  empirically  grounded  theo¬ 
ries,  and  (3)  increasing  the  robustness  of 
design  practice.  Characteristic  to  design 
research  is  that  it  is  interventionist,  iterative, 
process  oriented,  utility  oriented,  and  theory 
oriented. 

•  Within  design  research,  one  may  want  to 
distinguish  between  validation  studies  (pri¬ 
marily  aiming  to  prove  or  disprove  learning 
theories)  and  development  studies  (aiming  to 
solve  an  educational  problem  using  relevant 
theoretical  knowledge).  The  latter  typically 
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includes  the  following  stages:  (1)  a  thorough 
context  and  problem  analysis  along  with 
development  of  a  conceptual  framework 
based  on  literature  review  (preliminary 
research);  (2)  setting  out  substantive  design 
specifications  and  procedural  guidelines  and 
optimizing  prototypes  through  iterative 
cycles  of  design,  formative  evaluation,  and 
revision  (prototyping);  (3)  summative  eval¬ 
uation  of  effectiveness  and  preferably  also 
an  exploration  of  transferability  and  scaling; 
and  (4)  systematic  reflection  and  documen¬ 
tation. 

•  The  whole  cycle  of  research-based  educa¬ 
tional  engineering  departs  from  a  sound  the¬ 
oretical  base  by  validation  studies,  develops 
through  practical  understanding  from  devel¬ 
opment  studies,  and  is  eventually  tested  by 
(large-scale)  effectiveness  research. 

RESEARCH  ON  DESIGN 
AND  DEVELOPMENT 

Rita  C.  Richey  and  James  D.  Klein 

This  section  of  the  chapter  examines  design  and  devel¬ 
opment  research.  This  type  of  research  offers  a  way  to 
test  theory  and  to  validate  practice  that  has  been  per¬ 
petuated  essentially  through  unchallenged  tradition.  In 
addition,  it  is  a  way  to  empirically  establish  new  pro¬ 
cedures,  techniques,  and  tools  based  on  a  methodical 
analysis  of  specific  cases. 

The  practice  of  design  and  development  is  empir¬ 
ical  by  nature.  Most  design  models  parallel  the  scien¬ 
tific  problem-solving  processes;  therefore,  it  would  be 
reasonable  to  assume  that  design  and  development  pro¬ 
cesses  themselves  have  robust  empirical  support,  but 
historically  little  research  has  been  directed  toward  our 
design  processes  and  products.  Design  and  develop¬ 
ment  research  is  an  empirical  approach  to  address  this 
problem. 

In  the  following  pages,  we  examine  methods,  strat¬ 
egies,  and  issues  related  to  conducting  design  and 
development  research.  Its  definition  and  scope  are  dis¬ 
cussed,  and  the  two  major  types  of  design  and  devel¬ 
opment  research  — product  and  tool  research  and 
model  research— are  explored.  Three  main  sources  for 
locating  design  and  development  research  problems 
are  provided.  The  methods  and  strategies  commonly 
used  in  design  and  development  studies  including 
qualitative,  quantitative,  and  mixed-methods 
approaches  are  examined.  Examples  from  the  literature 
are  provided  to  demonstrate  how  these  approaches 


have  been  successfully  used  by  researchers.  Method¬ 
ological  concerns  that  design  and  development 
researchers  commonly  face  are  addressed,  including 
the  constraints  of  conducting  research  in  natural  work 
settings  and  issues  of  researcher-participant  dual  roles . 
Recent  trends  in  design  and  development  research  are 
also  discussed  as  is  the  future  of  this  type  of  scholarly 
endeavor.  This  examination  of  design  and  development 
research  is  necessarily  concise.  Readers  who  are  inter¬ 
ested  in  learning  more  can  find  a  detailed  discussion 
in  Richey  and  Klein  (2007). 

The  Definition  and  Scope  of 
Design  and  Development  Research 

The  Evolving  Nature  of  Design 
and  Development  Research 

Design  and  development  research  seeks  to  create 
knowledge  grounded  in  data  systematically  derived 
from  practice.  Many  examples  of  design  and  develop¬ 
ment  research  can  be  found  in  the  literature  spanning 
up  to  four  decades.  In  recent  years,  however,  there  has 
been  renewed  interest  in  this  research  orientation  and 
the  benefits  it  offers  the  field.  Furthermore,  notions  of 
design  and  development  research  continue  to  evolve. 
Design  and  development  research*  can  be  defined  as 
the  systematic  study  of  design,  development,  and  eval¬ 
uation  processes  with  the  aim  of  establishing  an  empir¬ 
ical  basis  for  the  creation  of  instructional  and  non- 
instructional  products  and  tools  and  new  or  enhanced 
models  that  govern  their  development  (Richey  and 
Klein,  2007). 

Design  and  development  research  covers  a  wide 
spectrum  of  activities  and  interests.  In  its  simplest 
form,  it  can  be  either  (1)  the  study  of  the  process  and 
impact  of  specific  design  and  development  efforts,  or 
(2)  the  study  of  the  design  and  development  process 
as  a  whole,  or  of  particular  process  components.  Such 
research  can  involve  a  situation  in  which  someone  is 
studying  the  design  and  development  work  of  others; 
however,  it  can  also  involve  a  situation  in  which  some¬ 
one  is  performing  design  and  development  activities 
and  studying  the  process  at  the  same  time.  In  either 
case,  a  distinction  is  made  between  doing  design  and 


*  The  term  developmental  research  has  been  used  previously  to  describe 
this  approach  (Richey  and  Klein,  2005;  Richey  and  Nelson,  1996; 
Richey  et  al.,  2004;  Seels  and  Richey,  1994).  Over  the  years  the  term 
development  has  been  ambiguous  to  many  in  the  instructional  design 
and  technology  field  and  has  generated  considerable  discussion  regard¬ 
ing  its  meaning.  It  refers  to  many  other  areas  of  study,  such  as  human 
development,  international  development,  organizational  development, 
and  staff  development.  A  more  complete  rationale  for  this  terminology 
change  is  presented  in  Richey  and  Klein  (2007). 
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TABLE  54.2 

Types  of  Design  and  Development  Research 
Product  and  Tool  Research 

Emphasis  Study  of  specific  product  or  tool  design  and 
development  projects 

Outcome  Lessons  learned  from  developing  specific 
products  and  analyzing  the  conditions  that 
facilitate  their  use 

Context-Specific  Conclusions 


Model  Research 

Study  of  model  development,  validation  or  use 

New  design  and  development  procedures  or 
models  and  conditions  that  facilitate  their  use 

— >  Generalized  Conclusions 


development  and  studying  the  processes.  Design  and 
development  studies  often  involve  collaboration 
between  researchers  and  practitioners  using  a  wide 
range  of  qualitative  and  quantitative  techniques. 
Together  they  work  to  add  to  the  instructional  design 
knowledge  base  by  studying  the  nature  of  the  designer 
and  the  design  processes,  often  using  projects  from 
natural  work  environments. 

Types  of  Design  and  Development  Research 

The  two  major  types  of  design  and  development 
research  are  ( 1 )  product  and  tool  research  and  (2)  model 
research.  The  two  categories  of  design  and  develop¬ 
ment  research  have  been  previously  called  Type  1  and 
Type  2  (Richey  et  al.,  2004),  but  it  may  be  easier  to 
grasp  the  breadth  of  these  types  by  looking  at  their 
general  outcomes.  The  first  type  of  research  pertains 
primarily  to  studies  of  the  design  and  development  of 
products  and  tools,  such  as  Preese  and  Foshay’s  (1999) 
research  on  the  development  and  impact  of  a  set  of 
object-oriented  authoring  tools.  Often  the  entire  design 
and  development  process  is  documented.  Some 
research,  however,  concentrates  on  one  aspect  of  design 
and  development  only  (such  as  production)  or  de- 
emphasizes  some  phases  (such  as  needs  assessment). 
Many  recent  studies  focus  on  the  design  and  develop¬ 
ment  of  technology -based  instruction.  This  type  of 
research  has  a  tendency  to  combine  the  task  of  doing 
design  and  development  and  studying  the  processes. 

The  second  type  of  design  and  development 
research  pertains  to  studies  of  the  development,  vali¬ 
dation,  and  use  of  design  and  development  models, 
such  as  Adamski’s  (1998)  research  on  models  for 
designing  job  aids  used  in  high-risk  situations.  These 
studies  focus  on  the  models  and  processes  themselves, 
rather  than  their  demonstration.  Although  it  is  possible 
to  conduct  model  research  in  conjunction  with  the 
development  of  a  product  or  program,  most  model 
studies  concentrate  on  previously  developed  instruc¬ 
tion  and  consequently  are  not  project  specific.  Model 


research  may  address  the  validity  or  effectiveness  of 
an  existing  or  newly  constructed  development  model, 
process,  or  technique.  In  addition,  these  studies  often 
seek  to  identify  and  describe  the  conditions  that  facil¬ 
itate  successful  design  and  development. 

Generalized  and  Context-Specific  Conclusions 

In  the  contemporary  orientation  toward  research,  most 
accept  the  premise  that  research  can  have  a  broader 
function  than  the  creation  of  generalizable  statements 
of  law;  context-specific  studies  also  are  valued.  Design 
and  development  research  encompasses  studies  with 
conclusions  that  are  both  generalizable  and  contextu¬ 
ally  specific .  This  reflects  the  fact  that  product  and  tool 
research  typically  involves  studies  that  describe  and 
analyze  the  design  and  development  processes  used  in 
particular  projects  and  are  thus  to  a  great  extent  con¬ 
text  bound.  Model  research  studies,  on  the  other  hand, 
are  oriented  toward  a  general  analysis  of  design  and 
development  processes.  These  studies  tend  to  be  more 
generalizable  than  product  studies.  Table  54.2  portrays 
the  relationships  between  the  two  major  types  of 
design  and  development  research. 

Sources  of  Design  and  Development 
Research  Problems 

Although  the  idiosyncratic  interests  of  individual 
researchers  guide  much  research,  other  common  sources 
of  design  and  development  research  problems  include: 

•  Actual  workplace  settings  and  projects 

•  New  technology 

•  Theoretical  questions  prompted  by  the  liter¬ 
ature 

The  workplace  and  practitioners  are  primary  sources 
of  research  problems.  Those  problems  that  are  typically 
viewed  as  researchable  reflect  situations  that  are  recur¬ 
ring  and  common  to  many  settings,  are  considered  to 
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be  basically  solvable,  and  reflect  areas  of  current  inter¬ 
est  in  the  field.  One  example  of  problems  generated 
by  the  workplace  can  be  found  in  the  research  by 
Sullivan  et  al.  (2000),  who  were  involved  in  the  devel¬ 
opment  and  implementation  of  a  comprehensive,  K-12 
energy  education  curriculum  over  a  20-year  period  that 
was  used  by  more  than  12  million  students  in  the 
United  States.  In  this  case,  the  workplace  is  K-12 
educational  settings  across  the  country.  Their  research 
report  described  the  components  of  the  program  itself, 
including  instructional  objectives,  test  items,  and 
instructional  materials  and  provided  guidelines  for 
long-term  instructional  development  projects.  The 
problems  addressed  by  Sullivan  et  al.  (2000)  were 
important  because  the  curriculum  in  question  is  com¬ 
monly  used  by  many  school  districts.  Moreover,  the 
scope  of  the  program  implementation  is  so  large  that 
the  results  are  viable  for  many  school  districts,  and  the 
long-term  nature  of  the  project  provides  an  opportunity 
to  examine  retrospective  data  collected  on  many  stu¬ 
dents  from  grades  K-12. 

Today,  many  of  the  cutting-edge  research  topics 
relate  to  the  development  and  use  of  emerging  technol¬ 
ogies.  In  general,  these  studies  focus  on  how  we  can 
best  take  advantage  of  the  potential  of  the  new  tech¬ 
nologies.  The  questions  that  people  have  with  respect 
to  technology-based  design  and  development  are  often 
simple— How  do  you  do  it?  What  are  the  problems  I 
can  expect?  How  can  I  avoid  them?  What  resources  are 
needed?  Problems  such  as  these  tend  to  lend  themselves 
to  research  on  product  development  in  which  an  actual 
project  is  documented  and  studied.  Corry  et  al.  (1997) 
conducted  a  study  that  involved  the  design  and  usability 
of  a  university  website.  The  object  was  to  identify  ways 
of  maximizing  the  usability  of  informational  websites. 
They  conducted  a  case  study  of  the  design  processes 
and  strategies  implemented  to  ensure  usability.  The 
topic  of  this  study  is  important  because  website  design 
is  a  topic  of  great  interest  today  and  one  in  which  many 
developers  are  currently  involved. 

Instructional  design  (ID)  models  continue  to  serve 
as  an  important  part  of  the  theory  base  of  the  field,  and 
as  such  provide  direction  for  identifying  design  and 
development  research  problems.  These  problems  tend 
to  focus  upon  a  validation  of  ID  models  themselves 
and  the  study  of  particular  elements  and  phases  of  the 
ID  process.  They  are  important  to  the  substantiation 
of  the  design  process.  ID  model  validation  can  be 
viewed  as  either  an  internal  validation  of  the  compo¬ 
nents  and  process  of  the  model  or  the  external  valida¬ 
tion  of  the  impact  of  the  products  of  a  model’s  use 
(Richey,  2005). 

Another  category  of  theory-based  problems 
attuned  to  design  and  development  research  techniques 


concerns  the  nature  of  designer  thinking  and  decision 
making.  The  study  by  Perez  and  Emery  (1995)  is  an 
example  of  this  type  of  research.  They  sought  to  iden¬ 
tify  the  differences  in  cognitive  processing  and  prob¬ 
lem-solving  paths  of  expert  and  novice  designers.  The 
design  task  for  each  participant  was  controlled;  each 
was  presented  with  the  same  design  task  and  was  given 
a  standard  time  limit.  This  study’s  problem  was  impor¬ 
tant  because  it  attempted  to  determine  the  actual  design 
processes  used,  understand  the  different  thinking  tac¬ 
tics  used  by  persons  of  varying  levels  of  design  exper¬ 
tise,  and  relate  the  cognitive  psychology  research  lit¬ 
erature  to  the  ID  process. 

The  Characteristics  of  Design  and 
Development  Research  Methodology 

Design  and  development  research  is  an  umbrella  term 
for  the  study  of  design  and  development  using  a  broad 
collection  of  traditional  methods  and  strategies,  both 
quantitative  and  qualitative.  Which  methods  are 
selected  and  how  these  methods  are  employed  depend 
not  only  on  the  nature  of  the  research  problem  and 
questions  but  also  on  whether  the  study  involves  prod¬ 
uct  and  tool  research  or  model  research.  The  following 
section  examines  three  aspects  of  design  and  develop¬ 
ment  research: 

•  Common  research  methodologies 

•  The  use  of  mixed  and  multiple  methods 
research  techniques 

•  Typical  participants  and  settings  employed 

Commonly  Used  Design  and 
Development  Methods  and  Strategies 

Design  and  development  research  uses  a  wide  variety 
of  methodologies.  Much  design  and  development 
research— both  product  and  tool  research  and  model 
research— relies  on  a  variety  of  qualitative  techniques, 
including  case  studies,  interviews,  document  reviews, 
and  observations.  Evaluation  research  techniques  (both 
qualitative  and  quantitative)  are  also  included  in  many 
studies  that  focus  on  product  and  tool  development. 
Model  development  and  use  studies  often  employ  sur¬ 
vey  research  techniques,  while  model  validation  stud¬ 
ies  frequently  use  traditional  experimental  designs. 
Some  of  the  most  commonly  used  design  and  devel¬ 
opment  research  methods  are  shown  in  Table  54.3. 
Design  and  development  research  tends  to  rely  more 
on  qualitative  techniques  than  on  quantitative  tech¬ 
niques.  Perhaps  this  is  because  this  research  often 
demands  the  use  of  exploratory  procedures,  rather  than 
employing  the  controls  required  by  many  quantitative 
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TABLE  54.3 

Common  Methods  Used  in  Design  and  Development  Research 

Type  of  Research  Project  Emphasis  Research  Methods  Employed 

Product  and  tool  research  Comprehensive  design  and  development  projects  Case  study,  content  analysis,  evaluation,  field  observation. 


Phases  of  design  and  development 

Tool  development  and  use 
Model  research  Model  development 

Model  validation 
Model  use 


in-depth  interview 

Case  study,  content  analysis,  expert  review,  field  observation, 
in-depth  interview,  survey 

Evaluation,  expert  review,  in-depth  interview,  survey 

Case  study,  Delphi,  in-depth  interview,  literature  review, 
survey,  think-aloud  methods 

Experimental,  expert  review,  in-depth  interview 

Case  study,  content  analysis,  field  observation,  in-depth 
interview,  survey,  think-aloud  methods 


methods.  Perhaps  it  is  because  these  studies  tend  to 
deal  more  with  real-life  projects  than  with  simulated 
or  contrived  projects.  In  any  case,  design  and  devel¬ 
opment  research  is  always  applied  research  and  related 
to  the  practice  of  ID,  rather  than  an  examination  of 
isolated  variables. 

Common  Use  of  Mixed  and 
Multiple  Methods  Research 

Design  and  development  research  tends  to  be  com¬ 
plex  methodologically.  This  is  typically  because  of 
the  complexities  of  real-life  situations  and  of  the 
design  and  development  processes  themselves.  As  a 
result,  this  research  tends  to  employ  either  a  mixed- 
methods  or  multiple  methods  approach.  The  term 
mixed-methods  research  describes  those  studies  that 
combine  qualitative  and  quantitative  methods.  An 
example  of  a  mixed-method  approach  is  a  study  by 
Plummer  et  al.  (1992),  who  used  two  research  meth¬ 
odologies— case  study  and  experimental— in  con¬ 
junction  with  an  ID  task.  This  project  involved  the 
development  of  a  job  aid  for  use  by  military  personnel 
operating  a  complicated  piece  of  communications 
equipment.  The  experimental  phase  involved  an  eval¬ 
uation  of  the  effectiveness  of  the  job  aid  and  com¬ 
pared  three  different  types  of  uses  of  the  job  aid.  This 
study  provided  impact  information,  as  well  as  infor¬ 
mation  relating  to  the  best  conditions  for  developing 
and  using  the  product. 

Many  design  and  development  studies,  however, 
that  employ  multiple  research  methods  are  not  mixing 
the  quantitative  and  qualitative  orientations.  They  are 
simply  making  use  of  a  variety  of  similar  strategies; 
for  example,  Corry  and  colleagues  (1997)  used  several 
qualitative  methods  when  they  studied  product  devel¬ 
opment  by  conducting  in-depth  interviews,  as  well  as 
conducting  field  observations. 


Typical  Participants  and  Settings 

Design  and  development  research  is  often  context 
bound,  and  the  nature  of  the  conditions  in  which  peo¬ 
ple  work  is  critical  to  the  processes  and  outcomes  of 
a  particular  project;  consequently,  the  setting  of  a  study 
is  typically  as  important  as  are  the  people  participating 
in  the  study.  Each  design  and  development  setting  has 
a  unique  culture  and  set  of  conditions  that  can  have  a 
profound  impact  on  the  activities  occurring  there.  This 
impact  is  most  critical  when  the  study  is  being  con¬ 
ducted  in  a  natural  work  environment.  The  researcher’s 
most  crucial  decision  with  respect  to  setting  is  whether 
it  includes  the  conditions  and  elements  addressed  in 
the  research  questions;  however,  additional  concerns 
include  feasibility  (i.e.,  how  practical  is  it  to  conduct 
a  study  in  that  setting?),  access  (i.e.,  will  the  organi¬ 
zation  allow  all  needed  personnel  to  participate  in  the 
research?),  and  proprietary  rights  (i.e.,  will  key  docu¬ 
ments  be  available  to  the  researcher  and  will  the  data 
be  publishable?). 

Participants  involved  in  design  and  development 
research  may  or  may  not  be  selected  because  of  their 
association  with  a  particular  organization;  however, 
they  are  almost  always  selected  because  of  their  par¬ 
ticular  role  in  the  design  and  development  process. 
Studies  documenting  and  analyzing  the  development 
of  an  instructional  product  would  obviously  include 
those  persons  involved  in  designing  the  actual  prod¬ 
uct-designers,  developers,  and  perhaps  clients  and 
evaluators.  At  times,  the  design  team  as  a  whole  par¬ 
ticipates  in  such  research.  Studies  of  the  impact  of  an 
innovative  product  or  tool  would  include  those  who 
are  affected  by  its  use— clients,  learners,  instructors, 
and  perhaps  supervisors.  Participants  in  model  devel¬ 
opment  and  model  use  studies  often  parallel  those  in 
product  development  and  use.  Model  validation  studies 
can  include  persons  in  all  of  these  roles.  Participants 
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in  design  and  development  research  tend  to  be  (not 
surprisingly)  designers  and  developers,  rather  than 
learners  and  instructors.  This  is  a  key  distinction 
between  design  and  development  research  and  tradi¬ 
tional  teaching-learning  research.  Even  so,  a  wide  vari¬ 
ety  of  people  serving  in  many  roles  are  critical  to 
design  and  development  research. 

What  is  rather  unique  to  design  and  development 
studies  are  the  non-human  entities  that  serve  as  research 
participants;  for  example,  it  is  not  unusual  for  the 
project  itself  to  serve  as  a  type  of  participant.  In  these 
situations,  the  project  characteristics  become  a  key 
focus  of  the  research.  For  example,  as  projects  can 
differ  in  terms  of  the  types  of  products  they  create,  with 
possible  delivery  system  variations  or  content  varia¬ 
tions.  These  differences  often  become  key  rationales 
for  the  inclusion  of  a  given  project  in  a  study.  In  these 
studies,  the  project  becomes  a  key  unit  of  analysis. 

In  some  design  and  development  studies,  the  par¬ 
ticipants  are  organizations  themselves.  Types  of  orga¬ 
nizations  may  be  selected  to  provide  a  systematic  dis¬ 
tribution  of  setting  constraints  within  a  particular  study. 
In  this  way,  environmental  effects  may  be  identified. 
The  organization’s  size  in  terms  of  personnel  or  budget, 
its  focus,  its  philosophy  — all  can  be  factors  in  selecting 
particular  organizations  to  participate  in  a  study.  Some 
studies,  for  example,  may  seek  to  determine  the  differ¬ 
ences  among  the  design  and  development  practices  in 
educational,  corporate,  or  military  settings. 

Product  and  Tool  Research: 

Methods  and  Strategies 

Methods  for  Research  on  Product  Development 

The  classic  product  development  study  is  descriptive 
research  using  case  study  methods.  Many  of  these 
studies  describe  the  entire  life  span  of  the  product 
development  process  in  detail.  Research  by  Visser  et 
al.  (2002)  is  an  example  of  a  product  development 
study,  from  an  initial  pilot  study  to  a  year-long  try-out 
to  evaluation  of  the  product.  The  entire  product  devel¬ 
opment  effort  was  an  empirical  process.  The  research 
culminated  in  an  instructional  product  that  addresses 
learner  motivation  and,  in  turn,  is  intended  to  reduce 
the  drop-out  and  noncompletion  rates  of  distance  edu¬ 
cation  programs. 

The  pilot  study  was  designed  to  identify  the  char¬ 
acteristics  of  effective  motivational  messages  and  to 
develop  a  prototype  of  such  a  message.  Participants 
were  17  international  students  enrolled  in  one  distance 
education  course.  Data  were  collected  from  student 
records,  interviews  with  program  staff,  and  end-of- 
course  questionnaires. 


The  main  study  was  extended  to  81  students 
enrolled  in  5  distance  education  courses,  of  which  2 
used  personalized  motivational  messages,  2  collective 
motivational  messages,  and  1  no  motivational  mes¬ 
sages.  Each  course  was  considered  as  a  separate  year¬ 
long  case  study.  Instructors  developed  the  motivational 
messages  for  their  own  courses  according  to  a  struc¬ 
tured  plan  devised  by  the  researcher.  A  variety  of  data 
collection  instruments  were  used  in  addition  to  those 
used  in  the  pilot  study.  Student-related  data  were  col¬ 
lected  with  pre-  and  post-course  questionnaires  and 
telephone  interviews.  Instructors  provided  existing 
course  completion  records,  questionnaires,  course  log¬ 
books,  and  time  monitoring  sheets  and  also  had  in- 
depth  interviews  with  the  researcher.  These  data 
enabled  the  researchers  to  determine  if  motivational 
messages  were  effective  and  efficient  and  which  types 
of  messages  were  superior. 

Methods  for  Research  on 
Tool  Development  and  Use 

Recently,  some  researchers  have  been  concentrating 
on  studying  the  development  and  use  of  tools  that 
either  make  design  and  development  itself  easier  or 
support  the  teaching/leaming  process.  Not  surpris¬ 
ingly,  the  bulk  of  these  have  been  computer-based 
tools,  and  research  has  been  directed  toward  automat¬ 
ing  design  and  development  through  their  use.  Below 
is  a  representative  example  of  a  tool  use  study. 

Nieveen  and  van  den  Akker’s  (1999)  tool  research 
focused  on  a  computer  system  that  served  as  a  perfor¬ 
mance  support  tool  for  designers  during  the  formative 
evaluation  phase  of  an  ID  project.  This  particular  study 
concentrated  on  the  use  of  the  tool,  seeking  to  assess 
not  only  the  effectiveness  of  the  tool  but  also  its  prac¬ 
ticality.  In  Phase  1 ,  65  designers  and  developers  with 
varied  experience  and  expertise  participated  in  a  work¬ 
shop  on  the  use  of  the  tool  and  then  had  an  hour  to 
use  it  to  develop  a  formative  evaluation  plan.  The 
computer  system  tracked  participant  actions  and  time- 
on-screen  data.  The  workshop  leader  and  assistant 
noted  comments  and  the  support  they  provided  to 
designers  during  the  exercise.  The  contents  of  the  eval¬ 
uation  plans  constructed  during  the  workshop  were 
collected  for  analysis,  and  a  post-workshop  question¬ 
naire  was  used  to  gather  perceptions  of  practicality, 
and  other  reactions  to  the  tool. 

Phase  2  involved  a  try-out  with  four  ID  projects 
currently  underway  in  two  different  work  settings. 
Three  of  the  designers  were  experienced  and  one  was 
a  novice.  Data  collected  included:  (1)  two  or  three 
interviews  per  designer  after  completion  of  signifi¬ 
cant  parts  of  the  formative  evaluation  task,  and  (2) 
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documentation  of  the  evaluation  activities  completed 
(evaluation  plan,  instruments,  final  report,  log  book  of 
activities).  Using  a  synthesis  of  the  data  collected,  a 
detailed  report  of  each  try-out  project  was  written. 
Summaries  of  these  project  reports  were  also  written. 
Participating  designers  then  reviewed  these  summaries 
and  recommended  minor  changes.  The  findings  of  this 
research  led  to  changes  in  the  tool,  confirmation  of  the 
assumption  that  the  tool  was  appropriate  for  both  nov¬ 
ice  and  experienced  designers,  and  identification  of  the 
constraints  of  daily  practice  that  impacted  the  use  and 
effectiveness  of  the  tool. 

Like  many  other  examples  of  design  and  develop¬ 
ment  research,  this  tool  use  study  used  a  mixed-meth¬ 
ods  design.  In  Phase  1,  the  methods  employed  were 
survey  research  techniques  and  content  analysis.  In 
Phase  2,  the  methods  were  in-depth  interviews  and 
content  analysis. 

Model  Research:  Methods  and  Strategies 

Methods  for  Research  on  Model  Development 

In  spite  of  the  widespread  use  of  models  in  the  field, 
there  is  a  lack  of  research  on  model  formation.  Many 
ID  models  are  constructed  by  conducting  a  literature 
review  of  theory  and  research  and  then  synthesizing 
the  findings;  however,  some  research  does  specifically 
address  model  construction  procedures.  A  study  by 
Jones  and  Richey  (2000)  is  an  example  of  model  devel¬ 
opment  research  using  multiple  qualitative  methods. 
The  outcome  of  the  study  was  a  revised  rapid  proto¬ 
typing  ID  model  that  describes  designer  tasks  per¬ 
formed,  the  concurrent  processing  of  those  tasks,  and 
the  nature  of  customer  involvement.  Data  were 
obtained  from  a  natural  design  and  development  work 
setting.  Designers  and  clients  from  two  contrasting 
projects  participated  in  the  study.  The  projects  varied 
in  terms  of  size,  product,  and  industry.  Both  projects 
had  been  completed  at  the  time  of  the  research. 

The  designers  constructed  task  logs  in  which  they 
identified  the  tasks  they  had  completed,  when  they 
were  completed,  and  the  time  it  took  to  do  each  job. 
As  part  of  the  logging  process,  designers  reviewed 
extant  data,  including  the  project  proposal,  time  sheets, 
design  memos,  prototype  specifications,  and  the  final 
product.  Next,  a  structured  hour-long  interview  was 
conducted  with  each  designer  to  determine  the  impact 
of  the  rapid  prototyping  methodologies  on  the  ID  pro¬ 
cess  and  on  product  quality.  Telephone  interviews  were 
conducted  with  the  customer.  All  interviews  were  tape- 
recorded. 

After  coding  and  synthesizing  the  data,  the 
researchers  could  determine  which  design  and  devel¬ 


opment  tasks  were  performed  by  whom  and  when.  The 
time  expended  on  each  task  was  determined  and  an  ID 
rapid  prototyping  procedural  model  was  devised.  The 
primary  methods  used  in  this  model  development 
research  were  content  analysis  of  extant  data  and 
newly  constructed  diaries,  as  well  as  in-depth  inter¬ 
views  . 

Methods  for  Research  on  Model  Validation 

Design  models  can  be  externally  validated  by  isolating 
the  effects  of  the  given  model  as  compared  to  the  use 
of  another  model  or  approach.  A  study  by  Tracey 
(2002)  is  an  example  of  controlled  testing  validation. 
Tracey  compared  use  of  the  Dick  and  Carey  (1996) 
model  with  an  instructional  systems  design  (ISD) 
model  enhanced  with  a  consideration  of  multiple  intel¬ 
ligences.  This  served  as  an  external  validation  of  the 
newly  constructed  multiple  intelligences  (MI)  design 
model.  She  established  two  design  teams,  each  with 
two  novice  designers.  One  team  worked  with  the  Dick 
and  Carey  model,  and  the  second  used  the  MI  model. 
Both  teams  were  instructed  to  design  a  2-hour,  instruc¬ 
tor-led  classroom-based  team  building  course  for  a 
nonprofit  organization.  The  teams  each  received:  (1) 
materials  regarding  the  organization;  (2)  written  con¬ 
tent  on  team  building;  (3)  audience,  environment,  and 
gap  analysis  information;  and  (4)  an  ISD  model. 

All  four  instructional  designers  met  as  a  group  to 
assess  the  needs ,  identify  the  goals  of  the  team  build¬ 
ing  training,  and  review  the  content.  The  teams  then 
separated  to  conduct  their  own  learner  and  environ¬ 
mental  analyses  based  on  the  information  given  and 
their  assigned  model.  Both  teams  worked  individu¬ 
ally  to  create  their  performance  objectives  based  on 
the  results  of  their  analyses  and  the  steps  of  the  design 
model  they  were  using.  The  teams  reconvened  to 
review  their  objectives  to  ensure  that  each  met  the 
same  team-building  goals,  and  together  they  wrote 
the  post-test  questions.  The  teams  then  separated 
again  and  worked  simultaneously  in  the  same  loca¬ 
tion  in  two  different  offices  during  the  remainder  of 
the  design  and  development  process.  Each  group  was 
provided  computers,  an  office,  any  desired  training 
materials,  and  a  desktop  publisher  to  create  the 
instructor  guide,  participant  guide  templates,  and  all 
finished  products. 

The  resulting  programs  were  implemented  and 
their  impact  compared  in  terms  of  learner  knowledge 
and  attitudes.  Instrumentation  used  during  the  study 
included  designer  logs  giving  reactions  to  their  tasks 
and  the  models  used,  learner  knowledge  post-tests, 
and  surveys  addressing  participant  reactions  to  the 
instruction. 
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Methods  for  Research  on  Model  Use 

Design  and  development  research  that  focuses  on  model 
use  addresses  ID  processes  as  they  occur  naturally  or 
concentrates  on  the  use  of  a  particular  model.  A  study 
by  Visscher-Voerman  and  Gustafson  (2004)  provides 
an  example  of  how  in-depth  interviews  can  allow  design 
and  development  researchers  to  explore  the  intricacies 
of  the  ID  process  as  it  actually  occurs.  Twenty-four 
different  designers,  including  those  with  and  without 
formal  design  training,  were  selected  according  to  pre¬ 
determined  criteria.  These  designers  represented  six  dif¬ 
ferent  design  settings.  The  study  had  two  phases.  In  the 
first  phase,  12  designers,  2  per  setting,  were  interviewed 
for  about  1-1/2  hours.  The  first  interview  included  dis¬ 
cussions  of  their  backgrounds  and  their  projects.  They 
were  asked  about  the  strategies  they  had  used  in  a  recent 
project  and  were  encouraged  to  give  concrete  examples, 
to  explain  why  they  worked  in  a  given  manner,  and  why 
they  deviated  from  general  plans.  These  interviews 
were  audiotaped  and  transcribed.  Interview  data  were 
corroborated  through  an  examination  of  related  project 
documents.  A  report  was  then  generated  based  on  the 
interviews  that  covered  each  designer’s  profile,  a  project 
summary,  and  a  project  description.  The  report  was 
revised  after  the  designer  reviewed  and  reacted  to  it. 
This  process  of  verifying  the  accuracy  of  one’s  data  is 
called  member  checking  (Creswell,  2002).  The  revised 
reports,  rather  than  the  interviews,  were  used  as  the 
basis  for  the  preliminary  analysis. 

The  preliminary  analysis  began  by  labeling  each 
design  activity  described  using  analysis,  design,  devel¬ 
opment,  implementation,  and  evaluation  (ADDIE)  ter¬ 
minology.  Next,  there  was  a  more  detailed  analysis  of 
each  major  activity  as  described  by  each  designer. 
These  analyses  were  then  synthesized  and  interpreted 
in  light  of  the  literature  to  produce  four  different  gen¬ 
eral  design  paradigms  (not  models),  each  of  which 
represents  a  unique  philosophical  orientation. 

In  the  second  round  of  in-depth  interviews,  two 
designers  from  each  setting  were  interviewed  follow¬ 
ing  the  same  procedures  as  had  been  previously  used, 
and  data  were  initially  analyzed  in  the  same  manner. 
All  data  were  then  combined  and  analyzed  to  deter¬ 
mine  the  underlying  rationales  for  designer  actions. 
Detailed  descriptions  of  designer  activities  and  their 
underlying  paradigms  resulted. 

Unique  Design  and  Development 
Research  Methodology  Concerns 

Methodological  issues  that  tend  to  be  more  problem¬ 
atic  for  design  and  development  researchers  than  for 
scholars  dealing  with  other  topics  include: 


•  Avoiding  biases  when  researchers  assume 
participant  roles 

•  Difficulties  working  with  recall  data 

•  Constraints  on  research  in  natural  work  envi¬ 
ronments 

Researcher-Participant  Dual  Roles 

In  most  research,  it  is  unusual  for  the  researcher  to  be 
a  part  of  the  observations.  In  design  and  development 
research,  especially  in  product  and  tool  development 
studies,  researchers  often  are  the  designers  and  devel¬ 
opers.  In  other  words,  by  design  the  researchers  are 
observing  themselves.  This  is  a  far  different  matter 
than  the  participant  observation  that  is  common  in 
ethnographic  and  field  research  and  invites  the  possi¬ 
bility  of  serious  flaws  in  the  data.  Any  time  that 
researchers  assume  dual  roles  in  a  project,  the  research 
design  of  the  study  must  delineate  specific  strategies 
for  ensuring  unbiased  data.  The  most  common  tech¬ 
niques  of  doing  this  are  by  using  carefully  structured 
data  collection  instruments  and  by  triangulating  mul¬ 
tiple  sources  of  data. 

In  an  illustrative  study,  McKenney  and  van  den 
Akker  (2005)  assumed  the  multiple  roles  of  not  only 
researchers  and  designers  but  also  developers,  program¬ 
mers,  and  evaluators.  They  used  the  following  strategies 
to  avoid  interjecting  personal  biases  into  the  dataset: 

•  Establish  procedures  to  segregate  data  and 
subjective  inferences. 

•  Triangulate  data  by  collecting  data  of  differ¬ 
ent  types  and  data  from  different  sources. 

•  Use  different  investigators  to  collect  data. 

•  Conduct  data  analysis  and  interpretation 
with  a  larger  group  of  colleagues  and 
research  assistants. 

These  are  reasonable  procedures  that  could  also  be 
applied  in  other  studies  in  which  the  researcher 
assumes  the  role  of  the  designer  or  developer. 

If  researchers  do  assume  participant  roles  in  prod¬ 
uct  and  tool  research,  it  is  more  likely  to  be  that  of  the 
designer,  but  not  always.  It  is  not  unusual  for  instruc¬ 
tors  to  be  involved,  especially  in  the  try-out  of  a  prod¬ 
uct.  Once  again,  procedures  must  be  established  to 
prevent  the  researcher  from  unintentionally  interject¬ 
ing  biases  into  the  dataset. 

Working  with  Recall  Data 

Although  it  is  always  preferred  that  design  and  devel¬ 
opment  research  be  conducted  while  the  project  is 
underway,  this  is  not  always  possible,  especially  if  one 
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is  dealing  with  the  time  constraints  of  real  projects 
occurring  in  natural  work  settings.  The  dangers  of 
using  recall  data  are  obvious— people  may  forget 
project  details.  When  it  is  necessary  to  use  recall  data, 
provisions  should  be  made  for:  (1)  using  systematic 
procedures  to  prompt  and  stimulate  participants’  mem¬ 
ories,  and  (2)  verifying  the  accuracy  of  the  data  that 
are  provided.  Many  techniques  for  accomplishing 
these  goals  are  available,  including: 

•  Having  participants  construct  project  task 
logs  and  using  extant  data  to  verify  these 
project  recollections 

•  Requiring  participants  to  complete  a  pre- 
interview  activity  form  to  stimulate  recall 

•  Triangulating  data  from  multiple  sources 

•  Allowing  participants  to  verify  the  accuracy 
of  their  reports 

Research  designs  using  recall  data  should  also  provide 
for  multiple  sources  of  data  to  help  ensure  the  reliabil¬ 
ity  of  a  dataset. 

Research  in  Natural  Work  Environments 

Design  and  development  research  is  by  nature  an 
applied  type  of  research.  Most  studies  seek  to  collect 
data  from  realistic  design  and  development  situations, 
be  they  simulated  or  naturally  occurring.  Product  and 
tool  research  is  almost  exclusively  the  study  of  actual 
design  and  development  efforts.  When  studying  ID 
processes  that  occur  in  real-world  settings,  one  is 
blessed  with  authenticity  and  cursed  with  a  myriad  of 
contextual  factors,  some  of  which  are  relevant  and 
others  that  are  not.  By  and  large,  such  studies  recog¬ 
nize  the  influence  of  the  work  environments  through 
full  descriptions;  for  example,  design  and  development 
studies  have  addressed  the  following  aspects  of  work 
and  instructional  contexts: 

•  Characteristics  and  backgrounds  of  the 
learners  for  whom  the  product,  program,  or 
tool  was  designed 

•  Characteristics  and  backgrounds  of  the 
designer/developers  for  whom  the  tool  was 
designed 

•  Structure  of  the  design  and  development  team 

•  Facilities  used  for  product  implementation 

•  Resources  and  procedures  utilized  in  the  dis¬ 
tribution  and  implementation  of  the  product, 
tool  or  program 

In  addition  to  including  contextual  variables  in  case 
studies,  some  studies  also  address  such  factors  in  quan¬ 
titative  analyses.  Typically,  this  occurs  in  a  product  or 


program  evaluation  phase;  for  example,  Sullivan  et  al. 
(2000)  measured  school  administrator  attitudes,  an 
organizational  climate  characteristic,  and  considered 
their  impact  on  the  success  of  their  energy  curriculum. 

Trends  in  Design  and  Development  Research 

Design  and  development  research  is  expanding 
(Richey  and  Klein,  2007;  Wang  and  Hannafin,  2005) 
with  more  persons  tackling  design  problems  from  this 
relatively  new  research  perspective  .  New  products  and 
programs  are  being  empirically  studied.  New  models 
are  being  devised  and  validated.  In  this  section,  we 
provide  a  brief  overview  of  these  trends  and  speculate 
on  the  future  role  of  design  and  development  research 
in  our  field. 

Trends  in  Product  and  Tool  Research 

The  design  and  development  of  instructional  products 
and  programs  are  considered  by  many  to  be  the  heart 
of  our  field.  Product  and  tool  studies  most  frequently 
concern  the  use  of  instructional  interventions  in 
schools  or  in  employee  training  situations.  Recent 
research  has  also  focused  on  non-instructional  prod¬ 
ucts  and  tools;  for  example,  Nguyen  (2005)  described 
a  needs  assessment  conducted  to  determine  the  types 
of  electronic  performance  support  systems  (EPSSs) 
that  training  professionals  and  end  users  think  are  valu¬ 
able  to  facilitate  performance.  In  addition,  Nguyen  et 
al.  (2005)  conducted  an  experiment  in  which  employ¬ 
ees  at  a  manufacturing  company  completed  a  proce¬ 
dural  task  and  either  received  support  from  an  intrinsic, 
extrinsic,  or  external  EPSS  or  received  no  support. 
Recent  trends  in  the  study  of  non-instructional  inter¬ 
ventions  have  caused  us  to  expand  our  definition  of 
design  and  development  research  (see  above).  This  is 
in  keeping  with  expanded  definitions  of  the  field  that 
encompass  notions  of  performance  improvement  and 
non-instructional  interventions  (Reiser,  2007). 

Another  trend  in  product  and  tool  research  reflects 
the  evolution  of  media  and  technology  use  in  the  field; 
for  example,  early  product  research  with  an  audio¬ 
visual  emphasis  is  exemplified  by  Greenhill’s  (1955) 
description  of  producing  low-cost  motion  pictures  with 
sound  for  instructional  purposes  and  by  Cambre’s 
(1979)  study  of  the  development  of  formative  evalua¬ 
tion  procedures  for  instructional  television.  With  the 
increased  computer-based  instruction  in  schools  and 
training  settings,  research  studies  focusing  on  the 
design,  development  and  evaluation  of  these  products 
and  tools  also  increased  (Buch,  1989;  Kearsley,  1986; 
Lewis  et  al.,  1993;  Munro  and  Towne,  1992).  Trends 
in  distance  learning  products  and  Web-based  tools 
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have  provided  another  stimulus  for  design  and  devel¬ 
opment  researchers  (Chou  and  Sun,  1996;  Corry  et  al., 
1997;  Koszalka,  2001;  Visser  et  al.,  2002). 

Trends  in  Model  Research 

Although  design  and  development  models  will 
undoubtedly  continue  to  proliferate  as  they  have  in  the 
past,  research  on  these  models  is  in  its  infancy;  none¬ 
theless,  a  growing  amount  of  empirical  work  supports 
the  model  development  process.  Much  of  this  work 
concerns  the  development  of  instruction  using  newer 
models  and  procedures  of  instructional  design  that 
reflect  the  influence  of  cognitive  science  and  construc¬ 
tivism.  Cronje  (2006),  for  example,  has  constructed 
and  tested  a  new  model  that  integrates  objectivist  and 
constructivist  design  orientations,  and  Schatz  (2005) 
developed  and  tested  a  model  for  developing  learning- 
object  tags  for  use  by  K-12  educators.  In  many  cases, 
newer  model  development  research  is  being  blended 
with  model  validation  efforts,  such  as  in  a  study  by 
Tracey  and  Richey  (2007). 

Model  use  research,  on  the  other  hand,  has  been 
an  important  research  genre  for  some  time,  especially 
in  an  array  of  studies  on  designer  decision  making 
(Goel  and  Pirolli,  1988;  Nelson,  1990).  This  line  of 
research  continues  with  a  new  emphasis  on  the  impact 
of  specific  designer  characteristics  on  the  design  pro¬ 
cess.  Most  notable  are  the  studies  of  the  nature  of 
designer  expertise  and  the  differences  between  novice 
and  expert  designers  (Kirschner  et  al.,  2002;  Perez  and 
Emery,  1995;  Rowland,  1992;  Tennyson,  1998).  Other 
characteristics,  however,  have  been  studied  to  deter¬ 
mine  their  impact  on  the  use  of  design  models;  for 
example.  Chase  (2003)  studied  the  effects  of  gender 
and  gender-related  thinking  processes  on  the  use  of 
design  models.  Model  research  is  growing  and  is  open 
to  the  exploration  of  a  very  broad  array  of  theoretical 
orientations,  work  settings,  and  variables  of  interest. 

The  Future  of  Design  and  Development  Research 

In  addition  to  the  traditional  design  and  development 
research  discussed  here,  other  researchers  are  using 
alternative  approaches  to  address  many  of  the  same 
problems.  The  most  prominent  of  these  approaches  is 
design-based  research.  Design-based  research  is  the 
study  of  learning  in  context  (Design-Based  Research 
Collective,  2003;  also  see  van  den  Akker  and  Kuiper  in 
this  chapter).  Using  this  approach  (Winn,  2004,  p.  104): 

...designers  build  tools  that  they  test  in  real  classrooms 
and  gather  data  that  contribute  both  to  the  construction 
of  theory  and  to  the  improvement  of  the  tools.  This  pro¬ 


cess  proceeds  iteratively,  over  a  period  of  time,  until  the 
tool  is  proven  to  be  effective  and  our  knowledge  of  why 
it  is  effective  has  been  acquired  and  assimilated  in  theory. 

It  seeks  to  clarify  the  nature  of  the  process  by  which 
people  learn,  typically  through  their  interactions  with 
instructional  materials  and  other  learners.  This  type  of 
research  design  has  been  extensively  used  in  the  study 
of  technology-based  instructional  materials.  It  is  typ¬ 
ically  context  specific .  Although  there  are  similarities 
to  design  and  development  research  (especially  to 
product  and  tool  studies),  this  is  a  somewhat  different 
orientation  compared  to  the  techniques  discussed  here 
which  concentrate  on  the  design  and  development  pro¬ 
cedures  themselves. 

These  various  research  methods,  however,  all 
reflect  the  preponderance  of  scholars  turning  to 
research  designs  and  methodologies  that  rely  on  real- 
life  and  complex  design  and  instructional  environ¬ 
ments  rather  than  controlled  settings.  These  studies 
also  speak  to  the  needs  of  practitioners  as  well  as  the 
development  of  theory.  Ultimately,  the  expansion  of 
design  and  development  research  will  depend  more  on 
the  pressure  exerted  by  critical  disciplinary  and  work¬ 
place  problems  than  the  voices  of  advocates  of  this 
type  of  research. 

The  sources  of  problems  that  have  generated  our 
current  design  and  development  research— emerging 
technology,  the  workplace,  and  theory  — are  likely  to 
be  the  same  problem  areas  that  will  continue  to  stim¬ 
ulate  this  research  in  the  future,  probably  in  this  order 
of  magnitude.  Technology  is  a  fairly  straightforward 
impetus  for  design  and  development  research,  and  this 
research  is  often  prompted  by  clear-cut  practitioner 
problems.  Similarly,  workplace  problems  demand  res¬ 
olution  and  researchers  respond.  This  is  typically  not 
the  case  with  theory-related  research.  Ultimately,  the 
particular  interests  of  individual  scholars  generate  the¬ 
ory-based  research.  Future  research,  however,  is  likely 
to  address  non-instructional  as  well  as  instructional 
problems  and  products. 

Summary 

This  chapter  section  focused  on  design  and  develop¬ 
ment  research.  Major  concepts  and  points  addressed 
included  the  following: 

•  Design  and  development  research  seeks  to 
create  knowledge  grounded  in  data  systemat¬ 
ically  derived  from  practice.  It  provides  a  way 
to  test  theory,  validate  practice,  and  establish 
new  procedures,  techniques,  and  tools  based 
on  a  systematic  analysis  of  specific  cases. 
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•  Such  research  can  involve  a  situation  in 
which  someone  is  studying  the  design  and 
development  work  of  others  or  in  which 
someone  is  performing  design  and  develop¬ 
ment  activities  and  studying  the  process  at 
the  same  time.  In  either  case,  a  distinction 
is  made  between  doing  design  and  develop¬ 
ment  and  studying  the  processes. 

•  The  two  major  types  of  design  and  develop¬ 
ment  research  are  product  and  tool  research 
and  model  research.  Product  and  tool 
research  usually  involves  studies  that 
describe  and  analyze  the  design  and  devel¬ 
opment  processes  used  in  a  particular 
project;  conclusions  are  context  bound. 
Model  research  studies  are  oriented  toward 
a  general  analysis  of  design  and  develop¬ 
ment  processes  and  tend  to  be  more  gener- 
alizable  than  product  studies. 

•  The  entire  design  and  development  process 
is  often  documented  in  product  and  tool 
research;  however,  some  product  and  tool 
studies  concentrate  on  one  phase  of  design 
and  development  only  or  de-emphasize 
some  phases. 

•  Model  research  encompasses  studies  of 
model  development,  model  validation,  and 
model  use. 

•  The  most  common  sources  of  design  and 
development  research  problems  are  actual 
workplace  settings  and  projects,  new  tech¬ 
nology,  and  theoretical  questions  prompted 
by  the  literature. 

•  Although  design  and  development  research¬ 
ers  use  a  wide  variety  of  methods,  these  stud¬ 
ies  tend  to  rely  more  on  qualitative 
approaches  than  on  quantitative  approaches. 
Qualitative  data-gathering  techniques, 
including  case  studies,  interviews,  document 
reviews,  and  observations,  are  frequently 
used.  Evaluation  research  techniques  are 
also  included  in  many  studies  that  focus  on 
product  and  tool  development.  Model  devel¬ 
opment  and  use  studies  often  employ  survey 
research  techniques,  while  model  validation 
studies  often  use  traditional  experimental 
designs. 

•  Due  to  the  complexity  of  studying  design 
and  development  processes  in  actual  work 
settings,  many  studies  employ  mixed-meth¬ 
ods  research  that  combine  qualitative  and 
quantitative  methods  or  multiple  research 
methods  that  make  use  of  a  variety  of  similar 
approaches. 


•  Design  and  development  researcher  should 
address  the  following  methodological  issues 
when  conducting  their  studies:  avoiding  bias 
when  assuming  a  participant  role,  the  danger 
in  relying  on  recall  data,  and  the  constraints 
on  research  in  natural  work  environments. 
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ABSTRACT 

The  focus  of  this  chapter  is  on  methods  of  data  collec¬ 
tion  and  analysis  for  the  assessment  of  learning  pro¬ 
cesses  and  complex  performance,  the  last  part  of  the 
empirical  cycle  after  theory  development  and  experi¬ 
mental  design.  In  the  introduction  (van  Gog  and  Paas), 
the  general  background  and  the  relation  between  the 
chapter  sections  are  briefly  described.  The  section  by 
Savenye,  Robinson,  Niemczyk,  and  Atkinson  focuses 
on  methods  of  data  collection  and  analysis  for  assess¬ 
ment  of  individual  learning  processes,  whereas  the  sec¬ 
tion  by  Johnson  and  O’Connor  is  concerned  with  meth¬ 
ods  for  assessment  of  group  learning  processes.  The 
chapter  section  by  van  Gog,  Rikers,  and  Ayres  dis¬ 
cusses  the  assessment  of  complex  performance,  and 
the  final  chapter  section  by  Duley,  Ward,  Szalma,  and 
Hancock  is  concerned  with  setting  up  laboratories  to 
measure  learning  and  complex  performance. 


KEYWORDS 

Assessment  criteria:  Describe  the  aspects  of  perfor¬ 
mance  that  will  be  assessed. 

Assessment  of  learning:  Measuring  learning  achieve¬ 
ment,  performance,  outcomes,  and  processes  by 
many  means. 

Assessment  standards:  Describe  the  quality  of  perfor¬ 
mance  on  each  of  the  criteria  that  can  be  expected 
of  participants  at  different  stages  (e.g.,  age,  grade) 
based  on  a  participant’s  past  performance  (self- 
referenced),  peer  group  performance  (norm-refer¬ 
enced),  or  an  objective  standard  (criterion-refer¬ 
enced). 

Collective  data  collection:  Obtaining  data  from  indi¬ 
vidual  group  members;  data  are  later  aggregated  or 
manipulated  into  a  representation  of  the  group  as 
a  whole. 


Complex  performance:  Refers  to  real-world  activities 
that  require  the  integration  of  disparate  measure¬ 
ment  instrumentation  as  well  as  the  need  for  time- 
critical  experimental  control. 

Direct  process  measure:  Continuous  elicitation  of  data 
from  beginning  to  end  of  the  (group)  process;  direct 
process  measures  involve  videotaping,  audiotap- 
ing,  direct  researcher  observation,  or  a  combination 
of  these  methods. 

Group:  Two  or  more  individuals  working  together  to 
achieve  a  common  goal. 

Group  learning  process:  Actions  and  interactions  per¬ 
formed  by  group  members  during  the  group  learn¬ 
ing  task. 

Holistic  data  collection:  Obtaining  data  from  the  group 
as  a  whole;  as  this  type  of  data  collection  results 
in  a  representation  of  the  group  rather  than  indi¬ 
vidual  group  member,  it  is  not  necessary  to  aggre¬ 
gate  or  manipulate  data. 

Indirect  process  measure:  Discrete  measure  at  a  spe¬ 
cific  point  in  time  during  the  (group)  process;  often 
involves  multiple  points  of  data  collection;  indirect 
process  measures  may  measure  processes,  out¬ 
comes,  products,  or  other  factors  related  to  group 
process. 

Instrumentation:  Hardware  devices  used  to  assist  with 
the  process  of  data  acquisition  and  measurement. 

Mixed-methods  research:  Studies  that  rely  on  quanti¬ 
tative  and  qualitative  as  well  as  other  methods  for 
formulating  research  questions,  collecting  and  ana¬ 
lyzing  data,  and  interpreting  findings. 

Online! offline  measures:  Online  measures  are  recorded 
during  task  performance,  offline  measures  are 
recorded  after  task  performance. 

Process-tracing  techniques:  Records  performance  pro¬ 
cess  data  such  as  verbal  reports,  eye  movements, 
and  actions  that  can  be  used  to  make  inferences 
about  the  cognitive  processes  or  knowledge  under¬ 
lying  task  performance. 
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Qualitative  research:  Sometimes  called  naturalistic; 
research  on  human  systems  whose  hallmarks 
include  researcher  as  instrument,  natural  settings, 
and  little  manipulation. 

Quantitative  research:  Often  conceived  of  as  more 
traditional  or  positivistic;  typified  by  experimental 
or  correlational  studies.  Data  and  findings  are  usu¬ 
ally  represented  through  numbers  and  results  of 
statistical  tests. 

Task  complexity:  Can  be  defined  subjectively  (individ¬ 
ual  characteristics ,  such  as  expertise  or  perception) , 
objectively  (task  characteristics,  such  as  multiple 
solution  paths  or  goals),  or  as  an  interaction  (indi¬ 
vidual  and  task  characteristics). 

INTRODUCTION 

Tamara  van  Gog  and  Fred  Paas 

The  most  important  rule  concerning  data  collection  and 
analysis  is  do  not  attempt  to  collect  or  analyze  all  pos¬ 
sible  kinds  of  data.  Unless  you  are  conducting  a  truly 
explorative  study  (which  is  hardly  ever  necessary  now¬ 
adays,  considering  the  abundance  of  literature  on  most 
topics),  the  first  part  of  the  empirical  cycle— the  process 
of  theory  development— should  result  in  clear  research 
questions  or  hypotheses  that  will  allow  you  to  choose 
an  appropriate  design  to  study  these.  These  hypotheses 
should  also  indicate  the  kind  of  data  you  will  need  to 
collect— that  is,  the  data  you  have  hypotheses  about  and 
some  necessary  control  data  (e.g.,  time  on  task),  and 
together  with  the  design  provide  some  indications  as  to 
how  to  analyze  those  data  (e.g.,  2  x  2  factorial  design, 
2x2  MANCOVA).  But,  these  are  just  indications,  and 
many  decisions  remain  to  be  made.  To  name  just  a  few 
issues  regarding  data  collection  (for  an  elaboration  on 
those  questions,  see,  for  example,  Christensen,  2006; 
Sapsford  and  Jupp,  1996):  Which  participants 
(human/nonhuman,  age,  educational  background,  gen¬ 
der)  and  how  many  to  use?  What  and  how  many  tasks 
or  stimuli  to  present  and  on  what  apparatus?  What  (con¬ 
trol)  measures  to  take?  What  instructions  to  give?  What 
procedure  to  use?  When  to  schedule  the  sessions? 

Making  those  decisions  is  not  an  easy  task,  and 
unfortunately  strict  guidelines  cannot  be  given  because 
acceptable  answers  are  highly  dependent  on  the  exact 
nature,  background,  goals,  and  context  of  the  study. 
To  give  you  some  direction,  it  might  help  to  have  a 
look  at  how  these  questions  have  been  dealt  with  in 
high-quality  studies  in  your  domain  (which  are  gener¬ 
ally  published  in  peer-reviewed,  high-impact  journals). 
Because  of  the  importance  and  difficulty  of  finding 
correct  operationalizations  of  these  issues,  it  is  gener¬ 


ally  advisable  to  conduct  a  pilot  study  to  test  your  data 
collection  and  analysis  procedures. 

In  educational  research  many  studies  share  the 
common  goal  of  assessing  learning  or  performance, 
and  the  chapter  sections  in  this  chapter  provide  infor¬ 
mation  on  methods  for  collecting  and  analyzing  learn¬ 
ing  and  performance  data.  Even  though  learning  and 
performance  are  conceptually  different,  many  of  the 
data  collection  and  analysis  techniques  can  be  used  to 
assess  both;  therefore,  we  first  discuss  the  differences 
between  the  assessment  of  learning  and  the  assessment 
of  performance  before  giving  a  brief  overview  of  the 
content  of  the  chapter  sections. 

Assessment  of  Learning  vs.  Performance 

The  definitions  of  learning  and  performance  have  an 
important  similarity,  in  that  they  can  be  used  to  refer 
both  to  an  outcome  or  product  and  to  a  process.  The 
term  learning  is  used  to  refer  to  the  knowledge  or  skill 
acquired  through  instruction  or  study  (note  that  this 
dictionary  definition  ignores  the  possibility  of  informal 
learning,  unless  this  is  encompassed  by  study),  as  well 
as  the  process  of  acquiring  knowledge  or  skill  through 
instruction  or  study.  The  term  performance  is  used  to 
refer  to  things  accomplished  (outcome  or  product)  and 
to  the  accomplishment  of  things  (process).  Performance 
implies  the  use  of  knowledge  rather  than  merely  pos¬ 
sessing  it.  It  seems  that  performance  is  more  closely 
related  to  skill  than  to  knowledge  acquisition  (i.e.,  learn¬ 
ing),  but  an  important  difference  between  the  definitions 
of  learning  and  performance  is  that  performance  can  be, 
but  is  not  defined  as,  a  result  of  instruction  or  study. 

The  similarities  and  differences  between  these 
terms  have  some  important  implications  for  educa¬ 
tional  research.  First  of  all,  the  fact  that  both  learning 
and  performance  can  refer  to  a  product  and  a  process 
enables  the  use  of  many  different  kinds  of  measures 
or  combinations  of  measures  to  assess  learning  or  per¬ 
formance.  This  can  make  it  quite  difficult  to  compare 
results  of  different  studies  on  learning  or  performance, 
as  they  might  have  assessed  different  aspects  of  the 
same  concept  and  come  to  very  different  conclusions. 

Second,  collection  and  analysis  of  data  about  the 
knowledge  an  individual  possesses  can  be  used  to 
assess  their  learning  but  not  their  performance.  That 
possessing  knowledge  does  not  guarantee  the  ability  to 
use  it  has  been  shown  in  many  studies  (see,  for  exam¬ 
ple,  Ericsson  and  Lehmann,  1996).  Nonetheless,  for  a 
long  time,  educational  certification  practices  were 
based  on  this  assumption:  Students  received  their  diplo¬ 
mas  after  completing  a  series  of  courses  successfully, 
and  success  was  usually  measured  by  the  amount  of 
knowledge  a  student  possessed.  Given  that  this  measure 
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has  no  one-to-one  mapping  with  successful  perfor¬ 
mance,  this  practice  posed  many  problems,  both  for 
students  and  employers,  when  students  went  to  work 
after  their  educational  trajectory.  Hence,  in  the  field  of 
education  it  is  recognized  now  that  knowledge  is  a 
necessary  but  not  sufficient  condition  for  performance, 
and  the  field  is  gradually  making  a  shift  from  a  knowl¬ 
edge-based  testing  culture  to  a  performance-based 
assessment  culture  (Birenbaum  and  Dochy,  1996). 

Finally,  because  performance  is  not  defined  as  a 
result  of  instruction  or  study,  it  can  be  assessed  in  all 
kinds  of  situations,  and  when  applied  in  instructional 
or  study  settings  it  may  be  assessed  before,  during, 
and  after  instruction  or  study  phases.  Note  though,  that 
in  that  case,  only  the  difference  between  performance 
assessed  before  and  after  instruction  or  study  is  indic¬ 
ative  for  learning.  One  should  be  careful  not  to  inter¬ 
pret  gains  in  performance  during  instruction  or  study 
as  indicators  for  learning,  as  these  may  be  artifacts  of 
instructional  methods  (Bjork,  1999). 

Brief  Overview  of  the  Chapter  Sections 

The  first  chapter  section.  Assessment  of  Individual 
Learning  Processes  by  Savenye,  Robinson,  Niemczyk, 
and  Atkinson,  introduces  educational  technology 
researchers  to  the  conceptual  basis  and  methods  of  data 
collection  and  analysis  for  investigating  individual 
learning  processes.  They  discuss  the  quantitative  and 
qualitative  research  paradigms  and  the  associated 
approaches  to  data  collection  and  analysis.  They  also 
point  out  the  benefits  of  combining  quantitative  and 
qualitative  approaches  by  conducting  mixed-methods 
studies. 

The  second  chapter  section.  Assessment  of  Group 
Learning  Processes  by  Johnson  and  O’Connor,  focuses 
on  the  study  of  group  learning  processes,  which  is 
more  complex  than  the  study  of  individual  learning 
processes.  They  discuss  several  issues  that  need  to  be 
considered  prior  to  setting  up  a  study  of  group  learning 
processes,  such  as  holistic  vs.  collective  data  collec¬ 
tion,  direct  vs.  indirect  methods  of  data  collection, 
aggregation  or  manipulation  of  individual  data  into 
group  level  data,  and  special  considerations  for  setting 
up  a  study  of  group  learning  processes. 

The  third  chapter  section.  Assessment  of  Complex 
Performance  by  van  Gog,  Rikers,  and  Ayres,  discusses 
data  collection  and  analysis  methods  for  assessment 
of  complex  performance.  In  line  with  the  two-edged 
definition  of  performance  as  a  thing  accomplished  or 
accomplishing  a  thing,  they  distinguish  product  and 
process  measures  and  subdivide  the  process  measures 
further  into  online  (while  working  on  a  task)  vs.  offline 
(after  task  completion)  measures.  They  also  discuss 


the  opportunities  to  combine  several  different  mea¬ 
sures  and  the  benefits  of  doing  so. 

The  fourth  and  final  chapter  section.  Setting  Up  a 
Laboratory  for  Measurement  of  Complex  Perfor¬ 
mances  by  Duley,  Ward,  Szalma,  and  Hancock,  pro¬ 
vides  insight  into  the  technical  setup  of  laboratories 
for  the  assessment  of  learning  processes  and  complex 
performance.  Rather  than  providing  a  list  of  available 
hardware,  software,  and  instruments,  they  have  chosen 
to  take  the  more  sensible  approach  of  familiarizing  the 
reader  with  setting  up  configurations  for  stimulus  pre¬ 
sentation,  control  options,  and  response  recording, 
which  are  relevant  for  many  laboratory  studies. 

ASSESSMENT  OF  INDIVIDUAL 
LEARNING  PROCESSES 

Wilhelmina  Savenye,  Rhonda  Robinson, 

Mary  Niemczyk,  and  Robert  Atkinson 

It  is  the  goal  of  this  section  to  introduce  educational 
technology  researchers  to  the  conceptual  basis  and 
methods  of  data  collection  and  analysis  for  investigat¬ 
ing  individual  learning  processes,  including  both  quan¬ 
titative  and  qualitative  research  techniques.  Learning 
processes,  of  course,  may  involve  both  individual  and 
group  efforts  of  learners  in  the  form  of  strategies  and 
activities  designed  to  facilitate  their  learning.  Though 
this  section  focuses  on  individual  processes  and  per¬ 
formances,  using  a  variety  of  methods,  these  may  be 
adapted  for  group  use  (see  the  chapter  section  by 
Johnson  and  O’Connor). 

Several  assumptions  guide  this  work.  Although 
methods  can  be  suggested  here,  the  researcher  must  be 
responsible  for  understanding  the  foundational  ideas  of 
any  study.  He  or  she  will  want  to  conduct  the  study 
with  the  utmost  attention  to  quality  and  therefore  will 
want  to  turn  to  specific  and  detailed  texts  to  learn  more 
deeply  how  to  apply  research  methods.  This  section  will 
point  the  researcher  to  such  references  and  resources. 

The  objectives  of  this  section  are  listed  below.  It 
is  hoped  that  after  reading  this  chapter,  educational 
technology  researchers  will  be  able  to: 

•  Describe  methods  and  techniques  for  con¬ 
ducting  research  on  individual  learning,  and 
compare  qualitative  and  quantitative  methods. 

•  Describe  common  problems  in  conducting 
and  evaluating  quantitative  and  qualitative 
research  methods  to  examine  learning  pro¬ 
cesses. 

•  Consider  issues  that  contribute  to  the  quality 
of  studies  using  mixed  methods. 
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Rationale  for  Using  Mixed  Methods 

The  terms  quantitative  and  qualitative  are  commonly 
used  to  describe  contrasting  research  approaches.  Typ¬ 
ically,  quantitative  research  is  considered  to  be  more 
numbers  driven,  positivistic,  and  traditional  (Borg  and 
Gall,  1989),  while  qualitative  research  is  often  used 
interchangeably  with  terms  such  as  naturalistic,  ethno¬ 
graphic  (Goetz  and  LeCompte,  1984),  subjective,  or 
post-positivistic.  We  define  qualitative  research  in  this 
section  as  research  that  is  devoted  to  developing  an 
understanding  of  human  systems,  be  they  small,  such 
as  a  technology-using  teacher  and  his  or  her  students 
and  classroom,  or  large,  such  as  a  cultural  system. 
Quantitative  and  qualitative  methods  for  data  collection 
derive  in  some  measure  from  a  difference  in  the  way 
one  sees  the  world,  which  results  in  what  some  consider 
a  paradigm  debate;  however,  in  assessing  learning  pro¬ 
cesses,  both  approaches  to  data  collection  have  impor¬ 
tance,  and  using  elements  from  both  approaches  can  be 
very  helpful.  Driscoll  (1995)  suggested  that  educational 
technologists  select  research  paradigms  based  on  what 
they  perceive  to  be  the  most  critical  questions.  Robin¬ 
son  (1995)  and  Reigeluth  (1989)  concurred,  noting  the 
considerable  debate  within  the  field  regarding  suitable 
research  questions  and  methods.  Learning  processes 
are  complex  and  individual.  Lowyck  and  Elen  (2004) 
argued  that  learning  processes  are  active,  constructive, 
self-regulated,  goal  oriented,  and  contextualized.  In 
addition,  digital  technologies  are  changing  the  nature 
of  knowledge  and  of  teaching  and  learning  (Cornu, 
2004).  It  is  clear  then  that  the  methods  required  to 
collect  and  analyze  how  learning  processes  work,  when 
they  work,  and  why  they  work  can  be  drawn  from  a 
mixed-method  approach.  Thus,  researchers  can  inves¬ 
tigate  carefully  and  creatively  any  questions  they 
choose  and  derive  valid  data  to  help  understand  learn¬ 
ing  processes  using  a  combination  of  methods  from 
both  perspectives.  Although  not  the  main  focus  of  this 
chapter,  it  is  assumed  that  researchers  will  submit  all 
procedures,  protocols,  instruments,  and  participation 
forms  to  the  appropriate  human-subjects  or  ethics 
review  unit  within  their  organizations.  In  any  case, 
researchers  should  be  specific  about  how  they  define 
the  assumptions  of  the  study  and  why  what  was  done 
was  done— in  short,  they  should  be  able  to  enter  into 
the  current  and  upcoming  discussions  as  thoughtful, 
critical,  and  creative  researchers. 

Analyzing  Learning  Using 
Quantitative  Methods  and  Techniques 

Learning  achievement  or  performance  in  educational 
technology  research  is  often  the  primary  outcome  mea¬ 
sure  or  dependent  variable  of  concern  to  the  researcher. 


Learning  is  often  therefore  studied  using  more  quan¬ 
titative  measures,  including  what  researchers  may  call 
tests,  assessments,  examinations,  or  quizzes.  These 
measures  may  be  administered  in  paper-and-pencil 
form  or  may  be  technology  based.  If  they  are  technol¬ 
ogy  based,  they  may  be  administered  at  a  testing  cen¬ 
ter,  with  tutors  or  proctors,  or  completed  on  the  stu¬ 
dent’s  own.  In  either  format,  they  may  be  scored  by 
an  instructor  or  tutor  or  may  be  automatically  scored 
by  the  computer  (Savenye,  2004a,b).  Issues  of  concern 
in  selecting  and  developing  tests  and  test  items  also 
are  relevant  when  learning  is  measured  en  route  as 
performance  on  practice  items.  Completion  time,  often 
in  conjunction  with  testing,  is  another  learning  process 
variable  that  can  efficiently  be  examined  using  quan¬ 
titative  methods. 

Learning  achievement  on  complex  tasks  may  also 
be  measured  more  quantitatively  using  numerically 
based  rubrics  and  checklists  to  evaluate  products  and 
performances  or  to  evaluate  essays  or  learner-created 
portfolios.  Rubrics  and  checklists  are  also  often  used 
to  derive  quantitative  data  for  measuring  learning  in 
online  discussions  or  to  build  frequencies  of  behaviors 
from  observations  of  learning  processes,  often  used  in 
conjunction  with  more  qualitative  methods  (discussed 
later  in  this  section).  Many  computer-based  course 
management  systems  now  routinely  collect  course  sta¬ 
tistics  that  may  be  examined  to  determine  how  learners 
proceed  through  instruction  and  what  choices  they 
make  as  they  go.  Self-evaluations  and  other  aspects  of 
learning,  such  as  learner  attitudes,  are  more  commonly 
measured  using  questionnaires.  Selected  types  of 
quantitative  methods  for  examining  learning  are  dis¬ 
cussed  in  turn: 

•  Tests,  examinations,  quizzes  (administered 
via  paper  or  technology,  including  self-eval¬ 
uations) 

•  Rubrics  or  checklists  to  measure  learner  per¬ 
formance 

•  Measuring  learning  processes  in  technology- 
mediated  communications 

•  Technology-based  course  statistics 

•  Attitude  measures  such  as  questionnaires 
using  Likert-type  items 

Selecting  Tests 

Educational  researchers  frequently  select  existing  tests 
to  assess  how  individual  learning  processes  are 
impacted  by  a  novel  educational  intervention.  During 
this  process,  the  researchers  must  be  conversant  with 
a  number  of  important  concepts ,  including  validity  and 
reliability.  In  the  following  sections,  these  concepts  are 
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described  in  greater  detail  with  a  specific  focus  on  what 
researchers  need  to  know  when  selecting  tests. 

Validity 

Arguably,  the  most  critical  aspect  of  a  test  is  its  quality 
or  validity.  Simply  put,  a  test  is  considered  valid  if  it 
measures  what  it  was  created  to  measure  (Borg  and 
Gall,  1989).  A  test  is  generally  considered  valid  if  the 
scores  it  produces  help  individuals  administering  the 
test  make  accurate  inferences  about  a  particular  char¬ 
acteristic,  trait,  or  attribute  intrinsic  to  the  test  taker. 
As  an  example,  researchers  exploring  the  relative 
impact  of  several  learning  environments  would  con¬ 
sider  a  test  valid  to  the  extent  to  which  it  helps  them 
make  an  accurate  determination  of  the  relative  quality 
and  quantity  of  learning  displayed  by  the  students 
exposed  to  the  respective  learning  environments. 

Validity  is  not  a  unitary  concept;  in  fact,  test  devel¬ 
opers  use  several  widely  accepted  procedures  to  doc¬ 
ument  the  level  of  validity  of  their  test,  including  con¬ 
tent-related,  criterion-related,  and  construct-related 
validity.  Content-related  validity  represents  the  extent 
to  which  the  content  of  a  test  is  a  representative  sample 
of  the  total  subject  matter  content  provided  in  the 
learning  environment.  Another  type  of  validity  is  cri¬ 
terion-related  validity ,  which  depicts  how  closely 
scores  on  a  given  test  correspond  to  or  predict  perfor¬ 
mance  on  a  criterion  measure  that  exists  outside  the 
test.  Unlike  content  validity,  this  type  of  validity  yields 
a  numeric  value  that  is  the  correlation  coefficient 
reported  on  a  scale  of  -1  (perfect,  negative  relation¬ 
ship)  to  +1  (perfect,  positive  relationship).  The  third 
type  of  validity  is  construct-related  validity,  which 
refers  to  the  extent  to  which  the  scores  on  a  test  cor¬ 
respond  with  a  particular  construct  or  hypothetical 
concept  originating  from  a  theory. 

Also  worth  mentioning  is  a  relatively  unsophisti¬ 
cated  type  of  validity  known  as  face  validity,  which  is 
based  on  the  outward  appearance  of  the  test.  Although 
this  is  considered  a  rather  rudimentary  approach  to 
establishing  validity,  it  is  considered  important 
because  of  its  potential  impact  on  the  test  taker’s  moti¬ 
vation.  In  particular,  respondents  may  be  reluctant  to 
complete  a  test  without  any  apparent  face  validity. 

Reliability 

Another  important  concept  involved  in  test  selection 
is  reliability.  Simply  put,  reliability  refers  to  the  con¬ 
sistency  with  which  a  test  yields  the  same  results  for 
a  respondent  across  repeated  administrations  (Borg 
and  Gall,  1989).  Assuming  that  the  focus  of  the  test— 
a  particular  attribute  or  characteristic  — remains 


unchanged  between  test  administrations  for  a  given 
individual,  reliability  sheds  light  on  the  following 
question:  Does  the  test  always  yield  the  same  score 
for  an  individual  when  it  is  administered  on  several 
occasions? 

Determining  and  Judging  Reliability 

The  three  basic  approaches  to  determining  the  reliabil¬ 
ity  of  a  test  are  test-re  test,  alternate  forms,  and  internal 
consistency  (Borg  and  Gall,  1989;  Morris  et  al.,  1987). 
Perhaps  the  simplest  technique  for  estimating  reliabil¬ 
ity  is  the  test-retest  method.  With  this  approach,  a  test 
developer  simply  administers  the  test  twice  to  the  same 
group  of  respondents  and  then  calculates  the  correla¬ 
tion  between  the  two  sets  of  scores.  As  a  general  rule, 
researchers  select  tests  displaying  the  highest  reliabil¬ 
ity  coefficient  because  values  approaching  +1.00  are 
indicative  of  a  strong  relationship  between  the  two  sets 
of  respondents’  scores;  that  is,  the  respondents’  rela¬ 
tive  performance  has  remained  similar  across  the  two 
testing  occasions.  Specifically,  values  above  .80  are 
preferable  (Chase,  1999). 

Another  approach  to  determining  reliability  is  the 
alternate  forms  method ,  in  which  two  equivalent  forms 
of  a  test  are  administered  to  a  group  of  respondents  on 
two  separate  occasions  and  the  resulting  scores  corre¬ 
lated.  As  with  the  test-retest  method,  the  higher  the 
reliability  coefficient,  the  more  confidence  a  test 
administer  can  place  in  the  ability  of  a  test  to  consis¬ 
tently  measure  what  it  was  designed  to  measure. 

The  final  method  for  estimating  the  reliability  of  a 
test  is  referred  to  as  internal  consistency.  Unlike  the 
two  previous  methods,  it  does  not  rely  on  testing  the 
same  group  of  respondents  twice  to  estimate  the  inter¬ 
nal  consistency  of  a  test.  Instead,  the  reliability  of  a 
test  is  estimated  based  on  a  single  test  administration, 
which  can  be  accomplished  in  two  ways— either  using 
the  split  halves  method  or  using  one  of  the  Kuder- 
Richardson  methods,  which  do  not  require  splitting  a 
test  in  half. 

Limits  of  Reliability 

A  number  of  caveats  are  associated  with  reliability. 
First,  it  is  important  to  recognize  that  high  reliability 
does  not  guarantee  validity;  in  other  words,  a  test  can 
consistently  measure  what  it  was  intended  to  measure 
while  still  lacking  validity.  Knowing  that  a  test  is  reli¬ 
able  does  not  permit  someone  to  make  judgments  about 
its  validity.  Reliability  is,  however,  necessary  for  valid¬ 
ity,  as  it  impacts  the  accuracy  with  which  one  can  draw 
inferences  about  a  particular  characteristic  or  attribute 
intrinsic  to  the  test  taker.  The  reliability  is  impacted  by 
several  factors.  Test  length  is  the  first.  All  things  being 
equal,  shorter  tests  tend  to  be  less  reliable  than  longer 
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tests  because  the  latter  afford  the  test  developer  more 
opportunities  to  accurately  measure  the  trait  or  charac¬ 
teristic  under  examination.  The  reliability  of  a  test  is 
also  impacted  by  the  format  of  its  items.  A  general 
heuristic  to  remember  is  that  tests  constructed  with 
select-type  items  tend  to  be  more  reliable  than  tests 
with  supply- type  or  other  subjectively  scored  items. 

Evaluating  and  Developing  Tests  and  Test  Items 

The  construction  of  learning  assessments  is  one  of  the 
most  important  responsibilities  of  instructors  and 
researchers.  Tests  should  be  comprised  of  items  that 
represent  important  and  clearly  stated  objectives  and 
that  adequately  sample  subject  matter  from  all  of  the 
learning  objectives.  The  most  effective  way  to  ensure 
adequate  representation  of  items  across  content,  cog¬ 
nitive  processes,  and  objectives  is  to  develop  a  test 
blueprint  or  table  of  specifications  (Sax,  1980).  Mul¬ 
tiple  types  of  performance  measures  allow  students  an 
opportunity  to  demonstrate  their  particular  skills  in 
defined  areas  and  to  receive  varied  feedback  on  their 
performances;  this  is  particularly  important  in  self- 
instructional  settings,  such  as  online  courses  (Savenye, 
2004a,b).  Multiple  learning  measures  in  online  settings 
also  offer  security  advantages  (Ko  and  Rossen,  2001). 

Tests  should  also  give  students  the  opportunity  to 
respond  to  different  types  of  item  formats  that  assess 
different  levels  of  cognition,  such  as  comprehension, 
application,  analysis,  and  synthesis  (Popham,  1991). 
Different  approaches  and  formats  can  yield  different 
diagnostic  information  to  instructors,  as  well;  for 
example,  well-developed  multiple-choice  items  con¬ 
tain  alternatives  that  represent  common  student  mis¬ 
conceptions  or  errors.  Short-answer  item  responses  can 
give  the  instructor  information  about  the  student’s 
thinking  underlying  the  answer  (Mehrens  et  al.,  1998). 

Because  the  test  item  is  the  essential  building  block 
of  any  test,  it  is  critical  to  determine  the  validity  of  the 
test  item  before  determining  the  validity  of  the  test 
itself.  Commercial  test  publishers  typically  conduct 
pilot  studies  (called  item  tryouts )  to  get  empirical  evi¬ 
dence  concerning  item  quality.  For  these  tryouts,  sev¬ 
eral  forms  of  the  test  are  prepared  with  different  subsets 
of  items,  so  each  item  appears  with  every  other  item. 
Each  form  may  be  given  to  several  hundred  examinees. 
Item  analysis  data  are  then  calculated,  followed  by 
assessment  of  the  performance  characteristics  of  the 
items,  such  as  item  difficulty  and  item  discrimination 
(i.e.,  how  well  the  item  separates,  or  discriminates, 
between  those  who  do  well  on  the  test  and  those  who 
do  poorly).  The  developers  discard  items  that  fail  to 
display  proper  statistical  properties  (Downing  and  Hal- 
adyna,  1997;  Nitko,  2001;  Thorndike,  1997). 


Scores  on  Numerically  Based  Rubrics  and  Checklists 

Assessing  performance  can  be  done  by  utilizing 
numerically  based  rubrics  and  checklists.  Typically, 
two  aspects  of  a  learner’s  performance  can  be  assessed: 
the  product  the  learner  produces  and  the  process  a 
learner  uses  to  complete  the  product.  Either  or  both  of 
these  elements  may  be  evaluated.  Because  perfor¬ 
mance  tasks  are  usually  complex,  each  task  provides 
an  opportunity  to  assess  students  on  several  learning 
goals  (Nitko,  2001).  Performance  criteria  are  the  spe¬ 
cific  behaviors  a  student  should  perform  to  properly 
carry  out  a  performance  or  produce  a  product.  The  key 
to  identifying  performance  criteria  is  to  break  down 
the  overall  performance  or  product  into  its  component 
parts.  It  is  important  that  performance  criteria  be  spe¬ 
cific,  observable,  and  clearly  stated  (Airasian,  1996). 

Scoring  rubrics  are  brief,  written  descriptions  of 
different  levels  of  performance.  They  can  be  used  to 
summarize  both  performances  and  products.  Scoring 
rubrics  summarize  performance  in  a  general  way, 
whereas  checklists  and  rating  scales  can  provide  spe¬ 
cific  diagnostic  information  about  student  strengths 
and  weaknesses  (Airasian,  1996).  Checklists  usually 
contain  lists  of  behaviors,  traits,  or  characteristics  that 
are  either  present  or  absent,  to  be  checked  off  by  an 
observer  (Sax,  1980).  Although  they  are  similar  to 
checklists,  rating  scales  allow  the  observer  to  judge 
performance  along  a  continuum  rather  than  as  a  dichot¬ 
omy  (Airasian,  1996). 

Measuring  Learning  Processes  in 
Technology-Mediated  Communications 

Tiffin  and  Rajasingham  (1995)  suggested  that  educa¬ 
tion  is  based  on  communication.  Online  technologies, 
therefore,  provide  tremendous  opportunities  for  learn¬ 
ing  and  allow  us  to  measure  learning  in  new  ways;  for 
example,  interactions  in  online  discussions  within 
Internet -based  courses  may  be  used  to  assess  students’ 
learning  processes.  Paulsen  (2003)  delineated  many 
types  of  learning  activities,  including  online  inter¬ 
views,  online  interest  groups,  role  plays,  brainstorm¬ 
ing,  and  project  groups.  These  activities,  generally 
involving  digital  records,  will  also  yield  online  com¬ 
munication  data  for  research  purposes,  provided  the 
appropriate  ethics  and  subject  guidelines  have  been 
followed. 

The  postings  learners  create  and  share  may  also  be 
evaluated  using  the  types  of  rubrics  and  checklists 
discussed  earlier.  These  are  of  particular  value  to  learn¬ 
ers  when  they  receive  the  assessment  tools  early  in  the 
course  and  use  them  to  self-evaluate  or  to  conduct  peer 
evaluations  to  improve  the  quality  of  their  work 
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(Savenye,  2006,  2007).  Goodyear  (2000)  reminded  us 
that  digital  technologies  add  to  the  research  and  devel¬ 
opment  enterprise  the  capability  for  multimedia  com¬ 
munications. 

Another  aspect  of  online  discussions  of  value  to 
researchers  is  that  the  types  of  postings  students  make 
and  the  ideas  they  discuss  can  be  quantified  to  illumi¬ 
nate  students’  learning  processes.  Chen  (2005),  in  an 
online  course  activity  conducted  with  groups  of  six 
students  who  did  not  know  each  other,  found  that 
learners  under  a  less-structured  forum  condition  posted 
many  more  socially  oriented  postings,  although  their 
performance  on  the  task  was  not  less  than  that  of  the 
students  who  did  not  post  as  many  social  postings.  She 
also  found  that  the  more  interactions  a  group  made, 
the  more  positive  students’  attitudes  were  toward  the 
course. 

Using  Technology-Based  Course  Statistics 
To  Examine  Learning  Processes 

In  addition  to  recording  learners’  performance  on  quiz¬ 
zes,  tests,  and  other  assignments,  most  online  course 
management  systems  automatically  collect  numerous 
types  of  data,  which  may  be  used  to  investigate  learn¬ 
ing  processes.  Such  data  may  include  information 
about  exactly  which  components  of  the  course  a 
learner  has  completed,  on  which  days,  and  for  how 
much  time.  Compilations  of  these  data  can  indicate 
patterns  of  use  of  course  components  and  features 
(Savenye,  2004a). 

Measuring  Attitudes  Using  Questionnaires 
That  Use  Likert-Type  Items 

Several  techniques  have  been  used  to  assess  attitudes 
and  feelings  of  learners  in  research  studies  and  as  part 
of  instruction.  Of  these  methods,  Likert-type  scales  are 
the  most  common.  Typically,  respondents  are  asked  to 
indicate  their  strength  of  feeling  toward  a  series  of 
statements,  often  in  terms  of  the  degree  to  which  they 
agree  or  disagree  with  the  position  being  described. 
Previous  research  has  found  that  responding  to  a  Lik¬ 
ert-type  item  is  an  easier  task  and  provides  more  infor¬ 
mation  than  ranking  and  paired  comparisons.  The 
advantage  of  a  Likert-type  item  scale  is  that  an  absolute 
level  of  an  individual’s  responses  can  be  obtained  to 
determine  the  strength  of  the  attitude  (O’Neal  and 
Chissom,  1993). 

Thorndike  (1997)  suggested  several  factors  to  con¬ 
sider  in  developing  a  Likert-type  scale,  including  the 
number  of  steps,  odd  or  even  number  of  steps,  and 
types  of  anchors.  The  number  of  steps  in  the  scale  is 
important  as  it  relates  to  reliability  — the  more  steps. 


the  greater  the  reliability.  The  increase  is  noticeable 
up  to  about  seven  steps;  after  this,  the  reliability  begins 
to  diminish,  as  it  becomes  difficult  to  develop  mean¬ 
ingful  anchors.  Five-point  scales  tend  to  be  the  most 
common.  Increasing  the  number  of  items  can  also 
increase  reliability.  Although  there  is  considerable 
debate  about  this,  many  researchers  hold  that  better 
results  can  be  obtained  by  using  an  odd  number  of 
steps,  which  provides  for  a  neutral  response.  The 
anchors  used  should  fit  the  meaning  of  the  statements 
and  the  goal  of  the  measurement.  Common  examples 
include  continua  such  as  agree-disagree,  effective- 
ineffective,  important-unimportant,  and  like  me-not 
like  me. 

Analyzing  Learning  Using  More 
Qualitative  Methods  and  Techniques 

Although  learning  outcomes  and  processes  can  be  pro¬ 
ductively  examined  using  the  quantitative  methods  dis¬ 
cussed  earlier,  in  a  mixed-methods  approach  many 
qualitative  methods  are  used  to  build  a  deeper  under¬ 
standing  of  what,  why,  and  how  learners  learn.  With 
the  increasing  use  of  interactive  and  distance  technol¬ 
ogies  in  education  and  industry,  opportunities  and  at 
times  the  responsibility  to  explore  new  questions  about 
the  processes  of  learning  and  instruction  have  evolved. 
New  technologies  also  enable  researchers  to  study 
learners  and  learning  processes  in  new  ways  and  to 
expand  our  views  of  what  we  should  investigate  and 
how;  for  example,  a  qualitative  view  of  how  instructors 
and  their  students  learn  through  a  new  technology  may 
yield  a  view  of  what  is  really  happening  when  the 
technology  is  used. 

As  in  any  research  project,  the  actual  research 
questions  guide  the  selection  of  appropriate  methods 
of  data  collection.  Once  a  question  or  issue  has  been 
selected,  the  choice  of  qualitative  methods  falls 
roughly  into  the  categories  of  observations,  interviews, 
and  document  and  artifact  analysis,  although  others 
have  conceptualized  the  methods  somewhat  differently 
(Bogdan  and  Biklen,  1992;  Goetz  and  LeCompte, 
1984;  Lincoln  and  Guba,  1985).  Qualitative  research¬ 
ers  have  basically  agreed  that  the  human  investigator 
is  the  primary  research  instrument  (Pelto  and  Pelto, 
1978). 

In  this  section,  we  begin  with  one  approach  to 
conducting  qualitative  research:  grounded  theory.  We 
then  discuss  specific  methods  that  may  be  called  obser¬ 
vations,  interviews,  and  document  and  artifact  analy¬ 
sis.  As  in  all  qualitative  research,  it  is  also  assumed 
that  educational  technology  researchers  will  use  and 
refine  methods  with  the  view  that  these  methods  vary 
in  their  degree  of  interactiveness  with  participants.  The 
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following  qualitative  methods,  along  with  several 
research  perspectives,  are  examined  next: 

•  Grounded  theory 

•  Participant  observations 

•  Nonparticipant  observations 

•  Interviews,  including  group  and  individual 

•  Document,  artifact,  and  online  communica¬ 
tions  and  activities  analysis 

Grounded  Theory 

In  their  overview  of  grounded  theory,  Strauss  and 
Corbin  (1994,  p.  273)  noted  that  it  is  “a  general  meth¬ 
odology  for  developing  theory  that  is  grounded  in  data 
systematically  gathered  and  analyzed,”  adding  that  it 
is  sometimes  referred  to  as  the  constant  comparative 
method  and  that  it  is  applicable  as  well  to  quantitative 
research.  In  grounded  theory,  the  data  may  come  from 
observations,  interviews,  and  video  or  document  anal¬ 
ysis,  and,  as  in  other  qualitative  research,  these  data 
may  be  considered  strictly  qualitative  or  may  be  quan¬ 
titative.  The  purpose  of  the  methodology  is  to  develop 
theory,  through  an  iterative  process  of  data  analysis 
and  theoretical  analysis,  with  verification  of  hypothe¬ 
ses  ongoing  throughout  the  study.  The  researcher 
begins  a  study  without  completely  preconceived 
notions  about  what  the  research  questions  should  be 
and  collects  and  analyzes  extensive  data  with  an  open 
mind.  As  the  study  progresses,  he  or  she  continually 
examines  the  data  for  patterns,  and  the  patterns  lead 
the  researcher  to  build  the  theory.  The  researcher  con¬ 
tinues  collecting  and  examining  data  until  the  patterns 
continue  to  repeat  and  few  new  patterns  emerge.  The 
researcher  builds  the  theory  from  the  data,  and  the 
theory  is  thus  built  on,  or  grounded  in,  the  phenomena. 

Participant  Observation 

In  participant  observation,  the  observer  becomes  part 
of  the  environment,  or  the  cultural  context.  The  hall¬ 
mark  of  participant  observation  is  continual  interaction 
between  the  researcher  and  the  participants;  for  exam¬ 
ple,  the  study  may  involve  periodic  interviews  inter¬ 
spersed  with  observations  so  the  researcher  can  ques¬ 
tion  the  participants  and  verify  perceptions  and 
patterns.  Results  of  these  interviews  may  then  deter¬ 
mine  what  will  initially  be  recorded  during  observa¬ 
tions.  Later,  after  patterns  begin  to  appear  in  the  obser¬ 
vational  data,  the  researcher  may  conduct  interviews 
asking  the  participants  about  these  patterns  and  why 
they  think  they  are  occurring. 

As  the  researcher  cannot  observe  and  record  every¬ 
thing,  in  most  educational  research  studies  the  inves¬ 


tigator  determines  ahead  of  time  what  will  be  observed 
and  recorded,  guided  but  not  limited  by  the  research 
questions.  Participant  observation  is  often  successfully 
used  to  describe  what  is  happening  in  a  context  and 
why  it  happens.  These  are  questions  that  cannot  be 
answered  in  the  standard  experiment. 

Many  researchers  have  utilized  participant  obser¬ 
vation  methods  to  examine  learning  processes.  Robin¬ 
son  (1994)  observed  classes  using  Channel  One  in  a 
Midwestern  middle  school;  she  focused  her  observa¬ 
tions  on  the  use  of  the  televised  news  show  and  the 
reaction  to  it  from  students,  teachers,  administrators, 
and  parents.  Reilly  (1994)  analyzed  video  recordings 
of  both  the  researcher  and  students  in  a  project  that 
involved  defining  a  new  type  of  literacy  that  combined 
print,  video,  and  computer  technologies.  Higgins  and 
Rice  (1991)  investigated  teachers’  perceptions  of  test¬ 
ing.  They  used  triangulation  and  a  variety  of  methods 
to  collect  data;  however,  a  key  feature  of  the  study  was 
participant  observation.  Researchers  observed  6  teach¬ 
ers  for  a  sample  of  10  hours  each  and  recorded 
instances  of  classroom  behaviors  that  could  be  classi¬ 
fied  as  assessment.  Similarly,  Moallem  (1994)  used 
multiple  methods  to  build  an  experienced  teacher’s 
model  of  teaching  and  thinking  by  conducting  a  series 
of  observations  and  interviews  over  a  7-month  period. 

Nonparticipant  Observation 

Nonparticipant  observation  is  one  of  several  methods 
for  collecting  data  considered  to  be  relatively  unobtru¬ 
sive.  Many  recent  authors  cite  the  early  work  of  Webb 
et  al.  (1966)  as  laying  the  groundwork  for  use  of  all 
types  of  unobtrusive  measures.  Several  types  of  non¬ 
participant  observation  have  been  identified  by  Goetz 
and  LeCompte  (1984).  These  include  stream-of-behav- 
ior  chronicles  recorded  in  written  narratives  or  using 
video  or  audio  recordings,  pro xemics  and  kinesics  (i.e., 
the  study  of  uses  of  social  space  and  movement),  and 
interaction  analysis  protocols,  typically  in  the  form  of 
observations  of  particular  types  of  behaviors  that  are 
categorized  and  coded  for  analysis  of  patterns.  In  non¬ 
participant  observation,  observers  do  not  interact  to  a 
great  degree  with  those  they  are  observing.  The 
researchers  primarily  observe  and  record,  using  obser¬ 
vational  forms  developed  for  the  study  or  in  the  form 
of  extensive  field  notes;  they  have  no  specific  roles  as 
participants. 

Examples  of  studies  in  which  observations  were 
conducted  that  could  be  considered  relatively  nonpar¬ 
ticipant  observation  include  Savenye  and  Strand  (1989) 
in  the  initial  pilot  test  and  Savenye  (1989)  in  the  sub¬ 
sequent  larger  field  test  of  a  multimedia-based  science 
curriculum.  Of  most  concern  during  implementation 
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was  how  teachers  used  the  curriculum.  A  careful  sam¬ 
ple  of  classroom  lessons  was  recorded  using  video, 
and  the  data  were  coded;  for  example,  teacher  ques¬ 
tions  were  coded,  and  the  results  indicated  that  teach¬ 
ers  typically  used  the  system  pauses  to  ask  recall-level 
rather  than  higher-level  questions.  Analysis  of  the 
coded  behaviors  for  what  teachers  added  indicated  that 
most  of  the  teachers  in  the  sample  added  examples  to 
the  lessons  that  would  provide  relevance  for  their  own 
learners.  Of  particular  value  to  the  developers  was  the 
finding  that  teachers  had  a  great  degree  of  freedom  in 
using  the  curriculum  and  the  students’  learning 
achievement  was  still  high. 

In  a  mixed-methods  study,  nonparticipant  observa¬ 
tions  may  be  used  along  with  more  quantitative  meth¬ 
ods  to  answer  focused  research  questions  about  what 
learners  do  while  learning.  In  a  mixed-methods  study 
investigating  the  effects  and  use  of  multimedia  learn¬ 
ing  materials,  the  researchers  collected  learning  out¬ 
come  data  using  periodic  tests.  They  also  observed 
learners  as  they  worked  together.  These  observations 
were  video  recorded  and  the  records  analyzed  to  exam¬ 
ine  many  learning  processes,  including  students’  level 
of  cognitive  processing,  exploratory  talk,  and  collab¬ 
orative  processing  (Olkinuora  et  al.,  2004).  Research¬ 
ers  may  also  be  interested  in  using  observations  to 
study  what  types  of  choices  learners  make  while  they 
proceed  through  a  lesson.  Klein  and  colleagues,  for 
instance,  developed  an  observational  instrument  used 
to  examine  cooperative  learning  behaviors  in  technol¬ 
ogy-based  lessons  (Crooks  et  al.,  1995;  Jones  et  al., 
1995;  Klein  and  Pridemore,  1994). 

A  variation  on  nonparticipant  observations  repre¬ 
sents  a  blend  with  trace-behavior,  artifact,  or  document 
analysis.  This  technique,  known  as  read-think-aloud 
protocols,  asks  learners  to  describe  what  they  do  and 
why  they  do  it  (i.e.,  their  thoughts  about  their  pro¬ 
cesses)  as  they  proceed  through  an  activity,  such  as  a 
lesson.  Smith  and  Wedman  (1988)  used  this  technique 
to  analyze  learner  tracking  and  choices.  Techniques 
for  coding  are  described  by  Spradley  (1980);  however, 
protocol  analysis  (Ericsson  and  Simon,  1984)  tech¬ 
niques  could  be  used  on  the  resulting  verbal  data. 

Issues  Related  to  Conducting  Observations 

Savenye  and  Robinson  (2004,  2005)  have  suggested 
several  issues  that  are  critical  to  using  observations  to 
studying  learning.  These  issues  include  those  related 
to  scope,  biases  and  the  observer’s  role,  sampling,  and 
the  use  of  multiple  observers.  They  caution  that  a 
researcher  can  become  lost  in  the  multitudes  of  obser¬ 
vational  data  that  can  be  collected,  both  in  person  and 
when  using  audio  or  video.  They  recommend  limiting 


the  scope  of  the  study  specifically  to  answering  the 
questions  at  hand.  Observers  must  be  careful  not  to 
influence  the  results  of  the  study;  that  is,  they  must  not 
make  things  happen  that  they  want  to  happen.  Potential 
bias  may  be  handled  by  simply  describing  the 
researcher’s  role  in  the  research  report,  but  investiga¬ 
tors  will  want  to  examine  periodically  what  their  role 
is  and  what  type  of  influences  may  result  from  it.  In 
observational  research,  sampling  becomes  not  random 
but  purposive  (Borg  and  Gall,  1989).  For  the  study  to 
be  valid,  the  reader  should  be  able  to  believe  that  a 
representative  sample  of  involved  individuals  was 
observed.  The  multiple  realities  of  any  cultural  context 
should  be  represented.  If  several  observers  will  be  used 
to  collect  the  data,  and  their  data  will  be  compared  or 
aggregated,  problems  with  reliability  of  data  may 
occur.  Observers  tend  to  see  and  subsequently  interpret 
the  same  phenomena  in  many  different  ways.  It 
becomes  necessary  to  train  the  observers  and  to  ensure 
that  observers  are  recording  the  same  phenomena  in 
the  same  ways.  When  multiple  observers  are  used  and 
behaviors  counted  or  categorized  and  tallied,  it  is  desir¬ 
able  to  calculate  and  report  inter-rater  reliability. 

Interviews 

In  contrast  with  the  relatively  non-interactive,  nonpar¬ 
ticipant  observation  methods  described  earlier,  inter¬ 
views  represent  a  classic  qualitative  research  method 
that  is  directly  interactive.  Interviews  may  be  struc¬ 
tured  or  unstructured  and  may  be  conducted  in  groups 
or  individually.  In  an  information  and  communication 
technologies  (ICT)  study  to  investigate  how  ICT  can 
be  introduced  into  the  context  of  a  traditional  school, 
Demetriadis  et  al.  (2005)  conducted  a  series  of  semi- 
structured  interviews  over  2  years  with  15  teachers/ 
mentors  who  offered  technology  training  to  other 
teachers. 

The  cornerstone  for  conducting  good  interviews  is 
to  be  sure  one  truly  listens  to  respondents  and  records 
what  they  say  rather  than  the  researcher’s  perceptions 
or  interpretations.  This  is  a  good  rule  of  thumb  in 
qualitative  research  in  general.  It  is  best  to  maintain 
the  integrity  of  the  raw  data  and  to  make  liberal  use 
of  the  respondents’  own  words,  including  quotes.  Most 
researchers,  as  a  study  progresses,  also  maintain  field 
notes  that  contain  interpretations  of  patterns  to  be 
refined  and  investigated  on  an  ongoing  basis. 

Many  old,  adapted,  and  exciting  techniques  for 
structured  interviewing  are  evolving.  One  example  of 
such  a  use  of  interviews  is  in  the  Higgins  and  Rice 
(1991)  study  mentioned  earlier.  In  this  study,  teachers 
sorted  the  types  of  assessment  they  had  named  previ¬ 
ously  in  interviews  into  sets  of  assessments  that  were 
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most  alike;  subsequently,  multidimensional  scaling 
was  used  to  analyze  these  data,  yielding  a  picture  of 
how  these  teachers’  viewed  testing.  Another  type  of 
structured  interview,  mentioned  by  Goetz  and 
LeCompte  (1984),  is  the  interview  using  projective 
techniques.  Photographs,  drawings,  and  other  visuals 
or  objects  may  be  used  to  elicit  individuals’  opinions 
or  feelings. 

Instructional  planning  and  design  processes  have 
long  been  of  interest  to  educational  technology 
researchers;  for  example,  using  a  case-study  approach. 
Reiser  and  Mory  (1991)  employed  interviews  to  exam¬ 
ine  two  teachers’  instructional  design  and  planning 
techniques.  One  of  the  models  proposed  for  the  design 
of  complex  learning  is  that  of  van  Merrienboer  et  al. 
(1992),  who  developed  the  four-component  model, 
which  subsequently  was  further  developed  as  the 
4C/ID  model  (van  Merrienboer,  1997).  Such  design 
models  have  been  effectively  studied  using  mixed 
methods,  including  interviews,  particularly  when  those 
processes  relate  to  complex  learning.  How  expert 
designers  go  about  complex  design  tasks  has  been 
investigated  using  both  interviews  and  examination  of 
the  designers  products  (Kirschner  et  al.,  2002). 

Problem-based  instructional  design,  blending 
many  aspects  of  curriculum,  instruction,  and  media 
options  (Dijkstra,  2004),  could  also  be  productively 
studied  using  interviews.  Interviews  to  examine  learn¬ 
ing  processes  may  be  conducted  individually  or  in 
groups.  A  specialized  group  interview  method  is  the 
focus  group  (Morgan,  1996),  which  is  typically  con¬ 
ducted  with  relatively  similar  participants  using  a 
structured  or  semi- structured  protocol  to  examine 
overall  patterns  in  learning  behaviors,  attitudes,  or 
interests. 

Suggestions  for  heightening  the  quality  of  inter¬ 
views  include  employing  careful  listening  and  record¬ 
ing  techniques;  taking  care  to  ask  probing  questions 
when  needed;  keeping  the  data  in  their  original  form, 
even  after  they  have  been  analyzed;  being  respectful 
of  participants;  and  debriefing  participants  after  the 
interviews  (Savenye  and  Robinson,  2005). 

Document,  Artifact,  and  Online 
Communications  and  Activities  Analysis 

Beyond  nonparticipant  observation,  many  unobtrusive 
methods  exist  for  collecting  information  about  human 
behaviors.  These  fall  roughly  into  the  categories  of 
document  and  artifact  analyses  but  overlap  with  other 
methods;  for  example,  verbal  or  nonverbal  behavior 
streams  produced  during  video  observations  may  be 
subjected  to  intense  microanalysis  to  answer  an  almost 
unlimited  number  of  research  questions.  Content  anal¬ 


ysis,  as  one  example,  may  be  done  on  these  narratives. 
In  the  Moallem  (1994),  Higgins  and  Rice  (1991),  and 
Reiser  and  Mory  (1991)  studies  of  teachers’  planning, 
thinking,  behaviors,  and  conceptions  of  testing,  docu¬ 
ments  developed  by  the  teachers,  such  as  instructional 
plans  and  actual  tests,  were  collected  and  analyzed. 

Goetz  and  LeCompte  (1984)  defined  artifacts  of 
interest  to  researchers  as  things  that  people  make  and 
do.  The  artifacts  of  interest  to  educational  technolo¬ 
gists  are  often  written,  but  computer  and  online  trails 
of  behavior  are  the  objects  of  analysis  as  well.  Exam¬ 
ples  of  artifacts  that  may  help  to  illuminate  research 
questions  include  textbooks  and  other  instructional 
materials,  such  as  media  materials;  memos,  letters,  and 
now  e-mail  records,  as  well  as  logs  of  meetings  and 
activities;  demographic  information,  such  as  enroll¬ 
ment,  attendance,  and  detailed  information  about  par¬ 
ticipants;  and  personal  logs  participants  may  keep. 

In  studies  in  educational  technology,  researchers 
often  analyze  the  patterns  of  learner  pathways,  deci¬ 
sions,  and  choices  they  make  as  they  proceed  through 
computer-based  lessons  (Savenye  et  al.,  1996;  Shin  et 
al.,  1994).  Content  analysis  of  prose  in  any  form  may 
also  be  considered  to  fall  into  this  artifact-and-docu- 
ment  category  of  qualitative  methodology.  Lawless 
(1994)  used  concept  maps  developed  by  students  in 
the  Open  University  to  check  for  student  understand¬ 
ing.  Entries  in  students’  journals  were  analyzed  by 
Perez-Prado  and  Thirunarayanan  (2002)  to  leam  about 
students’  perceptions  of  online  and  on-ground  versions 
of  the  same  college  course.  Espey  (2000)  studied  the 
content  of  a  school  district  technology  plan. 

Methods  for  Analyzing  Qualitative  Data 

One  of  the  major  hallmarks  of  conducting  qualitative 
research  is  that  data  are  analyzed  continually,  through¬ 
out  the  study,  from  conceptualization  through  the 
entire  data  collection  phase  and  into  the  interpretation 
and  writing  phases.  In  fact,  Goetz  and  LeCompte 
(1984)  described  the  processes  of  analyzing  and  writ¬ 
ing  together  in  what  they  called  analysis  and  interpre¬ 
tation. 

Data  Reduction 

Goetz  and  LeCompte  (1994)  described  the  conceptual 
basis  for  reducing  and  condensing  data  in  an  ongoing 
style  as  the  study  progresses.  Researchers  theorize  as 
the  study  begins  and  build  and  continually  test  theories 
based  on  observed  patterns  in  data.  Goetz  and 
LeCompte  described  the  analytic  procedures  research¬ 
ers  use  to  determine  what  the  data  mean.  These  pro¬ 
cedures  involve  looking  for  patterns,  links,  and  rela¬ 
tionships.  In  contrast  to  experimental  research,  the 
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qualitative  researcher  engages  in  speculation  while 
looking  for  meaning  in  data;  this  speculation  will  lead 
the  researcher  to  make  new  observations,  conduct  new 
interviews,  and  look  more  deeply  for  new  patterns  in 
this  recursive  process.  It  is  advisable  to  collect  data  in 
its  raw,  detailed  form  and  then  record  patterns.  This 
enables  the  researcher  later  to  analyze  the  original  data 
in  different  ways,  perhaps  to  answer  deeper  questions 
than  originally  conceived.  It  should  be  noted  that  vir¬ 
tually  all  researchers  who  use  an  ethnographic 
approach  advocate  writing  up  field  notes  immediately 
after  leaving  the  research  site  each  day.  If  researchers 
have  collected  documents  from  participants,  such  as 
logs,  journals,  diaries,  memos,  and  letters,  these  can 
also  be  analyzed  as  raw  data.  Similarly,  official  docu¬ 
ments  of  an  organization  can  be  subjected  to  analysis. 
Collecting  data  in  the  form  of  photographs,  films,  and 
videos,  either  produced  by  participants  or  the 
researcher,  has  a  long  tradition  in  anthropology  and 
education.  These  data,  too,  can  be  analyzed  for  mean¬ 
ing.  (Bellman  and  Jules-Rosette,  1977;  Bogaart  and 
Ketelaar,  1983;  Bogdan  and  Biklen,  1992;  Collier  and 
Collier,  1986;  Heider,  1976;  Hockings,  1975). 

Coding  Data 

Early  in  the  study,  the  researcher  will  begin  to  scan 
recorded  data  and  to  develop  categories  of  phenomena. 
These  categories  are  usually  called  codes.  They  enable 
the  researcher  to  manage  data  by  labeling,  storing,  and 
retrieving  it  according  to  the  codes.  Miles  and  Huber- 
man  (1994)  suggested  that  data  can  be  coded  descrip¬ 
tively  or  interpretively.  Bogdan  and  Biklen  (1992)  rec¬ 
ommended  reading  data  over  at  least  several  times  to 
begin  to  develop  a  coding  scheme.  In  one  of  the  many 
examples  he  provided,  Spradley  (1979)  described  in 
extensive  detail  how  to  code  and  analyze  interview 
data,  which  are  semantic  data  as  are  most  qualitative 
data.  He  also  described  how  to  construct  domain,  struc¬ 
tural,  taxonomic,  and  componential  analyses. 

Data  Management 

Analysis  of  data  requires  continually  examining,  sort¬ 
ing,  and  reexamining  data.  Qualitative  researchers  use 
many  means  to  organize,  retrieve,  and  analyze  their 
data.  To  code  data,  many  researchers  simply  use  note¬ 
books  and  boxes  of  paper,  which  can  then  be  resorted 
and  analyzed  on  an  ongoing  basis.  Computers  have 
long  been  used  for  managing  and  analyzing  qualitative 
data.  Several  resources  exist  to  aid  the  researcher  in 
finding  and  using  software  for  data  analysis  and  man¬ 
agement,  including  books  (Weitzman  and  Miles,  1995) 
and  websites  that  discuss  and  evaluate  research  soft¬ 
ware  (American  Evaluation  Association,  2007;  Cuneo, 
2000;  Horber,  2006). 


Writing  the  Research  Report 

In  some  respects,  writing  a  report  of  a  study  that  uses 
mixed-methods  may  not  differ  greatly  from  writing  a 
report  summarizing  a  more  traditional  experimental 
study;  for  example,  a  standard  format  for  preparing  a 
research  report  includes  an  introduction,  literature 
review,  description  of  methods,  and  presentation  of 
findings,  completed  by  a  summary  and  discussion 
(Borg  and  Gall,  1989).  A  mixed-methods  study,  how¬ 
ever,  allows  the  researcher  the  opportunity  to  create 
sections  of  the  report  that  may  expand  on  the  tradi¬ 
tional.  The  quantitative  findings  may  be  reported  in  the 
manner  of  an  experimental  study  (Ross  and  Morrison, 
2004).  The  qualitative  components  of  research  reports 
typically  will  be  woven  around  a  theme  or  central 
message  and  will  include  an  introduction,  core  mate¬ 
rial,  and  conclusion  (Bogdan  and  Biklen,  1992).  Qual¬ 
itative  findings  may  take  the  form  of  a  series  of  themes 
from  interview  data  or  the  form  of  a  case  study,  as  in 
the  Reiser  and  Mory  (1991)  study.  For  a  case  study, 
the  report  may  include  considerable  quantification  and 
tables  of  enumerated  data,  or  it  may  take  a  strictly 
narrative  form.  Recent  studies  have  been  reported  in 
more  nontraditional  forms,  such  as  stories,  plays,  and 
poems  that  show  participants’  views.  Suggestions  for 
writing  up  qualitative  research  are  many  (Meloy,  1994; 
Van  Maanen,  1988;  Wolcott,  1990). 

In  addition  to  the  studies  mentioned  earlier,  many 
excellent  examples  of  mixed-methods  studies  may  be 
examined  to  see  the  various  ways  in  which  the  results 
of  these  studies  have  been  written.  Seel  and  colleagues 
(2000),  in  an  investigation  of  mental  models  and 
model-centered  learning  environments,  used  quantita¬ 
tive  learning  measures  that  included  pretests,  post¬ 
tests,  and  a  measure  of  the  stability  of  learning  four 
months  after  the  instruction.  They  also  used  a  receptive 
interview  technique  they  called  causal  explanations  to 
investigate  learners’  mental  models  and  learning  pro¬ 
cesses.  In  this  and  subsequent  studies,  Seel  (2004)  also 
investigated  learners’  mental  models  of  dynamic  sys¬ 
tems  using  causal  diagrams  that  learners  developed 
and  teach-back  procedures,  in  which  a  student  explains 
a  model  to  another  student  and  this  epistemic  discourse 
is  then  examined. 

Conclusion 

The  challenges  to  educational  technology  researchers 
who  choose  to  use  multiple  methods  to  answer  their 
questions  are  many,  but  the  outcome  of  choosing 
mixed  methods  has  great  potential.  Issues  of  validity, 
reliability,  and  generalizability  are  central  to  experi¬ 
mental  research  (Ross  and  Morrison,  2004)  and  mixed- 
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methods  research;  however,  these  concerns  are 
addressed  quite  differently  when  using  qualitative 
methods  and  techniques.  Suggestions  and  criteria  for 
evaluating  the  quality  of  mixed-methods  studies  and 
research  activities  may  be  adapted  from  those  sug¬ 
gested  by  Savenye  and  Robinson  (2005): 

•  Learn  as  much  as  possible  about  the  context 
of  the  study,  and  build  in  enough  time  to 
conduct  the  study  well. 

•  Learn  more  about  the  methods  to  be  used, 
and  train  yourself  in  these  methods. 

•  Conduct  pilot  studies  whenever  possible. 

•  Use  triangulation  (simply  put,  use  multiple 
data  sources  to  yield  deeper,  more  true  views 
of  the  findings). 

•  Be  ethical  in  all  ways  when  conducting 
research. 

•  Listen  carefully  to  participants ,  and  carefully 
record  what  they  say  and  do. 

•  Keep  good  records,  including  audit  trails. 

•  Analyze  data  continually  throughout  the 
study,  and  consider  having  other  researchers 
and  participants  review  your  themes,  pat¬ 
terns,  and  findings  to  verify  them. 

•  Describe  well  all  methods,  decisions, 
assumptions,  and  biases. 

•  Using  the  appropriate  methods  (and  balance 
of  methods  when  using  mixed  methods)  is 
the  key  to  successful  educational  research. 

ASSESSMENT  OF  GROUP 
LEARNING  PROCESSES 

Tristan  E.  Johnson  and  Debra  L.  O’Connor 

Similar  to  organizations  that  rely  on  groups  of  workers 
to  address  a  variety  of  difficult  and  challenging  tasks 
(Salas  and  Fiore,  2004),  groups  are  formed  in  various 
learning  settings  to  meet  instructional  needs  as  well  as 
to  exploit  the  pedagogical,  learning,  and  pragmatic 
benefits  associated  with  group  learning  (Stahl,  2006). 
In  educational  settings,  small  groups  have  been  typi¬ 
cally  used  to  promote  participation  and  enhance  learn¬ 
ing.  One  of  the  main  reasons  for  creating  learning 
groups  is  to  facilitate  the  development  of  professional 
skills  that  are  promoted  from  group  learning,  such  as 
communication,  teamwork,  decision  making,  leader¬ 
ship,  valuing  others,  problem  solving,  negotiation, 
thinking  creatively,  and  working  as  a  member  of  a 
group  (Camevale  et  al.,  1989). 

Group  learning  processes  are  the  interactions  of 
two  or  more  individuals  with  themselves  and  their 


environment  with  the  intent  to  change  knowledge, 
skill,  or  attitude.  We  use  the  term  group  to  refer  to  the 
notion  of  small  groups  and  not  large  groups  character¬ 
ized  as  large  organizations  (Levine  and  Moreland, 
1990;  Woods  et  al.,  2000).  Interest  in  group  learning 
processes  can  be  found  not  only  in  traditional  educa¬ 
tional  settings  such  as  elementary  and  secondary 
schools  but  also  in  workplace  settings,  including  the 
military,  industry,  business,  and  even  sports  (Guzzo 
and  Shea,  1992;  Levine  and  Moreland,  1990). 

There  are  several  reasons  to  assess  group  learning 
processes.  These  include  the  need  to  measure  group 
learning  as  a  process  outcome  and  to  capture  the  learn¬ 
ing  process  to  provide  feedback  to  the  group  with  the 
intent  to  improve  team  interactions  and  thereby 
improve  overall  team  performance.  Studies  looking  at 
group  processes  have  led  to  improved  understanding 
about  what  groups  do  and  how  and  why  they  do  what 
they  do  (Salas  and  Cannon-Bowers,  2000).  Another 
reason  to  assess  group  learning  processes  is  to  capture 
highly  successful  group  process  behaviors  to  develop 
an  interaction  framework  that  could  then  be  used  to 
inform  the  design  and  development  of  group  instruc¬ 
tional  strategies.  Further,  because  the  roles  and  use  of 
groups  in  supporting  and  facilitating  learning  have 
increased,  the  interest  in  studying  the  underlying  group 
mechanisms  has  increased. 

Many  different  types  of  data  collection  and  analy¬ 
sis  methods  can  be  used  to  assess  group  learning  pro¬ 
cesses.  The  purpose  of  this  section  is  to  describe  these 
methods  by:  (1)  clarifying  how  these  methods  are  sim¬ 
ilar  to  and  different  from  single  learner  methods,  (2) 
describing  a  framework  of  data  collection  and  analysis 
techniques,  and  (3)  presenting  analysis  considerations 
specific  to  studying  group  learning  processes  along 
with  several  examples  of  existing  methodologies. 

Group  Learning  Processes  Compared 
with  Individual  Learning  Processes 
and  Group  Performance 

Traditional  research  on  learning  processes  has  focused 
on  psychological  perspectives  using  traditional  psy¬ 
chological  methods.  The  unit  of  analysis  for  these 
methods  emphasizes  the  behavior  or  mental  activity  of 
an  individual  concentrating  on  learning,  instructional 
outcomes,  meaning  making,  or  cognition,  all  at  an 
individual  level  (Koschmann,  1996;  Stahl,  2006).  In 
contrast,  the  framework  for  group  research  focuses  on 
research  traditions  of  multiple  disciplines,  such  as 
communication,  information,  sociology,  linguistics, 
military,  human  factors,  and  medicine,  as  well  as  fields 
of  applied  psychology  such  as  instructional,  educa¬ 
tional,  social,  industrial,  and  organization  psychology. 
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As  a  whole,  these  disciplines  extend  the  traditional 
psychological  perspectives  and  seek  understanding 
related  to  interaction,  spoken  language,  written  lan¬ 
guage,  culture,  and  other  aspects  related  to  social  sit¬ 
uations.  Stahl  (2006)  pointed  out  that  individuals  often 
think  and  learn  apart  from  others,  but  learning  and 
thinking  in  isolation  are  still  conditioned  and  mediated 
by  important  social  considerations. 

Group  research  considers  various  social  dimen¬ 
sions  but  primarily  focuses  on  either  group  perfor¬ 
mance  or  group  learning.  Group  learning  research  is 
focused  in  typical  learning  settings.  We  often  see  chil¬ 
dren  and  youth  engaged  in  group  learning  in  a  school 
setting.  Adult  group  learning  is  found  in  post-second- 
ary  education,  professional  schools,  vocational 
schools,  colleges  and  universities,  and  training  ses¬ 
sions,  as  well  as  in  on-the-job  training  environments. 
A  number  of  learning  methods  have  been  used  in  all 
of  these  settings.  A  few  specific  strategies  that  use 
groups  to  facilitate  learning  include  cooperative  learn¬ 
ing,  collaborative  learning  (Johnson  et  al.,  2000; 
Salomon  and  Perkins,  1998),  computer-supported  col¬ 
laborative  learning  (Stahl,  2006),  and  team-based 
learning  (Michaelsen,  2004).  General  terms  used  to 
refer  to  the  use  of  multiple  person  learning  activities 
include  learning  groups,  team  learning,  and  group 
learning.  Often,  these  terms  and  specific  strategies  are 
used  interchangeably  and  sometimes  not  in  the  ways 
just  described. 

In  addition  to  learning  groups,  adults  engage  in 
groups  activities  in  performance  (workplace)  settings. 
Although  a  distinction  is  made  between  learning  and 
performance,  the  processes  are  similar  for  groups 
whose  primary  intent  is  to  learn  and  for  those  focused 
on  performing.  The  literature  on  workplace  groups 
(whose  primary  focus  is  on  completing  a  task)  offers 
a  number  of  techniques  that  can  be  used  to  study  group 
learning  processes  much  like  the  literature  on  individ¬ 
ual  learning.  Group  learning  process  methods  include 
the  various  methods  typically  found  when  studying 
individuals  and  also  have  additional  methodologies 
unique  to  studying  groups. 

Methodological  Framework: 

Direct  and  Indirect  Process  Measures 

When  studying  group  learning  processes,  three  general 
categories  of  measures  can  be  employed:  (1)  the  pro¬ 
cess  or  action  of  a  group  (direct  process  measures), 
(2)  a  state  or  a  point  in  time  of  a  group  (indirect  process 
measure),  and  (3)  an  outcome  or  performance  of  a 
group  (indirect  non-process  measure).  Direct  process 
measures  are  techniques  that  directly  capture  the  pro¬ 
cess  of  a  group.  These  measures  are  continuous  in 


nature  and  capture  data  across  time  by  recording  the 
sound  and  sight  of  the  group  interactions.  Examples 
of  these  measures  include  recording  the  spoken  lan¬ 
guage,  written  language,  and  visible  interactions. 
These  recording  can  be  video  or  audio  recordings,  as 
well  as  observation  notes. 

Indirect  process  measures  use  techniques  that  indi¬ 
rectly  capture  group  processes.  These  measures  are 
discrete  and  capture  a  state  or  condition  of  the  group 
processes  at  a  particular  point  in  time,  either  during  or 
after  group  activity.  These  measures  involve  capturing 
group  member  or  observer  perceptions  and  reactions 
that  focus  on  group  processes.  Examples  of  these  mea¬ 
sures  involve  interviews,  surveys,  and  questionnaires 
that  focus  on  explicating  the  nature  of  a  group  learning 
process  at  a  given  point  in  time.  These  measures  focus 
on  collecting  group  member  responses  about  the  pro¬ 
cess  and  are  specifically  not  a  direct  observation  of  the 
process. 

Indirect  non-process  measures  capture  group 
learning  data  relating  to  outcomes,  products,  perfor¬ 
mance.  These  are  not  measures  of  the  actual  process 
but  are  measures  that  might  be  related  to  group  pro¬ 
cesses.  They  may  include  group  characteristics  such 
as  demographics,  beliefs,  efficacy,  preferences,  size, 
background,  experience,  diversity,  and  trust  (Mathieu 
et  al.,  2000).  These  types  of  measures  have  the  poten¬ 
tial  to  explicate  and  support  the  nature  of  group  learn¬ 
ing  processes.  These  measures  are  focused  on  collect¬ 
ing  products  or  performance  scores  as  well  as 
soliciting  group  member  responses  about  group  char¬ 
acteristics.  These  measures  are  not  a  direct  observation 
of  the  group  learning  process.  Examples  of  these  mea¬ 
sures  include  performance  scores,  product  evaluations, 
surveys,  questionnaires,  interview  transcripts,  and 
group  member  knowledge  structures.  These  measures 
focus  on  explicating  the  nature  of  a  given  group’s  non¬ 
process  characteristics. 

When  considering  how  to  assess  group  learning 
processes,  many  of  the  techniques  are  very  similar  or 
the  same  as  those  used  to  study  individuals.  Group 
learning  process  measures  can  be  collected  at  both  the 
individual  and  group  levels  (O’Neil  et  al.,  2000;  Webb, 
1982).  Because  the  techniques  can  be  similar  or  iden¬ 
tical  for  individuals  and  groups,  some  confusion  may 
arise  when  it  is  realized  that  individual-level  data  are 
not  in  a  form  that  can  be  analyzed;  the  data  must  be 
group-level  data  (group  dataset;  see  Figure  55.1)  for 
analysis. 

When  designing  a  study  on  group  learning  pro¬ 
cesses,  various  measurement  techniques  can  be  used 
depending  on  the  type  of  questions  being  asked. 
Although  numerous  possibilities  are  associated  with 
the  assessment  of  group  learning  processes,  three 
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Direct  Process  Measures 

A.  Direct  capture  of  group  level  process  data 
yielding  holistic  group  dataset 
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B.  Indirect  elicitation  of  individual  level  process 
data  yielding  collective  group  dataset 
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Figure  55.1  Alternative  process  measures  for  assessing  group  learning  processes. 


elements  must  be  considered  when  deciding  on  what 
techniques  to  use:  data  collection,  data  manipulation, 
or  data  analysis. 

Data  collection  techniques  involve  capturing  or 
eliciting  data  related  to  group  learning  processes  at  an 
individual  or  group  level.  Data  collected  at  the  group 
level  (capturing  group  interactions  or  eliciting  group 
data)  yield  holistic  group  datasets  (Figure  55.1).  When 
the  collected  data  are  in  this  format,  it  is  not  necessary 
to  manipulate  the  data.  In  this  form,  the  data  are  ready 
to  be  analyzed.  If,  however,  data  are  collected  at  the 
individual  level,  then  the  data  must  be  manipulated, 
typically  via  aggregation  (Stahl,  2006),  to  create  a 
dataset  that  represents  the  group  (collective  group 
dataset)  prior  to  data  analysis  (Figure  55.1).  Collecting 
data  at  the  individual  level  involves  collecting  individ¬ 
ual  group  members’  data  and  then  transforming  the 
individual  data  to  an  appropriate  form  (collective  group 
dataset)  for  analysis  (see  Figure  55.1).  This  technique 
of  creating  a  collective  group  dataset  from  individual 
data  is  similar  to  a  process  referred  to  as  analysis  con¬ 
structed  dataset  creation  (O’Connor  and  Johnson, 
2004).  In  this  form,  the  data  are  ready  to  be  analyzed. 


Data  Collection  and  Analysis  Techniques 

When  considering  the  different  group  learning  process 
assessment  techniques,  they  can  be  classified  based  on 
the  type  of  measure  (continuous  or  discrete).  The  cor¬ 
responding  analytical  techniques  that  can  be  used  are 
dependent  on  the  collected  data.  Many  techniques  have 
been  used  to  assess  groups.  The  following  section  pre¬ 
sents  the  major  categories  of  techniques  based  on  their 
ability  to  measure  group  processes  directly  (continu¬ 
ous  measures)  or  indirectly  (discrete  measures).  Table 
55.1  summarizes  the  nature  of  data  collection,  manip¬ 
ulation,  and  analysis  for  the  three  major  grouping  of 
measurement  techniques:  direct  process  measures  and 
the  two  variations  of  indirect  process  measures. 

Direct  Process  Data  Collection  and  Analysis 

Direct  process  measurement  techniques  focus  specifi¬ 
cally  on  capturing  the  continuous  process  interactions 
in  groups  (O’Neil  et  al.,  2000).  These  techniques 
include  measures  of  auditory  and  visual  interactions. 
Several  data  collection  and  data  analysis  techniques 
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TABLE  55.1 

Summary  of  Measurement  Techniques  Used  to  Assess  Group  Learning  Processes 

Direct  Process  Measure  Techniques— Holistic  Group  Dataset 

Data  collection  Directly  capturing  group  learning  processes  involves  techniques  that  are  used  by  all  group  members  at  the  same  time. 

Data  manipulation  Data  manipulation  not  needed  because  data  is  captured  at  group  level  (holistic  group  dataset). 

Data  analysis  Continuous  process  techniques  focus  on  interactions  of  group  members  generating  qualitative  and  quantitative  findings 

associated  with  continuous  measures. 

Indirect  Process  Measure  Techniques— Holistic  Group  Dataset 

Data  collection  Indirectly  eliciting  group  learning  processes  involves  techniques  that  are  used  by  all  group  members  at  the  same  time. 

Data  manipulation  Data  manipulation  not  needed  because  data  is  captured  at  group  level  (holistic  group  dataset). 

Data  analysis  Discrete  process  techniques  are  dependent  on  dataset  characteristics  (focus  on  process  or  performance) .  They  can  include 

qualitative  and  quantitative  data  analysis  techniques  associated  with  discrete  measures. 

Indirect  Process  Measure  Techniques— Collective  Group  Dataset 

Data  collection  Indirectly  eliciting  group  learning  processes  involves  techniques  that  are  used  by  each  group  member  separately. 

Data  manipulation  Individual  data  is  then  aggregated  to  create  a  dataset  that  represents  the  group  data  (analysis  constructed). 

Data  analysis  Discrete  process  techniques  are  dependent  on  dataset  characteristics  (focus  on  process  or  performance) .  They  can  include 

qualitative  and  quantitative  data  analysis  techniques  associated  with  discrete  measures. 


are  related  to  measuring  the  group  learning  processes 
directly.  The  two  key  techniques  for  capturing  actions 
and  language  are  (1)  technology  and  (2)  observation. 
Using  technology  to  capture  group  processes  can  pro¬ 
vide  researchers  with  data  different  from  the  observa¬ 
tion  data.  Researchers  can  combine  the  use  of  technol¬ 
ogy  and  observation  simultaneously  to  capture  group 
processes  (O’Neil  et  al.,  2000;  Paterson  et  al.,  2003). 
These  data  can  be  analyzed  in  the  captured  form  or 
transcribed  into  a  text  form. 

Use  of  Technology  to  Capture  Group  Process 

Spoken  Language  Processes 

Techniques  to  capture  a  group’s  spoken  language 
involve  either  audio  recording  or  video  recording 
(Schweiger,  1986;  Willis,  2002)  the  spoken  language 
that  occurs  during  group  interactions  (Pavitt,  1998).  It 
can  also  include  the  spoken  language  of  group  mem¬ 
bers  as  they  explain  their  thinking  during  group  pro¬ 
cesses  in  the  form  of  a  think-aloud  protocol  (Ericsson 
and  Simon,  1993). 

Written  Language  Processes 

Group  learning  processes  are  typically  thought  of  as 
using  spoken  language,  but  new  communication  tools 
are  available  that  allow  groups  to  communicate  and 
interact  using  written  language.  Examples  include  chat 
boards,  whiteboards  (although  these  are  not  limited  to 
written  language),  and  discussion  boards.  Also,  com¬ 
puter-supported  collaborative  learning  (CSCL)  is  a 
computer-based  network  system  that  supports  group 
learning  interactions  (Stahl,  2006). 


Visible  Processes 

Techniques  to  capture  a  group’s  visible  interactions 
include  video  recording  of  the  behaviors  and  actions 
that  occur  in  group  interactions  (Losada,  1990;  Prichard, 
2006;  Schweiger,  1986;  Sy,  2005;  Willis  et  al.,  2002). 

Use  of  Observations  to  Capture  Group  Process 

Although  the  use  of  technology  may  capture  data  with 
a  high  level  of  realism,  some  group  events  can  be  better 
captured  by  humans  because  of  their  ability  to  observe 
more  than  what  can  be  captured  by  technology.  Obser¬ 
vations  ideally  are  carried  out  with  a  set  of  carefully 
developed  observation  protocols  to  help  focus  the 
observers  and  to  teach  them  how  to  describe  key  pro¬ 
cess  events.  Observers  are  a  good  source  for  capturing 
various  types  of  information  (Patton,  2001),  such  as 
settings,  human  and  social  environments,  group  activ¬ 
ities,  style  and  types  of  language  used,  nonverbal  com¬ 
munication,  and  events  that  are  not  ordinary.  Observa¬ 
tional  data,  for  example,  are  important  for  studying 
group  learning  process  (Battistich  et  al.,  1993;  Lin- 
gard,  2002;  Sy,  2005;  Webb,  1982;  Willis  et  al.,  2002). 
The  type  of  information  typically  captured  includes 
location,  organization,  activities,  and  behaviors  (Bat¬ 
tistich  et  al.,  1993;  Losada,  1990),  as  well  as  the  fre¬ 
quency  and  quality  of  interactions  (Battistich,  1993). 

Direct  Process  Data  Analysis 

Data  that  are  a  direct  measure  of  group  processes  are 
captured  in  a  holistic  format  that  is  ready  to  be  ana¬ 
lyzed  (Figure  55.1  and  Table  55.1).  Several  analytical 
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techniques  are  available  that  can  be  used  for  analyzing 
group  data,  particularly  direct  process  data.  The  fol¬ 
lowing  list  is  a  representative  sample  of  the  analysis 
techniques  applied  to  spoken  or  written  language,  vis¬ 
ible  interaction,  and  observational  data:  sequential 
analysis  of  group  interactions  (Bowers,  2006;  Jeong, 
2003;  Rourke  et  al.,  2001),  analysis  of  interaction  com¬ 
munication  (Bales,  1950;  Qureshi,  1995),  communica¬ 
tion  analysis  (Bowers  et  al.,  1998),  anticipation  ratio 
(Eccles  and  Tenenbaum,  2004),  in-process  coordina¬ 
tion  (Eccles  and  Tenenbaum,  2004),  discourse  analysis 
(Aviv,  2003;  Hara  et  al.,  2000),  content  analysis  (Aviv, 
2003;  Hara  et  al.,  2000),  cohesion  analysis  (Aviv, 
2003),  and  protocol  analysis  (Ericsson  and  Simon, 
1980,  1993).  Visible  interactions  techniques  also 
include  using  a  behavior  time  series  analysis  (Losada, 
1990).  This  analysis  involves  looking  at  dominate  vs. 
submissive,  friendly  vs.  unfriendly,  or  task-oriented  vs. 
emotionally  expressive  behavior.  For  observational 
data,  researchers  focus  on  various  qualitative  tech¬ 
niques  associated  with  naturalistic  observations  (Adler 
and  Adler,  1994;  Patton,  2001).  Some  common  tasks 
associated  with  this  type  of  analysis  include  group  and 
character  sequence  analysis  and  assertion  evaluation 
(Garfinkel,  1967;  Jorgensen,  1989). 

Indirect  Process  Data  Collection  and  Analysis 

Many  data  collection  techniques  are  related  to  measur¬ 
ing  the  group  learning  processes  indirectly.  Indirect 
group  process,  characteristic,  and  product  measurement 
techniques  elicit  group  information  at  a  specific  point 
in  time.  These  discrete  measures  do  not  capture  group 
processes  directly  but  elicit  data  that  describe  group 
processes  or  process-related  data  such  as  group  charac¬ 
teristics  or  group  outcomes  (things  that  may  have  a 
relation  to  the  group  processes).  The  three  key  types  of 
data  related  to  group  learning  processes  are  indirect 
group  process  data ,  group  characteristic  data ,  and 
group  product  data ,  within  which  specific  factors  can 
be  measured.  Indirect  group  process  data  describe  group 
processes  and  can  include  factors  such  as  group  com¬ 
munication  (verbal/nonverbal),  group  actions,  group 
behaviors,  group  performance,  and  group  processes. 
Group  characteristic  data,  relating  to  group  processes, 
include  factors  such  as  group  knowledge,  group  skills, 
group  efficacy,  group  attitudes,  group  member  roles, 
group  environment,  and  group  leadership.  The  key  elic¬ 
itation  techniques  for  both  of  these  types  of  indirect  data 
include  interviews,  questionnaires,  and  conceptual 
methods  (Cooke  et  al.,  2000).  Each  technique  can  be 
focused  on  group  process  or  group  characteristics.  After 
reviewing  methods  to  analyze  group  processes,  we  dis¬ 
cuss  methods  for  analyzing  group  products. 


Interviews 

Interviews  are  a  good  technique  for  collecting  gen¬ 
eral  data  about  a  group.  The  various  types  of  inter¬ 
viewing  techniques  include  unstructured  interviews 
(Lingard,  2002)  and  more  structured  interviews, 
which  are  guided  by  a  predetermined  format  that  can 
provide  either  a  rigid  or  loosely  constrained  format. 
Structured  interviews  require  more  time  to  develop 
but  are  more  systematic  (Cooke  et  al.,  2000).  Inter¬ 
views  are  typically  conducted  with  a  single  person 
at  a  time;  however,  is  not  uncommon  to  conduct  a 
focus  group,  where  the  entire  group  is  simultaneously 
interviewed.  In  a  focus  group,  a  facilitator  interviews 
by  leading  a  free  and  open  group  discussion  (Myl- 
lyaho  et  al.,  2004).  The  analysis  of  interview  data 
requires  basic  qualitative  data  analysis  techniques 
(Adler  and  Adler,  1994;  Patton,  2001).  Conducting 
interviews  can  be  straightforward,  but  the  analysis  of 
the  data  relies  tremendously  on  the  interviewer’s 
interpretations  (Langan-Fox,  2000).  Key  steps  to  ana¬ 
lyzing  interviews  are  coding  the  data  for  themes  (Lin¬ 
gard,  2002)  and  then  studying  the  codes  for  meaning. 
Each  phrase  is  closely  examined  to  discover  impor¬ 
tant  concepts  and  reveal  overall  relationships.  For  a 
more  holistic  approach  to  analysis,  a  group  interview 
technique  can  be  used  to  discuss  findings  and  to 
generate  collective  meaning  given  specific  questions 
(Myllyaho  et  al.,  2004).  Content  analysis  is  com¬ 
monly  used  to  analyze  written  statements  (Langan- 
Fox  and  Tan,  1997).  Other  key  analysis  techniques 
focus  on  process  analysis  (Fussell  et  al.,  1998;  Pri¬ 
chard,  2006),  specifically  looking  at  discussion  top¬ 
ics,  group  coordination,  group  cognitive  overload, 
and  analysis  of  task  process.  Other  group  character¬ 
istic  analysis  techniques  could  include  role  analysis 
and  power  analysis  (Aviv,  2003). 

Questionnaires 

Questionnaires  are  a  commonly  used  technique  to  col¬ 
lect  data  about  group  processes  (O’Neil  et  al.,  2000; 
Sy,  2005;  Webb,  1982;  Willis  et  al.,  2002).  Similar  to 
highly  structured  interviews,  questionnaires  can  also 
look  at  relationship-oriented  processes  and  task-ori¬ 
ented  processes  (Urch  Druskat  and  Kayes,  2000). 
Questionnaires  can  be  either  closed  ended  or  open 
ended  (Alavi,  1994).  Open-ended  questionnaires  are 
more  closely  related  to  a  structured  interview;  the  data 
collected  using  this  format  can  be  focused  on  group 
processes  as  well  as  group  characteristics.  Closed- 
ended  questionnaires  offer  a  limited  set  of  responses. 
The  limited  responses  involve  some  form  of  scale  that 
could  be  nominal,  ordinal,  interval,  or  ratio.  Data  from 
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this  format  have  a  limited  ability  to  capture  group 
process  data,  but  this  is  the  typical  format  for  collecting 
data  associated  with  group  characteristics  such  as 
social  space,  group  efficacy  scales,  group  skills,  group 
efficacy,  group  attitudes,  group  member  roles,  leader¬ 
ship,  and  group  knowledge  .  Data  from  questionnaires 
can  be  analyzed  much  like  interview  data  if  the  items 
are  open  ended.  If  the  questionnaire  is  closed  ended, 
then  the  instrument  must  be  scrutinized  for  reliability 
prior  to  data  analysis.  Assuming  sufficient  evidence  of 
reliability,  analyzing  data  from  closed-ended  question¬ 
naires  involves  interpreting  a  measurement  based  on  a 
particular  theoretical  construct.  The  types  of  data  anal¬ 
ysis  techniques  that  are  appropriate  depend  on  the  type 
of  scale  used  in  a  questionnaire  (nominal,  ordinal, 
interval,  or  ratio). 

Conceptual  Methods 

Conceptual  methods  involve  assessing  individual  or 
group  understanding  about  a  given  topic.  Several  data 
collection  techniques  are  utilized  to  elicit  knowledge 
structures.  A  review  of  the  literature  by  Langan-Fox 
et  al.  (2000)  found  that  knowledge  in  teams  has  been 
investigated  by  several  qualitative  and  quantitative 
methods,  including  various  elicitation  techniques  (e.g., 
cognitive  interviewing,  observation,  card  sorting, 
causal  mapping,  pairwise  ratings)  and  representation 
techniques  (e.g.,  MDS,  distance  ratio  formulas.  Path¬ 
finder)  that  utilize  aggregate  methods. 

One  of  the  most  common  methods  for  assessing 
group  knowledge  is  the  use  of  concept  maps  (Herl  et 
al.,  1999;  Ifenthaler,  2005;  O’Connor  and  Johnson, 
2004;  O’Neil  et  al.,  2000).  Through  concept  mapping, 
similarity  of  group  mental  models  can  be  measured  in 
terms  of  the  proportion  of  nodes  and  links  shared 
between  one  concept  map  (mental  model)  and  another 
(Rowe  and  Cooke,  1995).  Several  researchers  believe 
that  group  knowledge  and  group  processes  are  linked. 
Research  has  shown  that  specific  group  interactions 
such  as  communication  and  coordination  mediate  the 
development  of  group  knowledge  and  thus  mediate 
group  performance  (Mathieu  et  al.,  2000).  Group  inter¬ 
actions  coupled  with  group  shared  knowledge  are  a 
predominate  force  in  the  construct  of  group  cognition. 
As  teammates  interact,  they  begin  to  share  knowledge, 
thus  enabling  them  to  interpret  cues  in  similar  ways, 
make  compatible  decisions,  and  take  proper  actions 
(Klimoski  and  Mohammed,  1994).  Group  shared 
knowledge  can  help  group  members  explain  other 
members’  actions,  understand  what  is  occurring  with 
the  task,  develop  accurate  expectations  about  future 
member  actions  and  task  states,  and  communicate 
meanings  efficiently. 


Analyzing  knowledge  data  can  certainly  involve 
qualitative  methods.  These  methods  tend  to  offer  more 
detail  and  depth  of  information  than  might  be  found 
through  statistical  analyses  (Miles  and  Huberman, 
1994;  Patton,  2001).  Using  qualitative  analysis,  we 
obtain  greater  understanding  about  the  relationships 
between  concepts  within  the  context  of  the  individual 
mental  model.  We  also  gain  better  insight  into  the 
sharedness  of  understanding  between  group  members. 
Quantitative  data  analysis  techniques  provide 
researchers  with  tools  to  draw  inferences  on  the  change 
in  group  knowledge  as  well  as  statistically  proving  a 
change  or  variation  in  knowledge  structures. 

Several  methods  have  been  developed  to  analyze 
data  regarding  group  knowledge.  Most  of  them  include 
an  elicitation  and  analysis  component.  Some  tech¬ 
niques  use  mixed  methods  such  as  the  analysis-con¬ 
structed  shared  mental  model  (ACSMM)  (O’Connor 
and  Johnson,  2004),  DEEP  (Spector  and  Koszalka, 

2004) ,  and  social  network  analysis  (Qureshi,  1995). 
Other  methods  are  quantitative  in  nature,  such  as  the 
Stanford  Microarray  Database  (SMD)  (Ifenthaler, 

2005) ,  Model  Inspection  Trace  of  Concepts  of  Rela¬ 
tions  (MITOCAR)  (Pirnay-Dummer,  2006),  multidi¬ 
mensional  scaling  (MDS),  distance  ratio  formula,  and 
Pathfinder  (Cooke  et  al.,  2000). 

Group  product  data  are  the  artifacts  created  from 
a  group  interaction.  Group  products  typically  do  not 
capture  the  process  that  a  group  undertook  to  create 
the  product  but  is  evidence  of  the  group’s  abilities. 
Many  research  studies  that  claim  to  study  group  pro¬ 
cesses  only  capture  group  product  data.  This  is  due  in 
part  to  the  claim  that  is  made  regarding  the  link 
between  group  products  and  group  processes  and  char¬ 
acteristics  (Cooke  et  al.,  2000;  Lesh  and  Dorr,  2003; 
Mathieu  et  al.,  2000;  Salas  and  Cannon-Bowers,  2001; 
Schweiger,  1986).  Although  some  evidence  suggests 
this  relationship  in  a  few  areas,  more  research  is 
required  to  substantiate  this  claim. 

Analysis  of  the  group  product  data  involves  tech¬ 
niques  used  when  analyzing  individual  products.  Ana¬ 
lyzing  the  quality  of  a  product  can  be  facilitated  by 
the  use  of  specified  criteria.  These  criteria  are  used  to 
create  a  product  rating  scale.  Rating  scales  can  include 
numerical  scales,  descriptive  scales,  or  checklists. 
Numerical  scales  present  a  range  of  numbers  (usually 
sequential)  that  are  defined  by  a  label  on  either  end. 
Each  item  in  the  questionnaire  is  rated  according  to 
the  numerical  scale.  There  is  no  specific  definition  of 
what  the  various  numbers  mean,  except  for  the  indi¬ 
cators  at  the  ends  of  the  scale;  for  example,  a  scale 
from  1  (very  weak)  to  5  (very  strong)  is  very  subjective 
but  relatively  easy  to  create.  Descriptive  scales  are 
similar,  but  focus  on  verbal  statements.  Numbers  can 
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be  assigned  to  each  statement.  Statements  are  typically 
in  a  logical  order.  A  common  example  of  a  descriptive 
scale  is  “strongly  disagree  (1),  disagree  (2),  neutral 
(3),  agree  (4),  and  strongly  agree  (5).”  A  checklist  can 
be  developed  to  delineate  specific  qualities  for  a  given 
criterion.  This  can  provide  a  high  level  of  reliability 
because  a  specific  quality  is  presented  and  the  rater 
simply  indicates  whether  an  item  is  present  or  not.  The 
validity  of  a  checklist  requires  a  careful  task  analysis 
to  ensure  scale  validity. 

General  Considerations  for  Group 
Learning  Process  Assessment 

In  assessment  of  group  learning  processes,  researchers 
should  consider  several  issues.  These  issues  fall  into 
four  categories:  (1)  group  setting,  (2)  variance  in  group 
member  participation,  (3)  overall  approach  to  data  col¬ 
lection  and  analysis,  and  (4)  thresholds. 

Group  Setting 

To  a  somewhat  lesser  degree  than  the  other  three  issues, 
group  settings  should  be  considered  when  determining 
the  best  approach  and  methods  for  a  particular  study. 
Finalizing  which  techniques  to  use  may  depend  on 
whether  the  groups  will  be  working  in  a  group  learning 
setting  or  individually  and  then  coming  together  as  a 
group  at  various  points.  Some  groups  may  meet  in  face- 
to-face  settings  or  other  settings  that  allow  for  synchro¬ 
nous  interactions:  however,  distributed  groups  have 
technology-enabled  synchronous  and  asynchronous 
tools  or  asynchronous  interactions  only.  These  varia¬ 
tions  in  group  setting  can  influence  the  selection  of 
specific  group  learning  process  assessment  methods. 

Variance  in  Group  Member  Participation 

When  collecting  multiple  sets  of  data  over  time, 
researchers  should  consider  how  they  will  deal  with  a 
variance  in  group  member  participation  (group  mem¬ 
bers  absent  during  data  collection  or  new  members 
joining  the  group  midstream).  There  are  benefits  and 
consequences  for  any  decision  made,  but  it  is  neces¬ 
sary  to  determine  whether  or  not  all  data  collected  will 
be  used,  regardless  of  the  group  members  present  at 
the  time  of  data  collection.  Researchers  who  choose 
not  to  use  all  data  might  consider  using  only  those  data 
submitted  by  group  members  who  were  present  during 
all  data  collection  sessions  (O’Connor  and  Johnson, 
2004).  If  data  analysis  will  be  based  on  a  consistent 
number  of  group  members,  it  will  be  necessary  to 
consider  how  to  handle  data  from  groups  that  may  not 
have  the  same  group  members  present  in  each  measure 


point.  Also,  with  fluctuations  in  group  compositions, 
it  is  important  to  consider  the  overall  group  demo¬ 
graphics  and  possible  influences  of  individual  group 
members  on  the  group  as  a  whole. 

Overall  Approach  to  Data  Collection  and  Analysis 

In  a  holistic  approach,  individual  group  members  work 
together  and  one  dataset  represents  the  group  as  a 
whole;  however,  the  processes  of  group  interaction 
naturally  changes  how  individual  group  members 
think.  The  alternative  is  to  capture  individual  measures 
and  perform  some  type  of  aggregate  analysis  methods 
to  represent  the  group;  however,  researchers  should 
consider  whether  or  not  the  aggregate  would  be  a  true 
representation  of  the  group  itself. 

Thresholds 

When  using  indirect  measures  that  require  an  aggre¬ 
gation  or  manipulation  of  data  prior  to  analysis, 
researchers  will  have  to  consider  such  issues  as  simi¬ 
larity  scores.  These  scores  define  the  parameters  for 
deciding  if  responses  from  one  individual  group  mem¬ 
ber  are  similar  to  the  responses  from  other  group  mem¬ 
bers  (O’Connor  and  Johnson,  2004;  Rentsch  and  Hall, 
1994;  Rentsch  et  al.,  in  press);  for  example,  will  the 
rating  of  3.5  on  a  5-point  scale  be  considered  similar 
to  a  rating  of  4.0  or  a  rating  of  3.0?  When  aggregating 
individual  data  into  a  representation  of  the  group,  will 
the  study  look  only  at  groups  where  a  certain  percent¬ 
age  of  the  group  responded  to  measures  (Ancona  and 
Caldwell,  1991)?  How  will  what  is  similar  or  shared 
across  individual  group  members  be  determined?  Will 
the  analysis  use  counts  (x  number  of  group  members) 
or  percentage  of  the  group  (e.g.,  50%)?  What  level  of 
similarity  or  sensitivity  will  be  used  to  compare  across 
groups  (O’Connor  and  Johnson,  2004)— 50%?  75%? 
What  about  the  level  of  mean  responses  in  question¬ 
naires  (Urch  Druskat  and  Kayes,  2000)?  Many  differ¬ 
ent  thresholds  that  must  be  considered  when  assessing 
group  learning  processes  and  analyzing  group  data  are 
not  concerns  when  studying  individuals. 

Conclusion 

Assessment  of  group  learning  processes  is  more  com¬ 
plex  than  assessment  of  individual  learning  processes 
because  of  the  additional  considerations  necessary  for 
selecting  data  collection  and  analysis  methods.  As  in 
most  research,  the  “very  type  of  experiment  set  up  by 
researchers  will  determine  the  type  of  data  and  there¬ 
fore  what  can  be  done  with  such  data  in  analysis  and 
interpretation”  (Langan-Fox  et  al.,  2004,  p.  348). 
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Indeed,  it  is  logical  to  allow  the  specific  research  ques¬ 
tions  to  drive  the  identification  of  data  collection  meth¬ 
ods.  The  selection  of  research  questions  and  subse¬ 
quent  identification  of  data  collection  methods  will 
naturally  place  limitations  on  suitable  data  analysis 
methods.  Careful  planning  for  the  study  of  group  learn¬ 
ing  processes,  from  the  selection  of  direct  or  indirect 
assessment  measures  to  considering  the  possible  influ¬ 
ences  group  characteristics  may  have  on  group  learn¬ 
ing  processes,  is  essential. 

Because  of  the  many  possible  combinations  of 
methods  and  techniques  available  for  studying  group 
learning  processes,  some  feel  that  research  has  not  yet 
done  enough  to  determine  the  best  methods  for  studying 
groups  (Langan-Fox  et  al.,  2000).  Many  group  learning 
process  studies  consider  only  outcome  measures  and 
do  not  directly  study  group  learning  processes  (Worchel 
et  al.,  1992).  Others  look  only  at  portions  of  the  group 
process  or  attempt  to  assess  group  learning  processes 
through  a  comparison  of  discrete  measures  of  the  group 
over  time.  Still  other  methods  for  data  collection  and 
analysis  of  group  data  are  being  developed  as  we  speak 
(Seel,  1999).  No  one  best  method  has  been  identified 
for  analyzing  group  learning  process  data,  so  we  sug¬ 
gest  that  studies  should  consider  utilizing  multiple 
methods  to  obtain  a  more  comprehensive  picture  of 
group  learning  processes.  If  we  are  to  better  understand 
the  notion  of  group  learning  processes  and  utilize  that 
understanding  in  design,  implementation,  and  manage¬ 
ment  of  learning  groups  in  the  future,  then  we  must 
address  the  basic  issues  that  are  related  with  conceptu¬ 
alization  and  measurement  (Langan-Fox  et  al.,  2004). 

ASSESSMENT  OF 
COMPLEX  PERFORMANCE 

Tamara  van  Gog,  Remy  M.  J.  P.  Rikers,  and  Paul  Ayres 

This  chapter  section  discusses  assessment  of  complex 
performance  from  an  educational  research  perspective, 
in  terms  of  data  collection  and  analysis.  It  begins  with 
a  short  introduction  on  complex  performance  and  a 
discussion  on  the  various  issues  related  to  selecting 
and  defining  appropriate  assessment  tasks,  criteria,  and 
standards  that  give  meaning  to  the  assessment. 
Although  many  of  the  issues  discussed  here  are  also 
important  for  performance  assessment  in  educational 
practice,  readers  particularly  interested  in  this  topic 
might  want  to  refer,  for  example  ,  to  Chapter  44  in  this 
Handbook  or  the  edited  books  by  Birenbaum  and 
Dochy  (1996)  and  Segers  et  al.  (2003).  For  a  discus¬ 
sion  of  laboratory  setups  for  data  collection,  see  the 
section  by  Duley  et  al.  in  this  chapter. 


Complex  performance  can  be  defined  as  perfor¬ 
mance  on  complex  tasks;  however,  definitions  of  task 
complexity  differ:  Campbell  (1988),  in  a  review  of  the 
literature,  categorized  complexity  as  primarily  subjec¬ 
tive  (psychological)  or  objective  (function  of  objective 
task  characteristics),  or  as  an  interaction  between 
objective  and  subjective  (individual)  characteristics. 
Campbell  reported  that  the  subjective  perspective 
emphasized  psychological  dimensions  such  as  task 
significance  and  identity.  On  the  other  hand,  objective 
definitions  consider  the  degree  of  structuredness  of  a 
task  or  of  the  possibility  of  multiple  solution  paths 
(Bystrom  and  Jarvelin,  1995;  Campbell,  1988).  When 
the  process  of  task  performance  can  be  described  in 
detail  a  priori  (very  structured),  a  task  is  considered 
less  complex;  in  contrast,  when  there  is  a  great  deal 
of  uncertainty,  it  is  considered  highly  complex.  Simi¬ 
larly,  complexity  can  vary  according  to  the  number  of 
solutions  paths  possible  .  When  there  is  just  one  correct 
solution  path,  a  task  is  considered  less  complex  than 
when  multiple  paths  can  lead  to  a  correct  solution  or 
when  multiple  solutions  are  possible. 

For  the  interaction  category,  Campbell  (1988) 
argued  that  both  the  problem  solver  and  the  task  are 
important.  By  defining  task  complexity  in  terms  of 
cognitive  load  (Chandler  and  Sweller,  1991;  Sweller, 
1988;  Sweller  et  al.,  1998)  an  example  of  this  interac¬ 
tion  can  readily  be  shown.  From  a  cognitive  load  per¬ 
spective,  complexity  is  defined  by  the  number  of  inter¬ 
acting  information  elements  a  task  contains,  which 
have  to  be  simultaneously  handled  in  working  mem¬ 
ory.  As  such,  complexity  is  influenced  by  expertise 
(i.e .,  subjective,  individual  characteristic);  what  may 
be  a  complex  task  for  a  novice  may  be  a  simple  task 
for  an  expert,  because  a  number  of  elements  have  been 
combined  into  a  cognitive  schema  that  can  be  handled 
as  a  single  element  in  the  expert’s  working  memory. 
Tasks  that  are  highly  complex  according  to  the  objec¬ 
tive  definition  (i.e.,  lack  of  structuredness  and  multiple 
possible  solution  paths)  will  also  be  complex  in  the 
interaction  definition;  however,  according  to  the  latter 
definition,  even  tasks  with  a  high  degree  of  structured¬ 
ness  or  one  correct  solution  path  can  be  considered 
complex,  given  a  high  number  of  interacting  informa¬ 
tion  elements  or  low  performer  expertise. 

In  this  chapter  section,  we  limit  our  discussion  to 
methods  of  assessment  of  complex  performance  on 
cognitive  tasks.  What  is  important  to  note  throughout 
this  discussion  is  that  the  methods  described  here  can 
be  used  to  assess  (improvements  in)  complex  perfor¬ 
mance  both  during  training  and  after  training,  depend¬ 
ing  on  the  research  questions  one  seeks  to  address. 
After  training,  performance  assessment  usually  has  the 
goal  to  assess  learning,  which  is  a  goal  of  many  studies 
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in  education  and  instructional  design.  If  one  seeks  to 
assess  learning,  one  must  be  careful  not  to  conclude 
that  participants  have  learned  because  their  perfor¬ 
mance  improved  during  training.  As  Bjork  (1999) 
points  out,  depending  on  the  training  conditions,  high 
performance  gains  during  training  may  not  be  associ¬ 
ated  with  learning,  whereas  low  performance  gains 
may  be.  It  is  important,  therefore,  to  assess  learning 
on  retention  or  transfer  tasks,  instead  of  on  practice 
tasks.  Selection  of  appropriate  assessment  tasks  is  an 
important  issue,  which  is  addressed  in  the  next  section. 

Assessment  Tasks 

An  essential  step  in  the  assessment  of  performance  is 
the  identification  of  a  collection  of  representative  tasks 
that  capture  those  aspects  of  the  participant’s  knowl¬ 
edge  and  skills  that  a  study  seeks  to  address  (Ericsson, 
2002).  Important  factors  for  representativeness  of  the 
collection  of  assessment  tasks  are  authenticity,  num¬ 
ber,  and  duration  of  tasks,  all  of  which  are  highly 
influenced  by  the  characteristics  of  the  domain  of 
study. 

Selecting  tasks  that  adequately  capture  perfor¬ 
mance  often  turns  out  to  be  very  difficult.  Selecting 
atypical  or  artificial  tasks  may  even  impede  learners 
in  demonstrating  their  true  level  of  understanding.  Tra¬ 
ditional  means  to  evaluate  the  learners’  knowledge  or 
skills  have  been  criticized  because  they  often  fail  to 
demonstrate  that  the  learner  can  actually  do  something 
in  real  life  or  in  their  future  workplace  with  their 
knowledge  and  skills  they  have  acquired  during  their 
training  (see,  for  example,  Anderson  et  al.,  1996;  Shep¬ 
ard,  2000;  Thompson,  2001). 

The  argument  for  the  use  of  authentic  tasks  to 
assess  the  learners’  understanding  has  a  long  tradition. 
It  started  in  the  days  of  John  Dewey  (1916)  and  con¬ 
tinues  to  the  present  day  (Merrill,  2002;  van  Merrien- 
boer,  1997);  however,  complete  authenticity  of  assess¬ 
ment  tasks  may  be  difficult  to  realize  in  research 
settings,  because  the  structuredness  of  the  domain 
plays  a  role  here.  For  structured  domains  such  as  chess 
and  bridge,  the  same  conditions  can  be  reproduced  in 
a  research  laboratory  as  those  under  which  perfor¬ 
mance  normally  takes  place;  for  less  or  ill-structured 
domains,  this  is  difficult  or  even  impossible  to  do 
(Ericsson  and  Lehmann,  1996).  Nonetheless,  one  can 
always  strive  for  a  high  degree  of  authenticity.  Gulikers 
et  al.  (2004)  defined  authentic  assessment  as  a  five¬ 
dimensional  construct  (i.e.,  task,  social  context,  phys¬ 
ical  context,  form/result,  and  criteria)  that  can  vary 
from  low  to  high  on  each  of  the  dimensions. 

The  number  of  tasks  in  the  collection  and  the  dura¬ 
tion  are  important  factors  influencing  the  reliability 


and  generalizability  of  a  study.  Choosing  too  few  tasks 
or  tasks  of  too  short  duration  will  negatively  affect 
reliability  and  generalizability.  On  the  other  hand, 
choosing  a  large  number  of  tasks  or  tasks  of  a  very 
long  duration  will  lead  to  many  practical  problems  and 
might  exhaust  both  participants  and  researchers.  In 
many  complex  domains  (e.g.,  medical  diagnosis),  it  is 
quite  common  and  often  inevitable  to  use  a  very  small 
set  of  cases  because  of  practical  circumstances  and 
because  detailed  analysis  of  the  learners’  responses  to 
these  complex  problems  is  very  difficult  and  time  con¬ 
suming  (Ericsson,  2004;  Ericsson  and  Smith,  1991). 
Unfortunately,  however,  there  are  no  golden  rules  for 
determining  the  adequate  number  of  tasks  to  use  or 
their  duration,  because  important  factors  are  highly 
dependent  upon  the  domain  and  specific  context  (Van 
der  Vleuten  and  Schuwirth,  2005).  It  is  often  easier  to 
identify  a  small  collection  of  representative  tasks  that 
capture  the  relevant  aspects  of  performance  in  highly 
structured  domains  (e.g.,  physics,  mathematics,  chess) 
than  in  ill-structured  domains  (e.g.,  political  science, 
medicine),  where  a  number  of  interacting  complex 
skills  are  required. 

Assessment  Criteria  and  Standards 

The  term  assessment  criteria  refers  to  a  description  of 
the  elements  or  aspects  of  performance  that  will  be 
assessed,  and  the  term  assessment  standards  refers  to 
a  description  of  the  quality  of  performance  (e.g.,  excel¬ 
lent/good/average/poor)  on  each  of  those  aspects  that 
can  be  expected  of  participants  at  different  stages  (e.g., 
age,  grade)  (Arter  and  Spandel,  1992).  As  Woolf 
(2004)  pointed  out,  however,  the  term  assessment  cri¬ 
teria  is  often  used  in  the  definition  of  standards  as  well. 
Depending  on  the  question  one  seeks  to  answer,  dif¬ 
ferent  standards  can  be  used,  such  as  a  participant’s 
past  performance  (self-referenced) ,  peer  group  perfor¬ 
mance  (norm-referenced),  or  an  objective  standard 
(criterion-referenced),  and  there  are  different  methods 
for  setting  standards  (Cascallar  and  Cascallar,  2003). 
Much  of  the  research  on  criteria  and  standard  setting 
has  been  conducted  in  the  context  of  educational  prac¬ 
tice  for  national  (or  statewide)  school  tests  (Hambleton 
et  al.,  2000)  and  for  highly  skilled  professions,  such 
as  medicine,  where  the  stakes  of  setting  appropriate 
standards  are  very  high  (Hobma  et  al.,  2004;  Van  der 
Vleuten  and  Schuwirth,  2005).  Although  formulation 
of  good  criteria  and  standards  is  extremely  important 
in  educational  practice,  where  certification  is  the  prime 
goal,  it  is  no  less  important  in  educational  research 
settings.  What  aspects  of  performance  are  measured 
and  what  standards  are  set  have  a  major  impact  on  the 
generalizability  and  value  of  a  study. 
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The  degree  to  which  the  domain  is  well  structured 
influences  not  only  the  creation  of  a  collection  of  rep¬ 
resentative  tasks  but  also  the  definition  of  criteria,  set¬ 
ting  of  standards,  and  interpretation  of  performance  in 
relation  to  standards.  In  highly  structured  domains, 
such  as  mathematics  or  chess,  assessing  the  quality  of 
the  learner’s  response  is  often  fairly  straightforward 
and  unproblematic.  In  less  structured  domains,  how¬ 
ever,  it  is  often  much  more  difficult  to  identify  clear 
standards;  for  example,  a  music  student’s  interpreta¬ 
tion  of  a  piano  concerto  is  more  difficult  to  assess  than 
the  student’s  technical  performance  on  the  piece.  The 
former  contains  many  more  subjective  elements  (e.g., 
taste)  or  cultural  differences  than  the  latter. 

Collecting  Performance  Data 

No  one  best  method  for  complex  performance  assess¬ 
ment  exists,  and  it  is  often  advisable  to  use  multiple 
measures  or  methods  in  combination  to  obtain  as  com¬ 
plete  a  picture  as  possible  of  the  performance.  A  num¬ 
ber  of  methods  are  described  here  for  collecting  per¬ 
formance  outcome  (product)  and  performance  process 
data.  Methods  are  classified  as  online  (during  task 
performance)  or  offline  (after  task  performance). 
Which  method  or  combination  of  methods  is  the  most 
useful  depends  on  the  particular  research  question,  the 
possible  constraints  of  the  research  context,  and  the 
domain.  In  ill-structured  domains,  for  example,  the 
added  value  of  process  measures  may  be  much  higher 
than  in  highly  structured  domains. 

Collecting  Performance  Outcome  ( Product )  Data 

Collecting  performance  outcome  data  is  quite  straight¬ 
forward.  One  takes  the  product  of  performance  (e.g.,  an 
electrical  circuit  that  was  malfunctioning  but  is  now 
repaired)  and  scores  it  along  the  defined  criteria  (e.g., 
do  all  the  components  function  as  they  should,  individ¬ 
ually  and  as  a  whole?).  Instead  of  assigning  points  for 
correct  aspects,  one  can  count  the  number  of  errors,  and 
analyze  the  types  of  errors  made;  however,  especially 
for  assessment  of  complex  performance,  collecting  per¬ 
formance  product  data  alone  is  not  very  informative. 
Taking  into  account  the  process  leading  up  to  the  prod¬ 
uct  and  the  cognitive  costs  at  which  it  was  obtained 
provides  equally  if  not  more  important  information. 

Collecting  Performance  Process  Data 

Time  on  Task  or  Speed 

An  important  indication  of  the  level  of  mastery  of  a 
particular  task  is  the  time  needed  to  complete  a  task. 
According  to  the  power  law  of  practice  (Newell  and 


Rosenbloom,  1981;  VanLehn,  1996),  the  time  needed 
to  complete  a  task  decreases  in  proportion  to  the  time 
spent  in  practice,  raised  to  some  power.  Newell  and 
Rosenbloom  (1981)  found  that  this  law  operates  across 
a  broad  range  of  tasks,  from  solving  geometry  prob¬ 
lems  to  keyboard  typing.  To  account  for  the  power  law 
of  practice,  several  theories  have  been  put  forward. 
Anderson’s  ACT-R  explains  the  speed-up  by  assuming 
that  slow  declarative  knowledge  is  transformed  into 
fast  procedural  knowledge  (Anderson,  1993;  Anderson 
and  Lebiere,  1998).  Another  explanation  suggested 
that  speed-up  is  the  result  of  repeated  encounters  with 
meaningful  patterns  (Ericsson  and  Staszewski,  1989); 
that  is,  as  a  result  of  frequent  encounters  with  similar 
elements,  these  elements  will  no  longer  be  perceived 
as  individual  units  but  will  be  perceived  as  a  meaning¬ 
ful  whole  (i.e.,  chunk).  In  addition  to  chunking,  auto¬ 
mation  processes  (Schneider  and  Shiffrin,  1977;  Shif- 
frin  and  Schneider,  1977)  occur  with  practice  that 
allow  for  faster  and  more  effortless  performance.  In 
summary,  as  expertise  develops  equal  performance  can 
be  attained  in  less  time;  therefore,  it  is  important  to 
collect  time-on-task  data  to  assess  improvements  in 
complex  performance. 

Cognitive  Load 

The  same  processes  of  chunking  and  automation  that 
are  associated  with  decreases  in  the  time  required  to 
perform  a  task  are  also  responsible  for  decreases  in 
the  cognitive  load  imposed  by  performing  the  task 
(Paas  and  van  Merrienboer,  1993;  Yeo  and  Neal ,  2004) . 
Cognitive  load  can  be  measured  using  both  online  and 
offline  techniques.  The  cognitive  capacity  that  is  allo¬ 
cated  to  performing  the  task  is  defined  as  mental  effort , 
which  is  considered  to  reflect  the  actual  cognitive  load 
a  task  imposes  (Paas  and  van  Merrienboer,  1994a;  Paas 
et  al.,  2003).  A  subjective  but  reliable  technique  for 
measuring  mental  effort  is  having  individuals  provide 
self-ratings  of  the  amount  of  mental  effort  invested.  A 
single-scale  subjective  rating  instrument  can  be  used, 
such  as  the  nine-point  rating  scale  developed  by  Paas 
(1992),  or  a  multiple-scale  instrument,  such  as  the 
NASA  Task  Load  Index  (TLX),  which  was  used,  for 
example  by  Gerjets  et  al.  (2004,  2006).  As  subjective 
cognitive  load  measures  are  usually  recorded  after 
each  task  or  after  a  series  of  tasks  has  been  completed 
they  are  usually  considered  to  be  offline  measure¬ 
ments,  although  there  are  some  exceptions;  for  exam¬ 
ple,  Ayres  (2006)  required  participants  to  rate  cogni¬ 
tive  load  at  specific  points  within  tasks. 

Objective  online  measures  include  physiological 
measures  such  as  heart-rate  variability  (Paas  and  van 
Merrienboer,  1994b),  eye-movement  data,  and  second¬ 
ary-task  procedures  (Briinken  et  al.,  2003).  Because 
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they  are  taken  during  task  performance,  those  online 
measures  can  show  fluctuations  in  cognitive  load  dur¬ 
ing  task  performance.  It  is  notable,  however,  that  Paas 
and  van  Merrienboer  (1994b)  found  the  heart-rate  vari¬ 
ability  measure  to  be  quite  intrusive  as  well  as  insen¬ 
sitive  to  subtle  fluctuations  in  cognitive  load.  The  sub¬ 
jective  offline  data  are  often  easier  to  collect  and 
analyze  and  provide  a  good  indication  of  the  overall 
cognitive  load  a  task  imposed  (Paas  et  al.,  2003). 

Actions:  Observation  and  Video  Records 

Process-tracing  techniques  are  very  well  suited  to 
assessing  the  different  types  of  actions  taken  during 
task  performance,  some  of  which  are  purely  cognitive, 
whereas  others  result  in  physical  actions,  because  the 
“data  that  are  recorded  are  of  a  pre-specified  type  (e.g., 
verbal  reports,  eye  movements,  actions)  and  are  used 
to  make  inferences  about  the  cognitive  processes  or 
knowledge  underlying  task  performance”  (Cooke, 
1994,  p.  814).  Ways  to  record  data  that  allow  the  infer¬ 
ence  of  cognitive  actions  are  addressed  in  the  follow¬ 
ing  sections.  The  following  options  are  available  for 
recording  the  physical  actions  taken  during  task  per¬ 
formance:  (1)  trained  observers  can  write  down  the 
actions  taken  or  check  them  off  on  an  a  priori  con¬ 
structed  list  (use  multiple  observers),  (2)  a  (digital) 
video  record  of  the  participants’  performance  can  be 
made,  or  (3)  for  computer-based  tasks,  an  action  record 
can  be  made  using  screen  recording  software  or  soft¬ 
ware  that  logs  key  presses  and  coordinates  of  mouse 
clicks. 

Attention  and  Cognitive  Actions: 

Eye-Movement  Records 

Eye  tracking  (Duchowski,  2003)— that  is,  recording 
eye-movement  data  while  a  participant  is  working  on 
a  (usually,  but  not  necessarily  computer-based) 
task— can  also  be  used  to  gather  online  performance 
process  data  but  is  much  less  used  in  educational 
research  than  the  above  methods .  Eye-movement  data 
give  insights  into  the  allocation  of  attention  and  pro¬ 
vide  a  researcher  with  detailed  information  of  what 
a  participant  is  looking  at,  for  how  long,  and  in  what 
order.  Such  data  allow  inferences  to  be  made  about 
cognitive  processes  (Rayner,  1998),  albeit  cautious 
inferences,  as  the  data  do  not  provide  information  on 
why  a  participant  was  looking  at  something  for  a 
certain  amount  of  time  or  in  a  certain  order.  Attention 
can  shift  in  response  to  exogenous  or  endogenous 
cues  (Rayner,  1998;  Stelmach  et  al.,  1997).  Exoge¬ 
nous  shifts  of  attention  occur  mainly  in  response  to 
environmental  features  or  changes  in  the  environment 
(e.g.,  if  something  brightly  colored  would  start  flash¬ 
ing  in  the  corner  of  a  computer  screen,  your  attention 


would  be  drawn  to  it).  Endogenous  shifts  are  driven 
by  knowledge  of  the  task,  of  the  environment,  and  of 
the  importance  of  available  information  sources  (i.e., 
influenced  by  expertise  level)  (Underwood  et  al., 
2003).  In  chess,  for  example,  it  was  found  that  experts 
fixated  proportionally  more  on  relevant  pieces  than 
non-expert  players  (Charness  et  al.,  2001).  In  elec¬ 
trical  circuits  troubleshooting,  van  Gog  et  al.  (2005a) 
also  found  that  participants  with  higher  expertise  fix¬ 
ated  more  on  a  fault-related  component  during  prob¬ 
lem  orientation  than  participants  with  lower  exper¬ 
tise.*  Please  note  that  this  is  not  an  exhaustive 
overview  and  that  we  have  no  commercial  or  other 
interest  in  any  of  the  programs  mentioned  here. 
Haider  and  Frensch  (1999)  used  eye-movement  data 
to  corroborate  their  information-reduction  hypothe¬ 
sis,  which  states  that  with  practice  people  learn  to 
ignore  task-redundant  information  and  limit  their 
processing  to  task-relevant  information. 

On  tasks  with  many  visual  performance  aspects 
(e.g.,  troubleshooting  technical  systems),  eye-move¬ 
ment  records  may  therefore  provide  much  more  infor¬ 
mation  than  video  records.  Some  important  problem¬ 
solving  actions  may  be  purely  visual  or  cognitive,  but 
those  will  show  up  in  an  eye-movement  record, 
whereas  a  video  record  will  only  allow  inferences  of 
visual  or  cognitive  actions  that  resulted  in  manual  or 
physical  actions  (van  Gog  et  al.,  2005b).  In  addition 
to  providing  information  on  the  allocation  of  attention, 
eye-movement  data  can  also  give  information  about 
the  cognitive  load  that  particular  aspects  of  task  per¬ 
formance  impose;  for  example,  whereas  pupil  dilation 
(Van  Gerven  et  al.,  2004)  and  fixation  duration  (Under¬ 
wood  et  al.,  2004)  are  known  to  increase  with  increased 
processing  demands,  the  length  of  saccades  (i.e.,  rapid 
eye  movements  from  one  location  to  another;  see 
Duchowski,  2003)  is  known  to  decrease.  (For  an  in- 
depth  discussion  of  eye-movement  data  and  cognitive 
processes,  see  Rayner,  1998.) 

Thought  Processes  and  Cognitive 
Actions:  Verbal  Reports 

Probably  the  most  widely  used  verbal  reporting  tech¬ 
niques  are  concurrent  and  retrospective  reporting 
(Ericsson  and  Simon,  1993).  As  their  names  imply, 
concurrent  reporting  is  an  online  technique,  whereas 
retrospective  reporting  is  an  offline  technique.  Concur¬ 
rent  reporting,  or  thinking  aloud ,  requires  participants 
to  verbalize  all  thoughts  that  come  to  mind  during  task 


*  Note  that  the  expertise  differences  between  groups  were  relatively 
small  (i.e.,  this  was  not  an  expert-novice  study),  suggesting  that  eye- 
movement  data  may  be  a  useful  tool  in  investigating  relatively  subtle 
expertise  differences  or  expertise  development. 
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performance.  Retrospective  reporting  requires  partici¬ 
pants  to  report  the  thoughts  they  had  during  task  per¬ 
formance  immediately  after  completing  it.  Although 
there  has  been  considerable  debate  over  the  use  of 
verbal  reports  as  data,  both  methods  are  considered  to 
allow  valid  inferences  to  be  made  about  the  cognitive 
processes  underlying  task  performance,  provided  that 
verbalization  instructions  and  prompts  are  carefully 
worded  (Ericsson  and  Simon,  1993). 

Instructions  and  prompts  should  be  worded  in 
such  a  way  that  the  evoked  responses  will  not  inter¬ 
fere  with  the  cognitive  processes  as  they  occur  during 
task  performance;  for  example,  instructions  for  con¬ 
current  reporting  should  tell  participants  to  think 
aloud  and  verbalize  everything  that  comes  to  mind 
but  should  not  ask  them  to  explain  any  thoughts. 
Prompts  should  be  as  unobtrusive  as  possible. 
Prompting  participants  to  “keep  thinking  aloud”  is 
preferable  over  asking  them  “what  are  you  thinking?” 
because  this  would  likely  evoke  self-reflection  and, 
hence,  interfere  with  the  cognitive  processes.  Devia¬ 
tions  from  these  instructional  and  prompting  tech¬ 
niques  can  change  either  the  actual  cognitive  pro¬ 
cesses  involved  or  the  processes  that  were  reported, 
thereby  compromising  the  validity  of  the  reports 
(Boren  and  Ramey,  2000;  Ericsson  and  Simon,  1993). 
Magliano  et  al.  (1999),  for  example,  found  that 
instructions  to  explain,  predict,  associate,  or  under¬ 
stand  during  reading  influenced  the  inferences  from 
the  text  that  participants  generated  while  thinking 
aloud.  Although  the  effect  of  instructions  on  cogni¬ 
tive  processes  is  an  interesting  topic  of  study,  when 
the  intention  is  to  elicit  reports  of  the  actual  cognitive 
processes  as  they  would  occur  without  intervention, 
Ericsson  and  Simon’s  (1993)  guidelines  for  wording 
instructions  and  prompts  should  be  adhered  to. 

Both  reporting  methods  can  result  in  verbal  proto¬ 
cols  that  allow  for  valid  inferences  about  cognitive 
processes;  however,  the  potential  for  differences  in  the 
information  they  contain  must  be  considered  when 
choosing  an  appropriate  method  for  answering  a  par¬ 
ticular  research  question.  According  to  Taylor  and 
Dionne  (2000),  concurrent  protocols  mostly  seem  to 
provide  information  on  actions  and  outcomes,  whereas 
retrospective  protocols  seem  to  provide  more  informa¬ 
tion  about  “strategies  that  control  the  problem  solving 
process”  and  “conditions  that  elicited  a  particular 
response”  (p.  414).  Kuusela  and  Paul  (2000)  reported 
that  concurrent  protocols  contained  more  information 
than  retrospective  protocols,  because  the  latter  often 
contained  only  references  to  the  effective  actions  that 
led  to  the  solution,  van  Gog  et  al.  (2005b)  investigated 
whether  the  technique  of  cued  retrospective  reporting, 
in  which  a  retrospective  report  is  cued  by  a  replay  of 


a  record  of  eye  movements  and  mouse/keyboard  oper¬ 
ations  made  during  the  task,  would  combine  the  advan¬ 
tages  of  concurrent  (i.e.,  more  action  information)  and 
retrospective  (i.e.,  more  strategic  and  conditional  infor¬ 
mation)  reporting.  They  found  that  both  concurrent  and 
cued  retrospective  reporting  resulted  in  more  action 
information,  as  well  as  in  more  strategic  and  condi¬ 
tional  information,  than  retrospective  reporting  with¬ 
out  a  cue. 

Contrary  to  expectations,  concurrent  reporting 
resulted  in  more  strategic  and  conditional  information 
than  retrospective  reporting.  This  may  (1)  reflect  a 
genuine  difference  from  Taylor  and  Dionne’s  results, 

(2)  have  been  due  to  different  operationalizations  of 
the  information  types  in  the  coding  scheme  used,  or 

(3)  have  been  due  to  the  use  of  a  different  segmentation 
method  than  those  used  by  Taylor  and  Dionne  (2000). 

An  explanation  for  the  finding  that  concurrent 
reports  result  in  more  information  on  actions  than 
retrospective  reports  may  be  that  concurrent  reporting 
occurs  online  rather  than  offline.  Whereas  concurrent 
reports  capture  information  available  in  short-term 
memory  during  the  process,  retrospective  reports 
reflect  memory  traces  of  the  process  retrieved  from 
short-term  memory  when  tasks  are  of  very  short  dura¬ 
tion  or  from  long-term  memory  when  tasks  are  of 
longer  duration  (Camps,  2003;  Ericsson  and  Simon, 
1993).  It  is  likely  that  only  the  correct  steps  that  have 
led  to  attainment  of  the  goal  are  stored  in  long-term 
memory,  because  only  these  steps  are  relevant  for 
future  use.  This  is  why  having  participants  report 
retrospectively  based  on  a  record  of  observations  or 
intermediate  products  of  their  problem-solving  pro¬ 
cess  is  known  to  lead  to  better  results  (due  to  fewer 
omissions)  than  retrospective  reporting  without  a  cue 
(van  Gog  et  al.,  2005b;  Van  Someren  et  al.,  1994). 
Possibly,  the  involvement  of  different  memory  sys¬ 
tems  might  also  explain  Taylor  and  Dionne’s  (2000) 
finding  that  retrospective  protocols  seem  to  contain 
more  conditional  and  strategic  information.  This 
knowledge  might  have  been  used  during  the  process 
but  may  have  been  omitted  in  concurrent  reporting 
as  a  result  of  the  greater  processing  demands  this 
method  places  on  short-term  memory  (Russo  et  al., 
1989).  Although  this  explanation  is  tentative,  there 
are  indications  that  concurrent  reporting  may  become 
difficult  to  maintain  under  high  cognitive  load  con¬ 
ditions  (Ericsson  and  Simon,  1993).  Indeed,  partici¬ 
pants  in  van  Gog  et  al.’s  study  who  experienced  a 
higher  cognitive  load  (i.e.,  reported  investment  of 
more  mental  effort)  in  performing  the  tasks  indicated 
during  a  debriefing  after  the  experiment  that  they 
disliked  concurrent  reporting  and  preferred  cued  ret¬ 
rospective  reporting  (van  Gog,  2006). 
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Neuroscientific  Data 

An  emerging  and  promising  area  of  educational 
research  is  the  use  of  neuroscience  methodologies  to 
study  (changes  in)  brain  functions  and  structures 
directly,  which  can  provide  detailed  data  on  learning 
processes,  memory  processes,  and  cognitive  develop¬ 
ment  (see,  for  example,  Goswami,  2004;  Katzir  and 
Pare-Blagoev,  2006).  Methods  such  as  magnetic  reso¬ 
nance  imaging  (MRI),  functional  magnetic  resonance 
imaging  (fMRI), electroencephalography  (EEG), mag¬ 
netoencephalography  (MEG),  positron-emission 
tomography  (PET),  and  single-photon  emission  com¬ 
puted  tomography  (SPECT)  provide  (indirect)  mea¬ 
sures  of  neuronal  activity.  The  reader  is  referred  to 
Katzir  and  Pare-Blagoev  (2006)  for  a  discussion  of 
these  methods  and  examples  of  their  use  in  educational 
research. 

Data  Analysis 

Analyzing  performance  product,  time  on  task,  and 
mental  effort  data  (at  least  when  the  subjective  rating 
scales  are  used)  is  a  very  straightforward  process,  so 
it  is  not  discussed  here.  In  this  section,  analysis  of 
observation,  eye  movement,  and  verbal  protocol  data 
is  discussed,  as  well  as  the  analysis  of  combined  meth¬ 
ods/measures. 

Analysis  of  Observation,  Eye  Movement, 
and  Verbal  Protocol  Data 

Observation  Data 

Coding  and  analysis  of  observation  data  can  take  many 
different  forms ,  again  depending  on  the  research  ques¬ 
tion.  Coding  schemes  are  developed  based  on  the  per¬ 
formance  aspects  (criteria)  one  wishes  to  assess  and 
sometimes  may  incorporate  evaluation  of  performance 
aspects.  Whether  coding  is  done  online  (during  per¬ 
formance  by  observers)  or  offline  (after  performance 
based  on  video,  screen  capture,  or  mouse-keyboard 
records),  the  use  of  multiple  observers  or  raters  is 
important  for  determining  reliability  of  the  coding. 
Quantitative  analysis  on  the  coded  data  can  take  the 
form  of  comparison  of  frequencies,  appropriateness 
(e.g.,  number  of  errors),  or  sequences  of  actions  (e.g., 
compared  to  an  ideal  or  expert  sequence)  and  inter¬ 
preting  the  outcome  in  relation  to  the  set  standard. 

Several  commercial  and  noncommercial  software 
programs  have  been  developed  to  assist  in  the  analysis 
of  action  data;*  for  example.  Observer  (Noldus  et  al.. 


*  Please  note  that  this  is  not  an  exhaustive  overview  and  that  we  have 
no  commercial  or  other  interest  in  any  of  the  programs  mentioned  here. 


2000)  is  commercial  software  for  coding  and  analysis 
of  digital  video  records;  NVivo  (Bazeley  and  Richards, 
2000)  is  commercial  software  for  accessing,  shaping, 
managing,  and  analyzing  non-numerical  qualitative 
data;  Multiple  Episode  Protocol  Analysis  (MEPA) 
(Erkens,  2002)  is  free  software  for  annotating,  coding, 
and  analyzing  both  nonverbal  and  verbal  protocols; 
and  ACT  Pro  (Fu,  2001)  can  be  used  for  sequential 
analysis  of  protocols  of  discrete  user  actions  such  as 
mouse  clicks  and  key  presses. 

Eye-Movement  Data 

For  analysis  of  fixation  data  it  is  important  to  identify 
the  gaze  data  points  that  together  represent  fixations. 
This  is  necessary  because  during  fixation  the  eyes  are 
not  entirely  motionless;  small  tremors  and  drifts  may 
occur  (Duchowski,  2003).  According  to  Salvucci 
(1999),  the  three  categories  of  fixation  identification 
methods  are  based  on  velocity,  dispersion,  or  region. 
Most  eye-tracking  software  allows  for  defining  val¬ 
ues  for  the  dispersion -based  method,  which  identifies 
fixation  points  as  a  minimum  number  of  data  points 
that  are  grouped  closely  together  (i.e.,  fall  within  a 
certain  dispersion,  defined  by  pixels)  and  last  a  min¬ 
imum  amount  of  time  (duration  threshold).  Once  fix¬ 
ations  have  been  defined,  defining  areas  of  interest 
(Aols)  in  the  stimulus  materials  will  make  analysis 
of  the  huge  data  files  more  manageable  by  allowing 
summaries  of  fixation  data  to  be  made  for  each  Aol, 
such  as  the  number  of  fixations,  the  mean  fixation 
duration,  and  the  total  time  spent  fixating.  Further¬ 
more,  a  chronological  listing  of  fixations  on  Aols  can 
be  sequentially  analyzed  to  detect  patterns  in  viewing 
behavior. 

Verbal  Protocol  Data 

When  verbal  protocols  have  been  transcribed,  they  can 
be  segmented  and  coded.  Segmentation  based  on  utter¬ 
ances  is  highly  reliable  because  it  uses  pauses  in  nat¬ 
ural  speech  (Ericsson  and  Simon,  1993);  however, 
many  researchers  apply  segmentation  based  on  mean¬ 
ing  (Taylor  and  Dionne,  2000).  In  this  case,  segmen¬ 
tation  and  coding  become  intertwined,  and  the  reliabil¬ 
ity  of  both  should  be  evaluated.  It  is,  again,  important 
to  use  multiple  raters  (at  least  on  a  substantial  subset 
of  data)  and  determine  the  reliability  of  the  coding 
scheme.  The  standard  work  by  Ericsson  and  Simon 
(1993)  provides  a  wealth  of  information  on  verbal 
protocol  coding  and  analysis  techniques.  The  software 
program  MEPA  (Erkens,  2002)  can  assist  in  the  devel¬ 
opment  of  a  coding  scheme  for  verbal  data,  as  well  as 
in  analysis  of  coded  data  with  a  variety  of  quantitative 
or  qualitative  methods. 
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Combining  Methods  and  Measures 

As  mentioned  before,  there  is  not  a  preferred  single 
method  for  the  assessment  of  complex  performances. 
By  combining  different  methods  and  measures,  a  more 
complete  or  a  more  detailed  picture  of  performance 
will  be  obtained;  for  example,  various  process-tracing 
techniques  such  as  eye  tracking  and  verbal  reporting 
can  be  collected  and  analyzed  in  combination  with 
other  methods  of  assessment  (van  Gog  et  al.,  2005a). 
Different  product  and  process  measures  can  easily  be 
combined,  and  it  can  be  argued  that  some  of  them 
should  be  combined  because  a  simple  performance 
score*  ignores  the  fact  that,  with  expertise  develop¬ 
ment,  time  on  task  and  cognitive  load  decrease, 
whereas  performance  increases. 

Consider  the  example  of  a  student  who  attains  the 
same  performance  score  on  two  comparable  tasks  that 
are  spread  over  time,  where  cognitive  load  measures 
indicate  that  the  learner  had  to  invest  a  lot  of  mental 
effort  to  complete  the  task  the  first  time  and  little  the 
second.  Looking  only  at  the  performance  score,  one 
might  erroneously  conclude  that  no  progress  was 
made,  whereas  the  learner  actually  made  a  subtle  step 
forward,  because  reduced  cognitive  load  means  that 
more  capacity  can  be  devoted  to  further  learning. 

The  mental  efficiency  measure  developed  by  Paas 
and  van  Merrienboer  (1993)  reflects  this  relation: 
Higher  performance  with  less  mental  effort  invested 
to  attain  that  performance  results  in  higher  efficiency. 
This  measure  is  obtained  by  standardizing  perfor¬ 
mance  and  mental  effort  scores,  and  then  subtracting 
the  mean  standardized  mental  effort  score  (zE)  from 
the  mean  standardized  performance  score  (zP)  and 
dividing  the  outcome  by  the  square  root  of  2: 

zP-zE 

When  tasks  are  performed  under  time  constraints,  the 
combination  of  mental  effort  and  performance  mea¬ 
sures  will  suffice;  however,  when  time  on  task  is  self- 
paced,  it  is  useful  to  include  the  additional  time  param¬ 
eter  in  the  efficiency  measure  (making  it  three-dimen¬ 
sional)  (Paas  et  al.,  2003;  Tuovinen  and  Paas,  2004): 

zP-zE-  z,T 
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*  This  term  is  somewhat  ambiguous,  as  we  have  previously  classified 
mental  effort  and  time-on-task  data  as  performance  process  data.  We 
feel  they  should  be  regarded  as  such;  however,  in  the  literature  perfor¬ 
mance  score  is  often  used  to  refer  to  the  grade  assigned  to  a  solution 
or  solution  procedure,  which  is  the  sense  in  which  the  term  is  used  in 
this  subsection. 


Discussion 

Much  of  the  research  into  learning  and  instruction 
involves  assessment  of  complex  performances  of  cog¬ 
nitive  tasks.  The  focus  of  this  chapter  section  was  on 
data  collection  and  analysis  methods  that  can  be  used 
for  such  assessments.  First,  the  important  issues  related 
to  selecting  an  appropriate  collection  of  assessment 
tasks  and  defining  appropriate  assessment  criteria  and 
standards  were  discussed.  Then,  different  ways  for 
collecting  performance  product  and  process  data, 
using  online  (during  task  performance)  or  offline  (after 
task  performance)  measurements,  were  described. 
Analysis  techniques  were  discussed  and,  given  the  lack 
of  a  single  preferred  method  for  complex  performance 
assessment,  ways  to  combine  measures  were  suggested 
that  will  foster  a  more  complete  or  more  detailed 
understanding  of  complex  performance. 

This  chapter  section  aimed  to  provide  an  overview 
of  the  important  issues  in  assessment  of  complex  per¬ 
formance  on  cognitive  tasks  and  of  available  data  col¬ 
lection  and  analysis  techniques  for  such  assessments, 
rather  than  any  definite  guidelines.  The  latter  would 
be  impossible  when  writing  for  a  broad  audience, 
because  what  constitutes  an  appropriate  collection  of 
tasks,  appropriate  criteria  and  standards,  and  appropri¬ 
ate  data  collection  and  analysis  techniques  is  highly 
dependent  on  the  research  question  one  seeks  to 
address  and  on  the  domain  in  which  one  wishes  to  do 
so.  We  hope  that  this  overview,  along  with  other  chap¬ 
ter  sections,  provides  the  reader  with  a  starting  point 
for  further  development  of  rewarding  and  informative 
studies. 


SETTING  UP  A  LABORATORY 
FOR  MEASUREMENT  OF 
COMPLEX  PERFORMANCES 

Aaron  R.  Duley,  Paul  Ward,  and  Peter  A.  Hancock 

This  chapter  section  describes  how  to  set  up  laborato¬ 
ries  for  the  measurement  of  complex  performance. 
Complex  performance  in  this  context  does  not  exclu¬ 
sively  refer  to  tasks  that  are  inherently  difficult  to 
perform;  rather,  the  term  is  used  here  in  a  broader  sense 
to  refer  to  the  measurement  of  real-world  activities  that 
require  the  integration  of  disparate  measurement 
instrumentation  as  well  as  the  need  for  time-critical 
experimental  control.  We  have  assumed  that  our  pri¬ 
mary  readership  is  comprised  of  graduate  students  and 
research  faculty,  although  the  chapter  addresses  issues 
relevant  to  all  who  seek  a  better  understanding  of 
behavioral  response. 
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The  central  theme  of  this  section  relates  to  labora¬ 
tory  instrumentation.  Because  instrumentation  is  a  req¬ 
uisite  element  for  complex  performance  measurement, 
a  common  problem  encountered  by  researchers  is  how 
to  overcome  the  various  technical  hurdles  that  often 
discourage  the  pursuit  of  difficult  research  objectives. 
Thus,  creating  a  testing  environment  suitable  to 
address  research  questions  is  a  major  issue  when  plan¬ 
ning  any  research  program;  however,  searching  the 
literature  for  resources  relating  to  laboratory  instru¬ 
mentation  configurations  yields  a  surprisingly  scant 
number  of  references  and  resources  that  address  these 
issues.  Having  made  just  such  an  attempt  for  the  pur¬ 
poses  of  this  section,  the  ability  to  articulate  a  general- 
purpose  exposition  on  laboratory  setup  is  indeed  a 
challenging  endeavor.  This  pursuit  is  made  more  dif¬ 
ficult  by  addressing  a  naturally  ambiguous  topic  such 
as  complex  performance;  nevertheless,  our  section 
looks  to  provide  the  bearings  needed  to  resolve  such 
questions.  In  particular,  we  cover  stimulus  presentation 
and  control  alternatives,  as  well  as  hardware  choices 
for  signal  routing  and  triggering,  while  offering  solu¬ 
tions  for  commonly  encountered  problems  when 
attempting  to  assemble  such  a  laboratory.  Some  por¬ 
tions  of  this  section  are  moderately  technical,  but  every 
attempt  has  been  made  to  ensure  that  the  content  is 
appropriate  for  our  target  audience. 

Instrumentation  and 
Common  Configurations 

Psychology  has  a  long  legacy  of  employing  tools  and 
instrumentation  to  support  scientific  inquiry.  The  online 
Museum  of  the  History  of  Psychological  Instrumenta¬ 
tion,  for  example,  has  illustrations  of  over  150  devices 
used  by  early  researchers  to  visualize  organ  function 
and  systematically  investigate  human  psychological 
processes  and  behavior  (see  http://www.chss. montclair. 
edu/psychology/museum/museum.htm).  At  this 
museum,  one  can  view  such  devices  as  an  early  Wundt- 
style  tachistiscope  or  the  Rotationsapparatus  fur  Komp- 
likations-Versuche  (rotary  apparatus  for  complication 
studies).  Titchener,  a  student  of  Wundt,  continued  this 
tradition  in  his  own  laboratory  at  Cornell  University 
and  described  the  building  requirements  and  the  costs 
associated  with  items  needed  for  establishing  the  ideal 
psychological  laboratory  (Titchener,  1900,  pp. 
252-253): 

For  optics,  there  should  be  two  rooms,  light  and  dark, 
facing  south  and  north  respectively,  and  the  later  divided 
into  antechamber  and  inner  room.  For  acoustics,  there 
should  be  one  large  room,  connected  directly  with  a  small, 
dark,  and  (so  far  as  is  possible  without  special  construc¬ 


tion)  sound-proof  chamber.  For  haptics,  there  should  be 
a  moderately  sized  room,  devoted  to  work  on  cutaneous 
pressure,  temperature,  and  pain,  and  a  larger  room  for 
investigations  of  movement  perceptions.  Taste  and  smell 
should  each  have  a  small  room,  the  latter  tiled  or  glazed, 
and  so  situated  that  ventilation  is  easy  and  so  does  not 
involve  the  opening  of  doors  or  transom-windows  into  the 
building.  There  should,  further,  be  a  clock-room,  for  the 
time-registering  instruments  and  their  controls;  and  a 
large  room  for  the  investigations  of  the  bodily  processes 
and  changes  underlying  affective  consciousness. 

Instrumentation  is  a  central  component  of  complex 
performance  measurement;  however,  the  process  by 
which  one  orchestrates  several  devices  in  the  broader 
context  of  addressing  an  experimental  question  is 
indeed  challenging.  Modem-day  approaches  reflect  a 
paradigm  shift  with  respect  to  early  psychological  pro¬ 
cedures.  Traditionally,  a  single  instrument  would  be 
used  for  an  entire  experiment.  Complex  performance 
evaluation,  however,  often  entails  situations  where  the 
presentation  of  a  stimulus  is  controlled  by  one  com¬ 
puter,  while  supplementary  instrumentation  collects  a 
stream  of  other  data  on  a  second  or  perhaps  yet  a  third 
computer.  Certainly,  an  ideal  testing  solution  would 
allow  one  to  minimize  the  time  needed  to  set  up  an 
experiment  and  maximize  the  experimental  degree  of 
automation,  thus  minimizing  investigator  intervention, 
without  compromising  the  scientific  integrity  of  the 
experiment.  Nevertheless,  the  measurement  of  com¬ 
plex  performance  is  often  in  conflict  with  this  idyllic 
vision.  It  is  not  sufficient  for  contemporary  researchers 
simply  to  design  experiments.  They  are  also  required 
to  have  access  to  the  manpower  and  the  monetary  or 
computational  resources  necessary  to  translate  a  sci¬ 
entific  question  into  a  tenable  methodological  test  bed. 

Design  Patterns  for 
Laboratory  Instrumentation 

Design  patterns  represent  structured  solutions  for  such 
recurring  assessment  problems  (Gamma  et  al.,  1995). 
The  formal  application  of  design  patterns  as  abstract 
blueprints  for  common  challenges  has  relevance  for 
laboratory  instrumentation  configuration  and  equip¬ 
ment  purchasing  decisions.  Although  research  ques¬ 
tions  vary,  experiments  will  regularly  share  a  compa¬ 
rable  solution.  These  commonalities  are  important  to 
identify,  as  the  ability  to  employ  a  single  set  of  tools 
has  distinct  advantages  compared  to  solutions  tailored 
for  only  one  particular  problem.  Such  advantages 
include  cost  savings,  instrumentation  sharing,  instru¬ 
mentation  longevity,  and  laboratory  scalability  (e.g.,  the 
capacity  to  run  multiple  experiments  simultaneously). 
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Figure  55.2  Stimulus  and  presentation  control  model. 

The  purpose  of  the  following  section  is  to  provide 
a  level  of  abstraction  for  instrumentation  configura¬ 
tions  commonly  encountered  in  the  design  of  experi¬ 
ments  related  to  complex  performance.  This  approach 
is  favored  beyond  simply  providing  a  list  of  items  and 
products  that  every  laboratory  should  own.  We 
acknowledge  the  considerable  between-laboratory 
variability  regarding  research  direction,  instrumenta¬ 
tion,  and  expertise;  therefore,  we  focus  primarily  on 
instrumentation  configuration  and  architecture  as  rep¬ 
resented  by  design  patterns  common  to  a  broad  array 
of  complex  performance  manipulations. 

Given  that  an  experiment  will  often  require  the 
manipulation  of  stimuli  in  a  structured  way  and  seeing 
the  impracticality  in  comprehensively  covering  all 
research  design  scenarios,  the  following  assumptions 
are  made:  (1)  Stimuli  are  physically  presented  to  par¬ 
ticipants,  (2)  some  stimulus  properties  are  required  to 
be  under  experimental  control  (e.g.,  presentation 
length),  (3)  measurable  responses  by  the  participant 
are  required,  and  (4)  control  or  communication  of  sec¬ 
ondary  instrumentation  may  also  be  necessary.  These 
assumptions  accommodate  a  broad  spectrum  of  possi¬ 
ble  designs  and  from  these  assumptions  several  frame¬ 
works  can  be  outlined. 

Stimulus  Presentation  and  Control  Model 

Figure  55.2  depicts  the  simplest  of  the  design  patterns, 
which  we  term  the  stimulus  presentation  and  control 
(SPC)  model.  The  SPC  model  is  a  building  block  for 
more  complex  configurations.  The  basic  framework  of 
the  SPC  model  includes  the  presentation  layer  and  the 
stimulus ,  control ,  and  response  layer.  The  presentation 
layer  represents  the  medium  used  to  physically  display 
a  stimulus  to  a  participant  (e.g.,  monitor,  projector. 


Example 


Monitor 


|  VGA  or  DVI 
Computing  Target 

SCRL  Application 


speaker,  headphones).  The  stimulus,  control,  and 
response  layer  (SCRL)  encapsulates  a  number  of  inter¬ 
related  functions  central  to  complex  performance 
experimentation,  such  as  the  experimental  protocol 
logic,  and  is  the  agent  that  coordinates  and  controls 
experimental  flow  and,  potentially,  participant 
response.  Broadly  speaking,  SCRL-type  roles  include 
stimulus  manipulation  and  timing,  instrument  logging 
and  coordination,  and  response  logging,  in  addition  to 
experiment  procedure  management. 

As  the  SCRL  often  contains  the  logic  necessary  to 
execute  the  experimental  paradigm,  it  is  almost  always 
implemented  in  software;  thus,  the  SCRL  application 
is  assumed  to  operate  on  a  computing  target  (e.g., 
desktop,  portable  digital  assistant),  which  is  repre¬ 
sented  by  the  dashed  box  in  Figure  55 .2 .  As  an  example 
implementation  of  the  SPC  model,  consider  a  hypo¬ 
thetical  experiment  in  which  participants  are  exposed 
to  a  number  of  visual  stimuli  for  6  sec  each.  Each 
visual  stimulus  occurs  after  a  fixed  foreperiod  of  1  sec 
and  a  subsequent  fixation  cross  (i.e.,  the  point  at  which 
participants  are  required  to  direct  their  gaze)  presented 
for  500  msec.  Each  visual  stimulus  is  followed  by  a 
2-sec  inter- trial  interval  (ITI).  The  only  requirement  of 
the  participant  is  to  view  the  visual  stimuli  for  the 
duration  of  its  presentation.  How  do  we  implement  this 
experiment? 

This  problem  has  several  possible  solutions.  A 
monitor  (presentation  layer)  and  Microsoft  PowerPoint 
(SCRL)  would  easily  accomplish  the  task;  however, 
the  SPC  model  is  suitable  to  handle  an  extensive 
arrangement  of  experimental  designs,  so  additional 
procedural  requirements  increase  the  need  for  added 
SCRL  functionality.  Consider  an  experiment  where 
both  a  monitor  and  speakers  are  required  to  present 
the  stimuli.  This  basic  pattern  still  reflects  an  SPC 


791 


Tamara  van  Gog,  Fred  Paas  et  al. 


TABLE  55.2 
SCRL-Type  Applications 


Name 

Description 

Type 

Platform 

Cogent  2000/ 

Cogent  Graphics 

Complete  PC-based  software  environment  for  functional  brain  mapping 
experiments;  contains  commands  useful  for  presenting  scanner- 
synchronized  visual  stimuli  (Cogent  Graphics),  auditory  stimuli, 
mechanical  stimuli,  and  taste  and  smell  stimuli.  It  is  also  used  in  monitoring 
key  presses  and  other  physiological  recordings  from  the  subject. 

Freeware 

Windows 

DMDX 

Win  32-based  display  system  used  in  psychology  laboratories  around  the 
world  to  measure  reaction  times  to  visual  and  auditory  stimuli. 

Freeware 

Windows 

E-Prime 

Suite  of  applications  to  design,  generate,  run,  collect  data,  edit,  and  analyze 
the  data;  includes:  (1)  a  graphical  environment  that  allows  visual  selection 
and  specification  of  experimental  functions;  (2)  a  comprehensive  scripting 
language;  (3)  data  management  and  analysis  tools. 

Commercial 

Windows 

Flashdot 

Program  for  generating  and  presenting  visual  perceptual  experiments  that 
require  a  high  temporal  precision.  It  is  controlled  by  a  simple  experiment 
building  language  and  allows  experiment  generation  with  either  a  text  or  a 
graphical  editor. 

Freeware 

Windows,  Linux 

FLXLab 

Program  for  running  psychology  experiments;  capabilities  include  presenting 
text  and  graphics,  playing  and  recording  sounds,  and  recording  reaction 
times  via  the  keyboard  or  a  voice  key. 

Freeware 

Windows,  Linux 

PEBL  (Psychology 
Experiment  Building 
Language) 

New  language  specifically  designed  to  be  used  to  create  psychology 
experiments. 

Freeware 

Linux,  Windows, 

Mac 

PsychoPy 

Psychology  stimulus  software  for  Python;  combines  the  graphical  strengths 
of  OpenGL  with  the  easy  Python  syntax  to  give  psychophysics  a  free  and 
simple  stimulus  presentation  and  control  package. 

Freeware 

Linux,  Mac 

PsyScope 

Interactive  graphic  system  for  experimental  design  and  control  on  the 
Macintosh. 

Freeware 

Mac 

PsyScript 

Application  for  scripting  psychology  experiments,  similar  to  SuperLab,  MEL, 
or  E-Prime 

Freeware 

Linux,  Mac 

PyEPL  (Python 
Experiment-Programming 
Library) 

Library  for  coding  psychology  experiments  in  Python;  supports  presentation 
of  both  visual  and  auditory  stimuli,  as  well  as  both  manual 
(keyboard/joystick)  and  sound  (microphone)  input  as  responses. 

Freeware 

Linux,  Mac 

Realtime  Experiment 

Interface 

Extensible  hard  real-time  platform  for  the  development  of  novel  experiment 
control  and  signal-processing  applications. 

Freeware 

Linux 

SuperLab 

Stimulus  presentation  software  with  features  that  support  the  presentation  of 

Commercial 

Windows,  Mac 

multiple  types  of  media  as  well  as  rapid  serial  visual  presentation  paradigms 
and  eye  tracking  integration,  among  other  features. 


model  and  PowerPoint  could  be  configured  to  present 
auditory  and  visual  stimuli  within  a  strict  set  of  param¬ 
eters.  On  the  other  hand,  a  real  experiment  would 
likely  require  that  the  foreperiod,  fixations  cross,  and 
ITI  appear  with  variable  and  not  fixed  timing.  Presen¬ 
tation  applications  like  PowerPoint,  however,  are  not 
specifically  designed  for  experimentation.  As  such, 
limitations  are  introduced  as  experimental  designs 
become  more  elaborate.  One  solution  to  this  problem 
is  to  use  the  Visual  Basic  for  Applications  (VBA)  func¬ 
tionality  embedded  in  Microsoft  Office;  however, 
requiring  features  such  as  variable  timing,  timing 
determinism  (i.e.,  executing  a  task  in  the  exact  amount 
of  time  specified),  support  for  randomization  and 


counterbalancing,  response  acquisition,  and  logging 
illustrates  the  advantages  for  obtaining  a  flexible 
SCRL  application  equipped  for  research  pursuits. 

A  number  of  commercial  and  freeware  applica¬ 
tions  have  been  created  over  the  past  several  decades 
to  assist  researchers  with  SCRL-type  functions.  The 
choice  to  select  one  application  over  the  other  may 
have  much  to  do  with  programming  requirements,  the 
operating  system  platform,  protocol  requirements,  or 
all  of  the  above.  Table  55.2  provides  a  list  of  some  of 
the  SCRL  applications  that  are  available  for  psycho¬ 
logical  and  psychophysical  experiments.  Additional 
information  for  these  and  other  SCRL  applications  can 
be  found  in  Florer  (2007).  The  description  column  is 
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Figure  55.3  SPC  model  with  external  hardware. 

text  taken  directly  from  narrative  about  the  product 
provided  by  Florer  (2007).  A  conventional  program¬ 
ming  language  is  best  equipped  for  SCRL  functional¬ 
ity.  This  alternative  to  the  applications  listed  in  Table 
55 .2  may  be  necessary  for  experiments  where  commu¬ 
nication  with  external  hardware,  interfacing  with 
external  code,  querying  databases,  or  program  perfor¬ 
mance  requirements  are  a  priority,  although  some  of 
the  SCRL  applications  listed  in  Table  55.2  provide 
some  varying  degrees  of  these  capabilities  (e.g.,  E- 
Prime,  SuperLab). 

The  prospect  of  laboratory  productivity  can  out¬ 
weigh  the  flexibility  and  functionality  afforded  by  a 
programming  language;  for  example,  from  a  labora¬ 
tory  management  perspective,  it  is  reasonable  for  all 
laboratory  members  to  have  a  single  platform  from 
which  they  create  experiments.  Given  the  investment 
required  to  familiarize  oneself  with  a  programming 
language,  the  single  platform  option  can  indeed  be 
challenging  to  implement  in  practice.  Formulating  a 
laboratory  in  this  manner  does  allow  members  to  share 
and  reuse  previous  testing  applications  or  utilize 
knowledge  about  the  use  and  idiosyncrasies  of  an 
SCRL  application. 

Despite  the  learning  curve,  a  programming  lan¬ 
guage  has  potential  benefits  that  cannot  be  realized  by 
turnkey  SCRL  applications.  As  mentioned,  high-level 
programming  languages  offer  inherently  greater  flex¬ 
ibility.  Although  it  is  important  to  consider  whether 
the  SCRL  application  can  be  used  to  generate  a  par¬ 
ticular  test  bed,  one  must  also  consider  the  analysis 


Example 


requirements  following  initial  data  collection.  The 
flexibility  of  a  programming  language  can  be  very 
helpful  in  this  regard.  One  might  also  consider  the 
large  support  base  in  the  form  of  books,  forums,  and 
websites  dedicated  to  a  particular  language  which  can 
mitigate  the  problems  that  may  arise  during  the  learn¬ 
ing  process. 

Stimulus  Presentation  and  Control 
Model  with  External  Hardware 

Communicating  with  external  hardware  is  essential  to 
complex  performance  design.  Building  upon  the  basic 
SPC  framework.  Figure  55.3  depicts  the  SPC  model 
with  support  for  external  hardware  (SPCxh).  Figure 
55.3  illustrates  a  scenario  where  the  SCRL  controls 
both  monitor  and  headphone  output.  The  SCRL  also 
interfaces  with  an  eye  tracker  and  a  bio-instrumenta¬ 
tion  device  via  the  parallel  port  and  a  data  acquisition 
device  (DAQ),  respectively.  DAQ  devices  are  an 
important  conduit  for  signal  routing  and  acquisition, 
and  we  discuss  them  in  greater  detail  later.  Extensions 
of  the  SPCxh  and  the  SPC  model  are  the  interface  and 
instrumentation  layers.  A  good  argument  can  be  made 
for  another  interface  layer  to  exist  between  the  presen¬ 
tation  layer  and  the  SCRL,  but  for  our  purposes  the 
interface  layer  specifically  refers  to  the  physical  con¬ 
nection  that  exists  between  the  SCRL  and  the  instru¬ 
mentation  layer.  Figure  55.3  depicts  the  stimulus  pre¬ 
sentation  and  control  model  with  external  hardware 
support  (SPCxh).  The  SPCxh  is  derived  from  the  basic 
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Figure  55.4  A  real- world  example  of  the  SPCxh  model. 


SPC  model,  with  two  additional  layers:  one  to  repre¬ 
sent  the  external  hardware,  the  second  to  interface  that 
hardware  with  the  SCRL. 

It  is  important  to  emphasize  that  the  SPC  and 
SPCxh  models  are  only  examples.  We  recognize  that 
an  actual  implementation  of  any  one  model  will  most 
certainly  differ  among  laboratories.  The  main  purpose 
of  illustrating  the  various  arrangements  in  this  manner 
is  to  address  the  major  question  of  how  complex  per¬ 
formance  design  paradigms  are  arranged  in  an  abstract 
sense.  When  the  necessary  components  are  identified 
for  their  specific  research  objective,  then  comes  the 
process  of  determining  the  specific  hardware  and  soft¬ 
ware  to  realize  that  goal. 

It  is  imperative  to  understand  the  connection 
between  a  given  model  (abstraction)  and  its  real-world 
counterpart.  Using  the  example  described  above,  con¬ 
sider  an  experiment  that  requires  the  additional  collec¬ 
tion  of  electrocortical  activity  in  response  to  the 
appearance  of  the  visual  stimulus.  This  type  of  phys¬ 
iological  data  collection  is  termed  event-related  poten¬ 
tial ,  as  we  are  evaluating  brain  potentials  time-locked 
to  some  event  (i.e.,  appearance  of  the  visual  stimulus 
in  this  example).  Thus,  we  need  to  mark  in  the  phys¬ 
iological  record  where  this  event  appears  for  offline 
analysis .  Figure  55 .4  depicts  one  method  to  implement 
this  requirement.  On  the  left,  the  SPCxh  model  dia¬ 
gram  is  illustrated  for  the  current  scenario.  A  monitor 
is  used  to  display  the  stimulus.  A  programming  lan¬ 


guage  called  LabVIEW  provides  the  SCRL  function¬ 
ality.  Because  the  bio-instrumentation  supports  digital 
input/output  (i.e.,  hardware  that  allows  one  to  send  and 
receive  digital  signals),  LabVIEW  utilizes  the  DAQ 
device  to  output  digital  markers  to  the  digital  input 
ports  of  the  bio-instrumentation  while  also  connecting 
to  the  instrument  to  collect  the  physiological  data  over 
the  network.  We  are  using  the  term  bio-instrumenta- 
tion  here  to  refer  to  the  hardware  used  for  the  collection 
and  assessment  of  physiological  data.  The  picture  on 
the  right  portrays  the  instantiation  of  the  diagram.  It 
should  be  observed  that  the  diagram  is  meant  to  rep¬ 
resent  tangible  software  and  hardware  entities. 
Although  LabVIEW  is  used  in  this  example  as  our 
SCRL  application,  any  number  of  alternatives  could 
have  also  been  employed  to  provide  the  linkage 
between  the  software  used  in  our  SCRL  and  the  instru¬ 
mentation. 

A  number  of  derivations  can  also  be  organized 
from  the  SPCxh  model;  for  example,  in  many  cases, 
the  SCRL  may  contain  only  the  logic  needed  to  exe¬ 
cute  the  experiment  but  not  the  application  program 
interfaces  (APIs)  required  to  directly  control  a  ven¬ 
dor’s  hardware.  In  these  situations,  it  may  be  necessary 
to  run  the  SCRL  application  alongside  (e.g.,  on  another 
machine)  the  vendor-specific  hardware  application. 
Figure  55.5  depicts  this  alternative,  where  the  vendor- 
specific  hardware  executes  its  procedures  at  the  same 
time  as  the  SCRL  application.  Because  the  layers  for 
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Figure  55.5  SCLEs  application  operating  concurrently  with  software. 


the  SPCxh  are  the  same  as  in  Figure  55.3,  Figure  55.5 
depicts  only  the  example  instantiation  of  the  model 
and  not  the  layers  for  the  SPCxh  model.  The  SPCxh 
example  in  Figure  55.5  is  a  common  configuration 
because  hardware  vendors  do  not  always  supply  soft¬ 
ware  interfaces  that  can  be  used  by  an  external  appli¬ 
cation.  The  major  difference  between  the  two  options, 
as  shown,  is  that  the  second  option  would  require  a 
total  of  three  computing  targets:  one  to  execute  the 
SCRL  application  and  for  stimulus  presentation,  one 
to  execute  the  bio- instrumentation  device  software, 
and  a  third  to  execute  the  eye  tracker  device  software. 

A  very  common  question  is  how  to  synchronize 
the  SCRL  application  and  the  device  instrumentation. 
As  with  the  previous  example,  the  method  of  choice 
for  the  example  is  via  the  DAQ  device  for  communi¬ 
cation  with  the  bio-instrumentation  and  through  the 
parallel  port  for  the  eye  tracker:  however,  the  option 
for  event  marking  and  synchronization  is  only  appli¬ 
cable  if  it  is  supported  by  the  particular  piece  of  instru¬ 
mentation.  Furthermore,  the  specific  interface  (e.g., 
digital  input/output,  serial/parallel)  is  dependent  on 
what  the  manufacturer  has  made  available  for  the  end- 
user.  Given  this  information,  one  should  ask  them¬ 
selves  the  following  questions  prior  to  investing 
resources  in  any  one  instrument  or  SCRL  alternative. 
First,  what  type  of  limitations  will  be  encountered 
when  attempting  to  interface  a  particular  instrument 
with  my  current  resources?  That  is,  does  the  manufac¬ 
turer  provide  support  for  external  communication  with 
other  instruments  or  applications?  Second,  does  my 
SCRL  application  of  choice  support  a  method  to  com¬ 


municate  with  my  external  hardware  if  this  option  is 
available?  Third,  does  the  device  manufacturer  provide 
or  sell  programming  libraries  or  application  program 
interfaces  if  data  collection  has  to  be  curtailed  in  some 
particular  way?  Fourth,  what  are  the  computational 
requirements  to  run  the  instrumentation  software?  Will 
the  software  be  so  processor  intensive  that  it  requires 
a  sole  execution  on  one  dedicated  machine? 

Common  Paradigms  and  Configurations 

A  number  of  common  paradigms  exist  in  psychologi¬ 
cal  research,  from  recall  and  recognition  paradigms  to 
interruption-type  paradigms.  Although  it  is  beyond  the 
scope  of  this  section  to  provide  an  example  configu¬ 
ration  for  each,  we  have  selected  one  example  that  is 
commonly  used  by  many  of  contemporary  researchers; 
this  is  the  secondary  task  paradigm.  Both  the  SPC  and 
SPCxh  models  are  sufficient  for  experiments  employ¬ 
ing  this  paradigm;  however,  a  common  problem  can 
occur  when  the  logic  for  the  primary  and  secondary 
tasks  is  implemented  as  mutually  exclusive  entities. 
An  experiment  that  employs  a  simulator  for  the  pri¬ 
mary  task  environment  can  be  viewed  as  a  self-con¬ 
tained  SPCxh  model  containing  a  simulation  presen¬ 
tation  environment  (presentation  layer),  simulation 
control  software  (SCRL  application),  and  simulation- 
specific  hardware  (interface  and  instrumentation  lay¬ 
ers).  The  question  now  is  how  can  we  interface  the 
primary  task  (operated  by  the  simulator  in  this  exam¬ 
ple)  with  another  SPRL  application  that  contains  the 
logic  for  the  secondary  task? 
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Figure  55.6  Secondary  task  paradigm. 


Figure  55.6  contains  a  graphical  representation  of 
a  possible  configuration:  an  SPCxh  model  for  the  sim¬ 
ulator  communicating  via  a  network  interface  that  is 
monitored  from  the  SCRL  application  on  the  second¬ 
ary  task  side.  On  the  left  side  of  Figure  55.6  is  the 
primary  task  configuration,  and  on  the  right  side  is  the 
secondary  task  configuration.  It  should  be  noted  that 
the  simulator  control  software,  our  SCRL  application, 
or  the  device-specific  software  does  not  necessarily 
need  to  be  executed  on  separate  computers;  however, 
depending  on  the  primary  or  secondary  task,  one  may 
find  that  the  processor  and  memory  requirements 
necessitate  multiple  computers.  On  the  secondary  task 
side,  the  diagram  represents  a  fairly  complex  role  for 
the  SCRL  application.  As  shown,  the  SCRL  applica¬ 
tion  has  output  responsibilities  to  a  monitor  and  head¬ 
phones  while  also  interfacing  with  an  eye  tracker  via 
the  serial  port,  interfacing  the  simulator  via  the  net¬ 
work,  and  sending  two  lines  of  digital  output  informa¬ 
tion  via  the  DAQ  device. 

Numerous  complex  performance  test  beds  require 
that  a  primary  and  secondary  task  paradigm  be  used 
to  explicate  the  relationship  among  any  number  of 
processes.  In  the  field  of  human  factors,  in  particular, 
it  is  not  uncommon  for  an  experiment  to  employ  sim¬ 
ulation  for  the  primary  task  and  then  connect  a  sec¬ 
ondary  task  to  events  that  may  occur  during  the  pri¬ 
mary  task.  A  major  problem  often  encountered  in 
simulation  research  is  that  the  simulators  are  often 
closed-systems;  nevertheless,  most  simulators  can  be 
viewed  as  SPCxh  arrangements  with  a  presentation 
layer  of  some  kind,  an  application  that  provides  SCRL 


function,  and  the  simulator  control  hardware  itself.  If 
one  wishes  to  generate  secondary  task  events  based  on 
the  occurrence  of  specific  events  in  the  simulation,  the 
question  then  becomes  one  of  how  we  might  go  about 
configuring  such  a  solution  when  there  is  no  ready¬ 
made  entry  point  between  the  two  systems  (i.e.,  pri¬ 
mary  task  system  and  secondary  task  system).  The 
diagram  on  the  left  shows  the  SPCxh  model  for  the 
secondary  task,  which  is  responsible  for  interacting 
with  a  number  of  additional  instruments .  The  diagram 
shows  a  connecting  line  between  the  secondary  sys¬ 
tems  network  interface  and  the  network  interface  of 
the  primary  task  as  controlled  by  the  simulation. 
Because  simulation  manufacturers  will  often  make 
technotes  available  that  specify  how  the  simulator  may 
communicate  with  other  computers  or  hardware,  one 
can  often  ascertain  this  information  for  integration 
with  the  secondary  tasks  SCRL. 

Summary  of  Design  Configurations 

The  above  examples  have  been  elaborated  in  limited 
detail.  Primarily,  information  pertaining  to  how  one 
would  configure  a  SCRL  application  for  external  soft¬ 
ware  and  hardware  communication  has  been  excluded. 
This  specification  is  not  practical  with  the  sheer  num¬ 
ber  of  SCRL  options  available  to  researchers.  As  well, 
the  diagrams  do  not  inform  the  role  that  the  SCRL 
application  plays  when  interfacing  with  instrumenta¬ 
tion.  It  may  be  the  case  that  the  SCRL  application  plays 
a  minimal  role  in  starting  and  stopping  the  instrument 
and  does  not  command  full  control  of  the  instrument 
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via  a  program  interface;  nevertheless,  one  should 
attempt  to  understand  the  various  configurations 
because  they  do  appear  with  great  regularity  in  com¬ 
plex  performance  designs.  Finally,  it  is  particularly 
important  to  consider  some  of  the  issues  raised  when 
making  purchasing  decisions  about  a  given  instrument, 
interface,  or  application. 

General-Purpose  Hardware 

It  is  evident  when  walking  through  a  home  improve¬ 
ment  store  that  countless  tools  have  proved  their  effec¬ 
tiveness  for  an  almost  limitless  number  of  tasks  (e.g., 
a  hammer).  An  analog  to  this  apparent  fact  is  that 
various  tools  are  also  exceedingly  useful  for  complex 
performance  research;  thus,  the  purpose  of  the  follow¬ 
ing  sections  is  to  discuss  some  of  these  tools  and  their 
role  in  complex  performance  evaluation. 

Data  Acquisition  Devices 

Given  the  ubiquity  of  DAQ  hardware  in  the  examples 
above,  it  is  critical  that  one  has  a  general  idea  of  the 
functionality  that  a  DAQ  device  can  provide.  DAQ 
devices  are  the  research  scientist’s  Swiss  Army  knife 
and  are  indispensable  tools  in  the  laboratory.  DAQ 
hardware  completes  the  bridge  between  the  SCRL  and 
the  array  of  instrumentation  implemented  in  an  exper¬ 
iment;  that  is,  the  DAQ  hardware  gives  a  properly 
supported  SCRL  application  a  number  of  useful  func¬ 
tions  for  complex  performance  measurement.  To  name 
a  few  examples,  DAQ  devices  offer  a  means  of  trans¬ 
mitting  important  data  between  instruments,  supports 
mechanisms  to  coordinate  complex  actions  or  event 
sequences,  provides  deterministic  timing  via  a  hard¬ 
ware  clock,  and  provides  methods  to  synchronize  inde¬ 
pendently  operating  devices. 

A  common  use  for  DAQ  devices  is  to  send  and 
receive  digital  signals;  however,  to  frame  this  applica¬ 
tion  within  the  context  of  complex  performance 
design,  it  is  important  that  one  be  familiar  with  a  few 
terms.  An  event  refers  to  any  information  as  it  occurs 
within  an  experiment;  for  example,  an  event  may  mark 
the  onset  or  offset  of  a  stimulus,  a  participant’s 
response,  or  the  beginning  or  end  of  a  trial.  A  common 
design  obstacle  requires  that  we  know  an  event’s  tem¬ 
poral  appearance  for  the  purpose  of  subsequent  anal¬ 
yses  or,  alternatively,  to  trigger  supplementary  instru¬ 
mentation.  The  term  trigger  is  often  paired  with  event 
to  describe  an  action  associated  with  an  the  occurrence 
of  an  event.  In  some  cases,  event  triggers  may  be 
completely  internal  to  a  single  SCRL  application,  but 
in  other  instances  event  triggers  may  include  external 
communications  between  devices  or  systems. 


Data  acquisition  devices  have  traditionally  been 
referred  to  as  A/D  boards  (analog-to-digital  boards) 
because  of  their  frequent  use  in  signal  acquisition.  A 
signal,  in  this  context,  loosely  refers  to  any  measurable 
physical  phenomenon.  Signals  can  be  divided  into  two 
primary  classes.  Analog  signals  can  vary  continuously 
over  an  infinite  range  of  values,  whereas  digital  signals 
contain  information  in  discrete  states.  To  remember 
the  difference  between  these  two  signals,  visualize  two 
graphs,  one  where  someone’s  voice  is  recorded  (ana¬ 
log)  and  another  plotting  when  a  light  is  switched  on 
or  off  (digital). 

The  traditional  role  of  a  DAQ  device  was  to  acquire 
and  translate  measured  phenomena  into  binary  units 
that  can  be  represented  by  a  digital  device  (e.g.,  com¬ 
puter,  oscilloscope).  Suppose  we  need  to  record  the 
force  exerted  on  a  force  plate.  A  DAQ  device  could  be 
configured  such  that  we  could  sample  the  data  derived 
from  a  force  plate  at  a  rate  of  1  msec  per  sample  and 
subsequently  record  that  data  to  a  computer  or  logging 
instrument. 

One  should  note  that  the  moniker  device  as  a 
replacement  for  board  is  more  appropriate  given  that 
modern  DAQ  alternatives  are  not  always  self-con¬ 
tained  boards  but  may  connect  to  a  computing  target 
in  a  few  different  ways.  DAQ  devices  are  available  for 
a  number  of  bus  types.  A  bus,  in  computing  vernacular, 
refers  to  a  method  of  transmission  for  digital  data  (e.g., 
USB,  FireWire,  PCI).  The  DAQ  device  pictured  in 
Figure  55.4,  for  example,  is  designed  to  connect  to  the 
USB  port  of  a  computer. 

Despite  their  traditional  role  in  signal  acquisition 
(e.g.,  analog  input),  most  DAQ  devices  contain  an 
analog  output  option.  Analog  output  reverses  the  pro¬ 
cess  of  an  A/D  conversion  and  can  be  used  to  convert 
digital  data  into  analog  data  (D/A  conversion).  Analog 
output  capabilities  are  useful  for  a  variety  of  reasons; 
for  example,  an  analog  output  signal  can  be  used  to 
produce  auditory  stimuli,  control  external  hardware, 
or  output  analog  data  to  supplementary  instrumenta¬ 
tion.  The  primary  and  secondary  task  example  above 
illustrates  one  application  for  analog  output.  Recall 
that  in  the  SPCxh  model  described  earlier  the  simulator 
was  a  closed-system  interfaced  with  the  SCRL  appli¬ 
cation  via  a  network  interface.  Suppose  that  it  was 
necessary  to  proxy  events  as  they  occurred  in  the  sim¬ 
ulation  to  secondary  hardware.  A  reason  for  this 
approach  might  be  as  simple  as  collapsing  data  to  a 
single  measurement  file;  for  example,  suppose  we  want 
to  evaluate  aiming  variability  in  a  weapons  simulation 
in  tandem  with  a  physiological  measure.  One  strategy 
would  require  that  we  merge  the  separate  streams  of 
data  after  all  events  have  been  recorded,  but,  by  using 
another  strategy  employing  the  analog  output  option. 
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Figure  55.7  Event  triggering  and  recording. 


we  could  route  data  from  the  simulator  and  proxy  it 
via  the  SCRL  application  controlling  the  DAQ  device 
connected  to  our  physiological  recording  device. 

In  addition  to  analog  output,  DAQ  functionality  for 
digital  input/output  is  a  critical  feature  to  solving  var¬ 
ious  complex  performance  measurement  issues.  Recall 
the  example  above  where  an  experiment  required  that 
the  SCRL  application  tell  the  physiological  control 
software  when  the  visual  stimulus  was  displayed.  This 
was  accomplished  by  the  SCRL  application  sending  a 
digital  output  from  the  DAQ  device  to  the  digital  input 
port  on  the  bio-instrument.  Figure  55.7  depicts  this 
occurrence  from  the  control  software  for  the  bio- 
instrument.  The  figure  shows  one  channel  of  data 
recorded  from  a  participant’s  scalp  (i.e.,  EEG);  another 
digital  channel  represents  the  onset  and  offset  of  the 
visual  stimulus. 

When  configuring  digital  signals,  it  is  also  impor¬ 
tant  that  one  understand  that  the  event  can  be  defined 
in  terms  of  the  leading  or  trailing  edge  of  the  digital 
signal.  As  depicted  in  Figure  55.7,  the  leading  edge 
(also  called  the  rising  edge)  refers  to  the  first  positive 
deflection  of  the  digital  waveform,  while  the  trailing 
edge  (also  called  th e,  falling  edge )  refers  to  the  negative 
going  portion  of  the  waveform.  This  distinction  is 
important,  because  in  many  cases  secondary  instru¬ 
mentation  will  provide  an  option  to  begin  or  end 
recording  from  the  leading  or  trailing  edge;  that  is,  if 
we  mistakenly  begin  recording  on  the  trailing  edge 
when  the  critical  event  occurs  on  the  leading  edge, 
then  the  secondary  instrument  may  be  triggered  late 
or  not  at  all.  Another  term  native  to  digital  events  is 
transistor-transistor  logic  (TTL),  which  is  often  used 
to  express  digital  triggering  that  operates  within  spe¬ 


cific  parameters.  TTL  refers  to  a  standard  where  a 
given  instrument’s  digital  line  will  change  state  (e.g., 
on  to  off)  if  the  incoming  input  is  within  a  given 
voltage  range.  If  the  voltage  supplied  to  the  digital  line 
is  0  then  it  is  off,  and  if  the  digital  line  is  supplied  5 
volts  then  it  is  on. 

Event  triggering  is  an  extremely  important  constit¬ 
uent  of  complex  performance  experimentation.  Know¬ 
ing  when  an  event  occurs  may  be  vital  for  data  analysis 
or  for  triggering  subsequent  events.  The  example  here 
depicts  a  scenario  where  we  are  interested  in  knowing 
the  onset  occurrence  of  a  visual  stimulus  so  we  can 
analyze  our  EEG  signal  for  event-related  changes.  The 
channel  with  the  square  wave  is  the  event  that  tells 
when  the  event  occurred,  with  the  leading  edge  repre¬ 
senting  the  onset  of  the  visual  stimuli  (5  volts)  and  the 
falling  edge  reflecting  its  offset  (0  volts).  A  setup  sim¬ 
ilar  to  that  demonstrated  in  Figure  55.4  could  easily 
be  configured  to  produce  such  an  example  .  Although 
this  example  only  shows  two  channels,  a  real-world 
testing  scenario  may  have  several  hundred  to  indicate 
when  certain  events  occur.  One  strategy  is  to  define 
different  events  as  different  channels.  One  channel 
may  represent  the  visibility  of  a  stimuli  (on  or  off), 
another  may  represent  a  change  in  its  color,  and  yet 
another  may  indicate  any  other  number  of  events.  An 
alternative  solution  is  to  configure  the  SCRL  applica¬ 
tion  to  send  data  to  a  single  channel  and  then  create  a 
coding  scheme  to  reflect  different  events  (e.g.,  0  volts, 
stimulus  hidden;  2  volts,  stimulus  visible;  3  volts,  color 
changed  to  black;  4  volts,  color  changed  to  white). 
This  approach  reduces  the  number  of  channels  and 
maximizes  the  number  of  digital  channels  that  one  can 
control. 
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The  utility  of  digital  event  triggering  cannot  be 
overstated.  Although  its  application  for  measurement 
of  complex  performance  requires  a  small  degree  of 
technical  expertise,  the  ability  to  implement  event  trig¬ 
gering  affords  a  great  degree  of  experimental  flexibility. 
Under  certain  circumstances,  however,  analog  trigger¬ 
ing  may  also  be  appropriate.  Consider  an  experiment 
where  the  threshold  of  a  participant’s  voice  serves  as 
the  eliciting  event  for  a  secondary  stimulus.  In  this  case, 
it  is  necessary  to  determine  whether  or  not  the  SCRL 
application  and  DAQ  interface  might  support  this  type 
of  triggering,  because  such  an  approach  offers  greater 
flexibility  for  some  design  configurations. 

Purchasing  a  DAQ  Device 

The  following  questions  are  relevant  to  purchasing  a 
DAQ  device  for  complex  performance  research.  First, 
is  the  DAQ  device  going  to  be  used  to  collect  physio¬ 
logical  or  other  data?  If  the  answer  is  yes,  one  should 
understand  that  the  price  of  a  DAQ  device  is  primarily 
a  function  of  its  resolution,  speed,  form  factor,  and  the 
number  of  input/output  channels  supported.  Although 
a  discussion  of  unipolar  vs.  bipolar  data  acquisition  is 
beyond  the  scope  of  this  chapter,  the  reader  should 
consult  Olansen  and  Rosow  (2002)  and  Stevenson  and 
Soejima  (2005)  for  additional  information  on  how  this 
may  effect  the  final  device  choice. 

Device  resolution,  on  the  other  hand,  refers  to  the 
fidelity  of  a  DAQ  device  to  resolve  analog  signals;  that 
is,  what  is  the  smallest  detectable  change  that  the 
device  can  discriminate?  When  choosing  a  board,  res¬ 
olution  is  described  in  terms  of  bits.  A  16-bit  board  can 
resolve  signals  with  greater  fidelity  than  a  12-bit  board. 
By  taking  the  board  resolution  as  an  exponent  of  2, 
one  can  see  why.  A  12-bit  board  has  2 12  or  4096  pos¬ 
sible  values,  while  a  16-bit  has  a  216or  65,536  possible 
values.  The  answer  to  this  question  is  also  a  function 
of  a  few  other  factors  (e.g.,  signal  range,  amplification) . 
A  complete  understanding  of  these  two  major  issues 
should  be  established  prior  to  deciding  on  any  one 
device.  As  well,  prior  to  investing  in  a  higher  resolution 
board,  which  will  cost  more  money  than  a  lower  res¬ 
olution  counterpart,  one  should  evaluate  what  is  the 
appropriate  resolution  for  the  application.  If  the  pri¬ 
mary  application  of  the  DAQ  device  is  for  digital  event 
triggering,  then  it  is  important  to  purchase  a  device  that 
is  suitable  for  handling  as  many  digital  channels  as  are 
necessary  for  a  particular  research  design. 

Third,  does  the  design  require  analog  output  capa¬ 
bilities?  Unlike  digital  channels,  which  are  generally 
reconfigurable  for  input  or  output,  analog  channels  are 
not,  so  it  is  important  to  know  in  advance  the  number 
of  analog  channels  that  a  DAQ  device  supports.  Fourth, 
does  the  testing  environment  require  a  level  of  timing 


determinism  that  cannot  be  provided  by  software  in 
less  than  1  msec?  For  these  scenarios,  researchers 
might  want  to  consider  a  DAQ  device  that  supports 
hardware  timing.  For  additional  information  about  A/D 
board  specifications  and  other  factors  that  may  affect 
purchasing  decisions,  see  S taller  (2005). 

Computers  as  Instrumentation 

Computers  are  essential  to  the  modem  laboratory.  Their 
value  is  evident  when  one  considers  their  versatile  role 
in  the  research  process;  consequently,  the  computer’s 
ubiquity  in  the  science  can  account  for  significant  costs. 
Because  academic  institutions  will  often  hold  contracts 
with  large  original  equipment  manufactures,  computing 
systems  are  competitively  priced  and  warranties  ensure 
maintenance  for  several  years.  Building  a  system  from 
the  ground  up  is  also  a  viable  option  that  will  often 
provide  a  cost-effective  alternative  to  purchasing 
through  an  original  equipment  manufacturer.  Although 
the  prospect  of  assembling  a  computer  may  sound 
daunting,  the  process  is  really  quite  simple,  and  numer¬ 
ous  books  and  websites  are  dedicated  to  the  topic  (see, 
for  example,  Hardwidge,  2006).  Customization  is  one 
of  the  greatest  advantages  to  building  a  machine,  and 
because  only  necessary  components  are  purchased 
overall  cost  is  usually  reduced.  On  the  other  hand,  a 
major  disadvantage  to  this  approach  is  the  time  asso¬ 
ciated  with  reviewing  the  components,  assembling  the 
hardware,  and  installing  the  necessary  software. 

Most  new  computers  will  likely  be  capable  of  han¬ 
dling  a  good  majority  of  laboratory  tasks;  however, 
one  should  have  a  basic  understanding  of  its  major 
components  to  make  an  informed  purchasing  decision 
when  planning  complex  performance  test  beds.  This 
is  important  given  that  one  can  potentially  save  con¬ 
siderable  resources  that  can  be  allocated  for  other 
equipment  and  be  assured  that  the  computer  is  ade¬ 
quate  for  a  given  experimental  paradigm. 

The  following  questions  should  be  considered 
whether  building  or  buying  a  complete  computing  sys¬ 
tem.  First,  does  the  number  of  expansion  ports  accom¬ 
modate  input  boards  that  may  be  needed  to  interface 
instrumentation?  For  example,  if  an  instrument  inter¬ 
faced  an  application  via  a  network  port  and  we  wanted 
to  maintain  the  ability  to  network  with  other  computers 
or  access  the  Internet,  it  would  be  important  to  confirm 
that  the  motherboard  of  the  computer  had  a  sufficient 
number  of  slots  to  accommodate  this  addition.  Fur¬ 
thermore,  because  DAQ  devices  are  often  sold  as  input 
boards  this  same  logic  would  apply. 

Computers  have  evolved  from  general-purpose 
machines  to  machines  with  specific  aptitudes  for  par¬ 
ticular  tasks.  A  recent  development  is  that  vendors  now 
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market  a  particular  computing  system  for  gaming  vs. 
video-editing  vs.  home  entertainment  purposes.  To 
understand  the  reasons  behind  these  configurations,  we 
strongly  advocate  developing  a  basic  understanding  of 
how  certain  components  facilitate  particular  tasks. 
Although  space  prevents  us  from  accomplishing  this 
within  this  chapter,  it  is  important  to  realize  that  com¬ 
puting  performance  can  alter  timing  determinism,  par¬ 
ticularly  in  complex  performance  environments. 

Discussion 

The  challenge  of  understanding  the  various  technical 
facets  of  laboratory  setup  and  configuration  represents 
a  major  hurdle  when  the  assessment  of  some  complex 
performance  is  a  central  objective.  This  section  has 
discussed  the  common  problems  and  design  configu¬ 
rations  that  one  may  encounter  when  setting  up  such 
a  laboratory.  This  approach,  abstract  in  some  respects, 
was  not  intended  to  illustrate  the  full  range  of  design 
configurations  available  for  complex  performance 
evaluation;  rather,  the  common  configurations  dis¬ 
cussed  here  should  only  be  viewed  as  general-purpose 
architectures,  independent  of  new  technologies  that 
may  emerge.  After  developing  an  understanding  of  the 
various  design  configurations,  one  must  determine  the 
specific  hardware  and  software  that  are  required  to 
address  the  research  question.  The  purpose  of  provid¬ 
ing  a  few  design  configurations  here  was  to  emphasize 
that,  in  many  complex  performance  testing  environ¬ 
ments,  one  must  specify  what  equipment  or  software 
will  fill  the  roles  of  presentation,  stimulus  control  and 
response,  instrumentation,  and  their  interfaces. 


CONCLUDING  REMARKS 

Setting  up  laboratories  for  the  measurement  of  com¬ 
plex  performances  can  indeed  be  a  challenging  pursuit; 
however,  becoming  knowledgeable  about  the  solutions 
and  tools  available  to  aid  in  achieving  the  research 
objectives  is  rewarding  on  a  number  of  levels.  The 
ability  to  identify  and  manipulate  multiple  software 
and  hardware  components  allows  quick  and  effective 
transitioning  from  a  research  question  into  a  tenable 
methodological  test  bed. 
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ABSTRACT 

In  this  chapter  the  editors  and  editorial  assistants  of 
this  Handbook  address  five  questions  as  a  way  of 
speculating  about  the  future  of  scientific  inquiry  per¬ 


taining  to  educational  communications  and  technol¬ 
ogy:  (1)  What  have  been  the  most  significant  devel¬ 
opments  in  educational  communications  and 
technology  in  the  last  five  years?  (2)  What  are  likely 
to  be  the  most  significant  developments  in  the  next 


*  The  authors  of  this  chapter  are  listed  in  arbitrary  alphabetical  order  because  they  contributed  equally. 
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live  years?  (3)  What  significant  research  findings  have 
occurred  in  the  last  five  years?  (4)  What  are  likely  to 
be  the  most  significant  research  issues  in  the  next  five 
years?  (5)  What  are  the  most  critical  foundational 
aspects  and  factors  for  future  research  and  develop¬ 
ment  in  educational  communications  and  technol¬ 
ogy?  In  addition  to  addressing  these  questions,  we 
also  comment  on  the  proper  study  of  instructional 
design  and  how  best  to  prepare  educational  technol¬ 
ogy  practitioners  and  researchers. 


KEYWORDS 

Core  reference:  A  reference  that  is  of  central  interest 
to  researchers  in  a  particular  area  and  generally 
considered  to  be  a  primary  source  of  data  or  infor¬ 
mation;  core  references  are  indicated  by  an  asterisk 
in  the  reference  section  of  each  chapter  in  this 
Handbook. 

Keyword:  A  word  or  phrase  that  is  critically  important 
to  a  topic  area  or  which  has  a  particular  definition 
important  for  understanding  issues  and  further  dis¬ 
cussion. 


INTRODUCTION 

When  we  accepted  the  task  of  editing  this  third  edition 
of  Handbook ,  we  decided  that  our  basic  perspective 
would  be  that  the  Handbook  was  owned  by  our  pro¬ 
fessional  community.  This  meant  that  its  contents 
should  be  largely  driven  by  the  interests  of  the  com¬ 
munity,  so  we  conducted  a  survey  hosted  by  the  Asso¬ 
ciation  for  Educational  Communications  and  Technol¬ 
ogy  (AECT),  the  sponsor  of  the  Handbook ,  to 
determine  how  previous  editions  were  used  and  what 
the  community  would  like  to  see  in  this  edition,  and 
we  responded  accordingly. 

A  second  aspect  of  community  ownership  was  that 
our  own  views  and  biases  should  not  dominate.  We 
broadened  this  notion  to  include  the  concept  that  no 
one’s  views  should  dominate,  and  we  operationalized 
this  notion  in  the  form  of  a  rule  that  no  one  could  be 
lead  author  on  more  than  one  chapter.  We  actively 
recruited  a  broader  group  of  contributors  than  in  past 
editions,  and  we  encouraged  experienced  authors  to 
coauthor  with  more  junior  authors  to  ensure  diversity 
of  content  and  views.  We  sincerely  hope  that  we  have 
achieved  these  objectives. 

Now  it  is  our  turn,  though,  to  collectively  comment 
on  what  we  regard  as  important  foundation  issues  for 
the  future  of  research  and  development  pertaining  to 


educational  communications  and  technologies.*  We 
decided  to  do  this  by  addressing  five  questions:  (1) 
What  have  been  the  most  significant  developments  in 
educational  communications  and  technology  in  the 
last  five  years?  (2)  What  are  likely  to  be  the  most 
significant  developments  in  the  next  five  years?  (3) 
What  significant  research  findings  have  occurred  in 
the  last  five  years?  (4)  What  are  likely  to  be  the  most 
significant  research  issues  in  the  next  five  years?  (5) 
What  are  the  most  critical  foundational  aspects  and 
factors  for  future  research  and  development  in  educa¬ 
tional  communications  and  technology?  Our  various 
responses  to  these  questions  are  woven  into  subse¬ 
quent  sections  of  this  chapter.  We  conclude  with 
thoughts  about  the  study  of  instructional  design  and 
technology  and  how  best  to  prepare  practitioners  and 
researchers. 


SIGNIFICANT  DEVELOPMENTS 
IN  THE  LAST  FIVE  YEARS 

With  regard  to  instructional  strategies  and  learning 
technologies,  the  major  developments  in  the  last  five 
years  took  place  in  the  field  of  Web-based  learning  and 
more  broadly  in  e-learning.  Although  many  interpret 
the  “e”  in  e-leaming  to  refer  to  the  Internet,  we  inter¬ 
pret  the  “e”  to  refer  more  traditionally  to  electronic, 
which  includes  the  Internet  and  many  other  digital 
technologies.  The  focus  quickly  shifted  from  the  Web 
as  a  medium  for  information  distribution  to  a  medium 
to  sustain  learning  networks,  in  which  social  interac¬ 
tion  between  learners  is  at  least  as  important,  if  not 
more  important,  than  the  content  to  be  learned. 

Content  has  become  readily  available  and  rich  in 
representational  formats.  One  consequence  is  that  the 
emphasis  in  educational  research  and  development  is 
shifting  from  a  content-centric  perspective  to  a  user¬ 
centric  perspective— it  is  what  one  does  with  the  con¬ 
tent  that  has  become  the  focus  of  interest  for  many 
developers  and  researchers,  and  of  course  what  one 
does  is  often  done  in  concert,  collaboration,  or  com¬ 
petition  with  others.  Socially  situated  activities  with 
rich  content  have  become  the  focus  of  much  educa¬ 
tional  technology  research  and  development. 


*  In  this  chapter  we  make  frequent  use  of  the  terms  educational  tech¬ 
nology ,  instructional  design ,  and  instructional  technology.  In  some 
cases,  we  make  no  effort  to  distinguish  these  terms  as  we  individually 
have  slightly  different  meanings  for  these  terms.  Roughly  speaking, 
educational  technology  is  broader  in  scope  than  instructional  technol¬ 
ogy ,  with  the  latter  focused  on  the  use  of  technology  to  support  specific, 
intended,  and  planned  learning  outcomes.  Instructional  design,  broadly 
speaking,  is  the  discipline  devoted  to  the  study  of  these  technologies 
and  their  use  in  support  of  learning  and  instruction. 
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A  movement  to  merge  computer-based  technolo¬ 
gies  with  the  psychology  of  learning  has  been  ongoing 
for  many  years.  An  increased  emphasis  on  using  new 
instructional  technologies  to  support  learning  (blogs, 
podcasts,  wikis,  and  so  on)  creates  a  rich  area  for 
educational  technology  research  as  there  are  always 
new  tools  and  interventions  to  test;  however,  it  also 
creates  a  burden  for  practitioners  to  learn  how  to  make 
effective  use  of  new  technologies.  The  burden  is  per¬ 
haps  most  strongly  felt  by  teachers,  who  are  continu¬ 
ally  challenged  to  keep  up  with  their  students  and  with 
technology  trends. 

Technological  advances  have  certainly  made  it 
possible  to  provide  educational  resources  and  access 
to  learners  separated  from  their  co-learners,  tutors,  and 
teachers  by  time,  distance,  culture,  and  language. 
Although  many  schools  and  universities  now  make 
regular  use  of  online  learning  management  systems, 
the  challenge  is  to  create  meaningful  learning  environ¬ 
ments  and  experiences. 

Technology  advances  have  also  occurred  with 
regard  to  the  affective  aspects  in  learning— namely,  in 
using  technology  to  optimize  affective  responses  such 
as  with  pedagogical  agents  (Craig  et  al.,  2004;  Kim 
and  Baylor,  2006).  Affective  aspects  such  as  attitudes, 
motivation,  and  emotions  obviously  influence  the  pro¬ 
cess  of  learning,  so  it  has  been  a  natural  extension  of 
educational  technology  research  to  develop  affective 
supports  for  learning.  Recently,  advanced  technologies 
such  as  virtual  humanlike  animations  have  been  devel¬ 
oped  for  more  active  interaction  with  learners.  Exam¬ 
ples  of  these  technologies  include  affective  computing 
at  MIT  (http://affect.media.mit.edu/index.php),  per¬ 
suasive  technologies  at  Stanford  University  (http://cap- 
tology.stanford.edu/),  virtual  humans  at  the  University 
of  Southern  California  (http://www.ict.usc.edu/con- 
tent/view/32/85),  and  anthropomorphized  interfaces  at 
Utah  State  University  (http://www.create.usu.edu/). 

With  regard  to  instructional  models  and  design  and 
development  practices,  there  has  been  a  significant 
reduction  in  the  gap  and  tension  between  constructivist 
and  engineering  models  of  instructional  design.  For  a 
long  time,  constructivist  and  systems  approaches  were 
perceived  as  incompatible  and  often  as  antagonist  per¬ 
spectives  about  the  planning  and  implementation  of 
support  for  learning.  More  recently,  whole-task  models 
(van  Merrienboer,  1997;  van  Merrienboer  and  Kir- 
schner,  2007;  see  also  Chapter  35  in  this  Handbook) 
as  well  as  models  for  model-facilitated  learning  (Mil- 
rad  et  al.,  2002;  Seel,  2003;  Seel  et  al.,  2000;  Spector, 
2006;  Spector  and  Koszalka,  2004)  and  problem-based 
learning  (Jonassen,  2000;  see  also  Chapter  38  in  this 
Handbook)  more  and  more  combine  a  systematic  and 
systemic  approach  to  the  design  of  instruction  with 


constructivist  learning  principles.  This  rapprochement 
(perhaps  the  wrong  word  but  conveys  the  coming  to 
terms  with  those  previously  regarded  as  belonging  to 
an  enemy  camp)  of  constructivism  and  systems  think¬ 
ing  is  in  part  a  response  to  efforts  to  support  learning 
in  and  about  complex,  dynamic,  and  ill-structured 
tasks.  Innovative  learning  environments  have  been 
developed  that  take  into  consideration  the  rich  variety 
and  diversity  of  all  components  of  a  system,  including 
interrelationships  among  the  components  and  relation¬ 
ships  with  other  systems  and  users  (see  Dorner,  1996). 
These  considerations  include  cognitive,  physical,  and 
affective  elements  within  each  person  involved,  as  well 
as  social  and  cultural  relationships. 

With  regard  to  research  methods,  the  balance 
between  research  on  instructional  strategies  and  tech¬ 
nologies,  on  the  one  hand,  and  research  on  instruc¬ 
tional  models  and  design  and  development  practices, 
on  the  other  hand,  drastically  changed.  Design  exper¬ 
iments  and  developmental  research  gained  in  impor¬ 
tance,  which  may  be  seen  as  a  shift  from  model  build¬ 
ing  to  model  testing  and  a  sign  that  the  field  is  maturing 
into  a  real  design  science  (see  Chapters  47  and  54  in 
this  Handbook  for  elaborations  of  this  line  of  thought) . 

LIKELY  DEVELOPMENTS 
IN  THE  NEXT  FIVE  YEARS 

Will  the  future  resemble  the  past?  Affirmation  of  the 
general  principle  that  the  future  will  resemble  the  past 
is  essential  for  inductive  reasoning  and  considered  a 
cornerstone  of  progress  in  science;  however,  there  is 
no  evidence  to  support  this  principle  of  induction,  as 
David  Hume  pointed  out  in  An  Enquiry  Concerning 
Human  Understanding  (Hume,  1777/1910).  Nonethe¬ 
less,  we  naturally  form  expectations  about  the  future, 
and  these  expectations  appear  quite  useful  in  coming 
to  understand  our  worlds.  Still,  we  ought  to  be  modest 
in  making  claims  about  the  future;  the  truth  is  that  we 
do  not  know  what  the  future  will  be  like,  nor  do  you. 

With  regard  to  instructional  strategies  and  technol¬ 
ogies,  mobile  devices  such  as  portable  digital  assistants 
(PDAs),  smart  mobile  phones,  and  e-paper  are  likely 
to  become  increasingly  important  to  learning  and 
instruction.  These  technologies  are  likely  to  offer  new 
ways  to  extend  learning  to  informal  settings  outside 
schools  as  well  as  to  changing  and  dynamic  work 
situations.  Powerful  mobile  technologies  enable  rich 
learning  scenarios  in  which  information  provision,  col¬ 
laboration,  and  work  on  real-life  learning  tasks  can  be 
combined  with  each  other. 

Web-based  technologies  are  likely  to  continue  to 
expand  in  terms  of  access,  use,  and  functionality.  More 
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efficient  and  powerful  Web-based  technologies  will  be 
developed.  Blogs  will  become  more  commonplace  and 
easier  to  use.  Virtual  environments  in  which  partici¬ 
pants  can  dynamically  exchange  thoughts  and  reac¬ 
tions  will  become  commonplace  in  both  instructional 
and  work  environments.  Such  developments  are  likely 
to  mean  the  end  of  emphasis  on  the  “e”  in  e-learning. 
In  short,  we  will  come  to  expect  nearly  all  learning  to 
be  e-leaming— that  is,  facilitated  by  or  supported  with 
some  form  of  electronic  technology.  With  the  wither¬ 
ing  away  of  e-leaming  and  our  fascination  with  elec¬ 
tronic  technologies,  we  are  likely  to  see  increased 
emphasis  on  learning  and  performance  outcomes,  sup¬ 
ported  of  course  by  a  rich  variety  of  e-solutions. 

Another  important  technology  development  will 
involve  open  educational  resources,  such  as  Wikipedia 
(http://en.wikipedia.org/wiki/Main_Page).  An  increas¬ 
ing  number  of  educational  institutions  will  make  their 
learning  content  freely  available  to  the  public  via  the 
Internet.  With  the  accessibility  of  more  and  more  open 
content,  learner  support,  guidance,  and  assessment  in 
resource -based  learning  (RBL)  will  become  increas¬ 
ingly  important  (see  Chapter  40  in  this  Handbook) . 

In  the  next  five  years,  instructional  models  will 
be  better  adapted  to  support  the  design  and  develop¬ 
ment  of  highly  flexible  learning  scenarios  and  just¬ 
in-time/task  performance  and  instructional  supports. 
Indeed,  this  will  be  one  outcome  of  recent  interest  in 
and  emphasis  on  design  research.  Instructional  mod¬ 
els  will  become  more  flexible  with  regard  to  time, 
place,  and  content  and  will  also  allow  for  richer  vari¬ 
eties  and  mixes  of  learning  support,  including  more 
support  for  guided  and  self-directed  learning.  The 
combination  of  place-  and  time-independent  accessi¬ 
bility,  as  well  as  new  learning  technologies  that  allow 
for  individual  adaptation  and  personalization,  will 
allow  for  cost-effective  forms  of  mass  customization . 
In  the  past,  adaptive  and  personalized  instruction  have 
been  conceptualized  and  implemented  on  an  individ¬ 
ual  basis,  in  response  to  a  particular  learner’s  experi¬ 
ence,  proficiency,  and  (mis)understanding.  Construct¬ 
ing  intelligent  tutoring  systems  based  on  robust 
student  models  proved  to  be  much  more  difficult  than 
imagined  and  has  only  succeeded  with  regard  to  a 
narrow  range  of  learning  tasks  and  situations.  As 
intelligent  tutoring  technology  has  matured,  content 
databases  have  become  much  more  vast  and  more 
widely  accessible,  as  have  databases  containing  infor¬ 
mation  about  individual  experience  and  training.  This 
combination  of  events  provides  the  foundation  for 
systems  capable  of  personalization  on  a  grand  scale. 
Commercial  examples  already  exist  (e.g.,  Dell  Com¬ 
puter’s  build -to-order  system),  so  educational  appli¬ 
cations  will  be  emerging  soon. 


SIGNIFICANT  RESEARCH 
FINDINGS  IN  THE  LAST  FIVE  YEARS 

Questions  of  conscience  occur  to  researchers  just  as 
they  occur  to  others.  One  such  question  is  this:  What 
will  come  from  what  one  is  now  doing  and  likely  to 
do  tomorrow?  Indeed,  what  has  resulted  from  what 
we,  as  a  community  of  researchers,  have  done  that  can 
be  considered  significant  contributions  to  learning  and 
instruction?  It  is  hoped  that  this  Handbook  is  filled 
with  good  examples.  We  shall  mention  a  few. 

One  important  research  result  is  that  the  limitations 
of  constructivist  learning  principles,  such  as  discovery 
methods  and  inquiry  learning,  are  becoming  more 
clear  (Kirschner  et  al.,  2006).  Whereas  the  debate  on 
this  issue  will  probably  never  be  fully  resolved,  the 
identification  of  conditions  under  which  particular 
instructional  strategies  do  or  do  not  work  is  a  major 
concern  in  the  held  of  educational  communications 
and  technology.  The  fact  that  the  issue  is  being  re¬ 
focused  in  terms  of  evidence  of  learning  rather  than  in 
terms  of  advocacy  for  a  particular  position  is  encour¬ 
aging.  The  classical  model  for  instructional  design 
research  (Reigeluth,  1983)  is  alive  and  well,  which 
bodes  well  for  the  future  of  educational  technology 
research,  in  our  view. 

Merrill’s  (2002)  work  on  the  first  principles  of 
instruction  made  it  possible  to  identify  similarities 
between  models  and  approaches  that  at  first  sight 
seemed  to  have  little  in  common.  In  particular,  his 
analyses  of  current  instructional  models  show  that 
learning  tasks  based  on  real-life  problems  and  situa¬ 
tions  are  a  particularly  powerful  driving  force  for 
learning.  This  finding  is  echoed  in  research  on  prob¬ 
lem-based  learning  (Jonassen,  2000),  and  in  whole- 
task  training  (van  Merrienboer  and  Kirschner,  2007; 
see  also  Chapter  35  in  this  Handbook). 

Another  important  finding  relates  to  the  expertise 
reversal  effect  (Kalyuga  et  al.,  2003).  Many  research¬ 
ers  have  long  argued  that  prior  knowledge  is  the  most 
important  factor  influencing  learning,  with  the  tacit 
assumption  that  instructional  methods  that  work  for 
novice  learners  are  unnecessary  but  harmless  for  more 
advanced  learners.  In  contradiction  to  this,  the  exper¬ 
tise  reversal  effect  shows  that  methods  that  work  well 
for  novice  learners  may  have  counterproductive  effects 
for  advanced  learners  and  vice  versa.  This  finding  is 
supported  by  research  pertaining  to  the  progressive 
development  of  mental  models  in  individual  learners 
(Seel  et  al.,  2000).  Seel  and  colleagues  noticed  that  a 
conceptual  model  was  often  useful  for  learners  new  to 
a  domain  and  lacking  relevant  knowledge  and  experi¬ 
ence  (devised  in  accordance  with  a  cognitive  appren¬ 
ticeship  instructional  model),  but  that  the  same  model 
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often  conflicted  with  those  constructed  internally  by 
more  experienced  and  knowledgeable  learners  and 
detracted  from  learning  in  those  cases.  What  works  for 
one  may  not  work  for  all. 

LIKELY  RESEARCH  ISSUES 
IN  THE  NEXT  FIVE  YEARS 

Flaving  reviewed  some  of  the  more  significant  devel¬ 
opments  and  findings  in  recent  years  and  provided 
some  sense  of  likely  technology  developments  in  the 
next  five  years,  we  are  in  a  position  to  speculate  about 
likely  research  issues  that  will  soon  be  emerging. 

An  important  research  issue  for  the  next  five  years 
concerns  technology  integration  in  rich  learning  sce¬ 
narios.  In  the  field  of  educational  communications  and 
technology,  there  has  always  been  a  focus  on  particular 
media  and  particular  technologies;  however,  in  future 
learning  scenarios,  wired  and  wireless  as  well  as  for¬ 
mal  and  informal  networks  will  connect  people, 
resources,  and  tools  thanks  to  the  availability  of  inter¬ 
connected  sets  of  (mobile)  devices.  There  is  now  much 
talk  about  technology  integration  but  very  little  real 
work  is  going  on.  First,  we  need  a  robust  definition  of 
technology  integration  that  is  relevant  for  learning  and 
instruction.  This  is  likely  to  be  a  real  issue  given  the 
increasing  variety  of  technologies  and  means  for  sup¬ 
porting  learning.  As  a  step  toward  such  a  definition  of 
technology  integration,  we  would  like  to  offer  the 
notion  that  technology  is  successfully  integrated  into 
learning  and  instruction  when  the  interest  and  focus 
are  not  on  the  technology  but  rather  on  that  which  the 
technology  makes  possible— the  affordance  (e.g.,  the 
dialog  itself  in  a  video-based  dialog  via  the  Internet  or 
formulation  and  testing  of  a  hypothesis  in  a  Web-based 
interactive  simulation) .  In  an  odd  sort  of  way,  success¬ 
ful  integration  is  blind  to  the  technologies  involved. 
We  believe  that  technology  integration— what  it  is, 
what  makes  it  more  or  less  effective,  how  and  why  it 
contributes  to  learning— is  a  particularly  rich  area  for 
research  in  the  coming  years. 

Research  will  more  and  more  focus  on  strategies 
and  models  for  complex  learning,  because  general 
problem-solving  and  reasoning  skills  and  self-directed 
learning  skills  are  needed  to  cope  with  fast  changes  in 
technologies  and  jobs.  Indeed,  society  increasingly 
demands  workers  who  are  able  to  cope  with  complex¬ 
ity  and  make  rapid  and  flexible  adjustments  to  chang¬ 
ing  work  situations.  Such  models  increasingly  aim  at 
learning  in  rich  (simulated)  task  environments,  gaming 
environments,  social  networks,  and  so  forth. 

Related  to  increasing  interest  in  complex  learning 
will  be  increasing  emphasis  on  assessing  learning  and 


performance  in  ill-structured  problem  and  task 
domains.  Without  reliable  means  to  determine  the  rel¬ 
ative  progress  of  learning  and  performance  with  regard 
to  problems  that  have  multiple  solutions  and  solution 
approaches,  it  will  not  be  possible  to  devise  systematic 
and  systemic  means  of  supporting  learning  effectively 
(Spector,  2006;  Spector  and  Koszalka,  2004). 

Associated  with  rapid  technological  and  societal 
changes,  life-long  learning  in  informal  and  professional 
settings  will  become  increasingly  important.  This  poses 
new  challenges  to  the  field  of  educational  communica¬ 
tions  and  technology,  which  in  the  past  mainly  focused 
on  learning  in  more  or  less  formal  and  much  less 
dynamic  settings.  A  consequence  of  this  emphasis  on 
complex  and  informal  learning  will  be  new  demands 
for  skilled  instructional  and  performance  technologists 
as  well  as  focused  research  in  these  areas.  We  will 
address  some  of  these  demands  subsequently. 

CRITICAL  FOUNDATIONS  FOR 
RESEARCH  AND  DEVELOPMENT 

Perhaps  the  most  critical  foundation  for  research  and 
development  in  educational  communications  and  tech¬ 
nology  are  sound  scientific  and  engineering  practices. 
Science  basically  involves  the  development  and  testing 
of  theories  that  explain  or  predict  a  range  of  phenom¬ 
ena.  Engineering  involves  the  systematic  application  of 
these  theories  in  the  design  and  development  of  solu¬ 
tions  to  practical  problems.  Without  firm  grounding  in 
both  science  and  engineering,  instructional  design 
(broadly  conceived)  is  not  likely  to  thrive  or  contribute 
significantly  to  improving  learning  and  performance. 

We  know  that  technology  will  change.  Technology 
will  change  what  we  do  and  what  we  can  do.  This  is 
true  for  learning  and  instruction  just  as  it  is  true  for 
shopping  and  entertainment.  Learning  tools  are  chang¬ 
ing.  Learning  tasks  are  changing.  Learning  perspec¬ 
tives  are  changing.  The  task  is  to  have  a  strong  foun¬ 
dation  when  these  “winds  of  changes  shift”  (to  borrow 
from  Bob  Dylan’s  “May  You  Stay  Lorever  Young”). 
Part  of  a  stable  foundation  is  firm  grounding  in  science 
and  engineering  and  a  realistic  and  humble  sense  of 
what  works  when  and  why.  The  willingness  to  be 
wrong  and  explore  alternatives  is  an  important  piece 
of  the  foundation. 

Critical  for  future  research  is  to  find  an  optimal 
balance,  and  to  create  good  interfaces,  between  dif¬ 
ferent  types  of  research.  Educational  communica¬ 
tions  and  technology  is  an  eclectic  field,  in  which 
proper  theory  and  model  development  is  only  possi¬ 
bly  through  a  measured  interplay  among,  for  exam¬ 
ple,  experimental  studies,  proofs  of  concept,  design 
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experiments,  and  developmental  research.  Fruitful 
interfaces  between  research  communities  require  com¬ 
mon  vocabularies  and  design  languages.  As  humble 
instructional  technology  researchers,  we  should  be  suf¬ 
ficiently  flexible  to  choose  useful  parts  from  different 
paradigms  for  specific  instructional  and  research  pur¬ 
poses.  We  might  even  occasionally  consider  trying  to 
disprove  or  discredit  our  hypotheses  rather  than  exclu¬ 
sively  seeking  confirmatory  evidence. 

For  a  long  time,  atomistic  approaches  dominated 
the  field  of  educational  communications  and  technol¬ 
ogy.  The  new  focus  on  integrated  learning  objectives, 
higher  order  skills  (problem  solving,  reasoning,  cre¬ 
ativity)  and  transfer  of  learning  requires  a  reconsider¬ 
ation  (Spector,  2001;  Spector  and  Anderson,  2000). 
Flolistic  approaches  to  learning  and  instruction  are 
expected  to  offer  better  opportunities  to  support  com¬ 
plex  learning. 

Finally,  enormous  progress  is  being  made  in  brain 
research,  and  methods  for  neurocognitive  imaging 
(e.g.,  functional  MRI)  yield  more  and  more  knowledge 
of  the  role  that  different  parts  of  the  brain  play  in 
learning.  Without  doubt,  this  will  influence  the  field  of 
learning  and  instruction  in  the  coming  decade.  There 
is  much  that  we  do  not  know  about  the  physical  mech¬ 
anisms  involved  in  learning.  A  particular  goal  of  edu¬ 
cational  technology  research  in  the  next  five  years 
ought  to  be  develop  close  ties  with  brain  researchers 
and  participate  actively  in  that  community  of  scholars. 

THE  STUDY  OF 
INSTRUCTIONAL  DESIGN 

We  have  made  frequent  use  of  the  terms  educational 
technology,  instructional  design ,  and  instructional 
technology  in  this  chapter.  In  some  cases,  we  used 
these  terms  interchangeably,  although  we  believe  they 
have  slightly  different  meanings.  Roughly  speaking, 
educational  technology  is  broad  in  scope  and  includes 
technologies  that  support  any  kind  of  learning  in  any 
environment.  The  term  instructional  technology  is 
more  narrowly  focused  on  the  use  of  technology  to 
support  specific,  intended,  and  planned  learning  out¬ 
comes.  Instructional  design,  broadly  conceived,  is  the 
discipline  devoted  to  the  study  of  these  technologies 
and  their  creation  and  use  in  support  of  learning  and 
instruction.  What,  then,  constitutes  the  proper  study  of 
instructional  design/systems/technology?  One  could 
focus  on  the  objects  involved,  such  as  learning  activ¬ 
ities,  materials,  supports,  technologies,  and  so  on. 
These  certainly  need  to  be  categorized  and  character¬ 
ized.  A  more  challenging  enterprise  is  studying  these 
various  kinds  of  objects  in  use.  What  do  particular 


learners  do  with  specific  objects  and  how  does  that 
contribute  to  improved  learning  or  performance? 

One  could  focus  on  processes  involved  in  devel¬ 
oping  these  various  objects.  Some  processes  might  be 
more  efficient  than  others.  Some  processes  might  jeop¬ 
ardize  quality  while  others  might  not  be  replicable. 
Closely  related  is  the  question  of  how  design  processes 
are  best  supported.  What  are  the  requirements  for  use¬ 
ful  design  models  and  tools? 

Both  the  objects  involved  in  instructional  design 
and  the  processes  used  to  develop  those  objects  are 
influenced  by  many  factors,  such  as  tasks  to  be  taught, 
target  group  characteristics,  and  learning  contexts.  In 
modem  society,  these  factors  are  subject  to  enormous 
changes,  such  as  a  change  from  well-structured  pro¬ 
cedural  tasks  to  ill-structured  problem-solving  tasks, 
from  rather  homogeneous  target  groups  (in  terms  of 
age,  prior  knowledge,  cultural  background,  etc.)  to 
highly  heterogeneous  target  groups,  and  from  formal 
settings  to  informal  and  professional  settings  that  are 
becoming  increasingly  important  for  lifelong  learning. 
These  changes  have  great  impact  on  the  study  of 
objects  and  processes  in  instructional  design. 

Finally,  the  study  of  instructional  design  includes 
the  study  of  designers.  The  performance  of  instruc¬ 
tional  designers  may  vary  greatly  depending  on  their 
experience.  Even  more  important,  only  a  very  small 
part  of  all  instruction  is  developed  by  designers  with 
some  formal  preparation.  By  far  the  biggest  part  of 
instruction  is  developed  by  designers-by-assignment 
and  teachers.  Moreover,  instruction  may  be  developed 
by  one  individual  but  will  more  typically  be  developed 
by  a  team  including  content  experts,  programmers, 
graphical  artists,  and  so  forth.  Changes  in  objects  and 
processes  of  instructional  design  will  no  doubt  have 
an  important  impact  on  the  work  of  designers  and 
design  teams,  the  division  of  roles  in  design  teams, 
and  required  instructional  design  competencies  of 
team  members. 

Yet  another  conceptualization  of  the  enterprise  of 
instructional  design  is  to  consider  three  distinct  activ¬ 
ities:  (1)  developing  tools  and  artifacts  to  support 
instruction,  (2)  demonstrating  the  utility  and  efficacy 
of  these  tools  in  designing  and  developing  instruction, 
and  (3)  predicting  and  evaluating  the  impact  of  the  use 
of  these  tools  on  learning  and  instruction.  This  way  of 
thinking  about  instructional  design  proceeds  roughly 
along  a  continuum  from  the  practice  of  instructional 
development  (using  instructional  technology)  to 
instructional  technology  research  (the  study  of  what 
works  when  and  why)  and  eventually  to  theories  about 
various  aspects  of  instructional  design  (determining 
how  can  we  make  tools  and  technologies  that  system¬ 
atically  improve  learning  and  instruction).  One  might 
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argue  that  such  a  continuum  might  correspond  to 
undergraduate  education,  to  master’s-level  training, 
and  to  doctoral-level  training.  How  might  we  think 
about  training  at  these  three  levels? 


PREPARING  PRACTITIONERS 
AND  RESEARCHERS 

How  can  we  adequately  prepare  practitioners  and 
researchers  for  productive  careers  in  the  first  half  of 
the  21st  century?  Graduate  programs  of  study  in 
instructional  design/systems/technology  vary  consid¬ 
erably  and  are  undergoing  continual  change.  Various 
professional  associations  are  developing  suggested 
curricula  standards  for  educational  technology  pro¬ 
grams— for  example,  the  IEEE  Technical  Committee 
on  Learning  Technology  (LTTC;  http://lttf.ieee.org/). 

Evidence  shows  that  most  instructional  design 
practitioners  have  very  little  formal  preparation.  They 
have  become  instructional  designers  by  being  assigned 
that  position  or  job— possibly  as  a  reward  for  superior 
performance  on  the  job  or  possibly  because  no  one 
else  was  available  or  wanted  the  task.  Actually,  this 
fits  the  general  picture  that  nowadays  employees 
change  jobs  more  often.  In  addition,  knowledge  of 
instructional  design/systems/technology  is  becoming 
obsolete  at  an  increasingly  faster  rate— just  as  the 
knowledge  in  many  other  professional  fields.  This 
clearly  indicates  a  need  for  increasingly  more  flexible 
educational  technology  programs,  in  terms  of  both 
time  and  place,  but  also  in  terms  of  content  (e.g.,  via 
accreditation  of  prior  learning,  or  APL). 

The  further  development  of  electronic  job  aids, 
performance  support  systems,  and  computer-based 
design  tools  may  also  contribute  to  lifelong  learning 
in  the  field  of  instructional  design/systems/technology. 
This  signifies  an  interesting  and  growing  relationship 
between  the  fields  of  educational  communications  and 
technology  and  human  performance  technology  (HPT; 
see  www.ispi.org).  On  the  one  hand,  life-long  learning 
in  nonformal  settings  is  best  supported  by  a  combina¬ 
tion  or  integration  of  approaches  rooted  in  both  disci¬ 
plines;  on  the  other  hand,  HPT  may  contribute  to  the 
development  of  more  flexible  approaches  to  teaching 
educational  technology. 

It  is  common  for  people  to  believe  that  they  know 
how  to  teach  something  at  which  they  are  reasonably 
competent.  This  belief  in  rampant  in  universities  — 
because  so-and-so  is  an  expert  logician,  many  believe 
that  person  will  be  a  good  logic  instructor.  The  tran¬ 
sition  from  people  believing  they  are  good  at  teaching 
things  they  know  and  understand  to  believing  that  they 
know  how  to  plan  effective  instructional  supports  and 


learning  activities  is  all  too  easy.  The  real-world  situ¬ 
ation  is  that  instructor  training  is  grossly  neglected 
outside  teacher  preparation  programs,  and  in  those  pro¬ 
grams  all  too  little  emphasis  is  placed  on  instructor 
competencies  (Klein  et  al.,  2004). 

Designers-by-assignment*  (in  contrast  with  those 
who  made  a  career  choice  to  pursue  instructional 
design  and  who  have  significant  professional  prepara¬ 
tion)  have  managed  to  develop  some  impressive  learn¬ 
ing  environments ,  especially  when  working  in  a  well- 
defined  area  with  prior  examples  of  effective  instruc¬ 
tion.  Changes  in  society,  however,  increasingly 
demand  new  training  environments  for  new  tasks  — 
prior  training  models  and  methods  are  not  easily 
applied  in  such  situations,  and  designers-by-assign- 
ment  struggle  to  develop  effective  instruction.  What  is 
needed  is  widespread  recognition  that  professional 
preparation  is  required  for  instructional  designers  — 
that  instructional  design  is  an  established  trade  with 
established  competencies  (Richey  et  al.,  2001;  see  also 
Chapter  42  in  this  Handbook).  In  addition,  what  is  also 
required  are  professional  preparation  programs  that 
train  practitioners  in  those  competencies.  Many  grad¬ 
uate  programs  in  instructional  design  do  address  com¬ 
petencies  at  the  master’s  level,  but  the  emphasis  in 
many  graduate  programs  is  on  research  as  opposed  to 
practice.  The  best  programs  emphasize  both  practice 
and  research  in  recognition  of  the  dual  need  for 
research  to  inform  practice  and  for  practice  to  inform 
research. 

What  are  the  requirements  for  effective  preparation 
in  instructional  design?  As  just  suggested,  it  is  impor¬ 
tant  to  realize  that  practitioners  have  different  require¬ 
ments  than  do  researchers,  for  whom  this  Handbook 
is  written.  Serving  both  in  the  same  graduate  program 
is  a  challenge.  A  more  natural  progression  in  terms  of 
instructional  design  education  and  training  might  be 
to  address  basic  competencies  in  an  undergraduate  pro¬ 
gram.  This  is  especially  true  if  one  believes  that 
instructional  design  is  a  profession  with  associated 
knowledge  and  skills  (e.g.,  theories,  principles,  mod¬ 
els,  standards,  competencies),  with  professional  soci¬ 
eties  organized  around  that  body  of  knowledge  and 
skills,  with  examples  of  best  practice  and  what  works, 
and  with  research  aimed  at  improving  practice  and 
fundamental  understanding.  This  would  seem  to  be  the 
case  with  instructional  design,  so  we  find  the  paucity 
of  undergraduate  programs  in  instructional  design  or 
educational  technology  to  be  an  anomaly.  A  relatively 


*  The  term  designer-by-assignment  was  introduced  by  M.  David  Merrill 
to  describe  the  common  practice  of  assigning  a  subject-matter  specialist 
or  someone  with  particular  technology  skills  the  task  of  designing  and 
developing  instructional  materials  and  associated  technology  support. 
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new  undergraduate  program  in  educational  science  at 
the  University  of  Freiburg  is  aimed  at  this  gap  (http:// 
www.ezw.uni-freiburg.de/),  and  we  need  more  such 
programs  at  the  undergraduate  level.  That  we  do  not 
have  more  programs  should  cause  some  concern 
within  the  instructional  design  community.  The  IEEE 
Technical  Committee  on  Learning  Technology  (http:// 
lttf.ieee.org/)  agrees  and  has  initiated  an  effort  to  spec¬ 
ify  curriculum  standards  for  undergraduate  preparation 
of  educational  technologists.  It  is  hoped  that  this  effort 
will  lead  universities  to  develop  associated  programs 
of  study.  When  the  initial  effort  completes  its  work 
(probably  in  2008),  the  committee  will  then  address 
graduate  and  postgraduate  programs  of  study  for  pro¬ 
fessional  educational  technologists. 

A  common  conception  of  an  instructional  technol¬ 
ogist  is  someone  who  knows  how  to  use  particular 
technologies.  As  technologies  become  easier  to  use 
and  more  widely  accessible,  the  number  of  instruc¬ 
tional  technologists,  according  to  this  common  con¬ 
ception,  will  increase.  Developments  in  instructional 
technology  often  follow  developments  in  computer 
science  with  about  a  generation  delay.  As  computer 
science  matured  as  a  discipline,  it  shifted  its  focus  from 
training  programmers  to  studying  computing  and  the 
use  of  programs  and  other  tools. 

Instructional  technology  is  reaching  a  similar  cross¬ 
roads.  Designers-by-assignment  constitute  a  significant 
practitioner  group.  Even  with  successful  programs  like 
the  one  at  the  University  of  Freiburg,  and  even  with 
new  undergraduate  programs  in  educational  technology 
that  follow  new  and  innovative  curriculum  standards, 
we  will  still  have  many  designers-by-assignment.  One 
response  from  academia  would  be  for  graduate  training 
in  instructional  design  to  shift  its  focus  from  training 
instructional  designers  to  the  study  of  instruction  and 
instructional  technologies,  including  training  advanced 
students  in  the  creation  of  instructional  design  tools. 

If  one  obtains  a  master’s  degree  in  instructional 
technology  and  goes  to  work  in  a  training  department, 
one  is  likely  to  spend  little  time  developing  instruc¬ 
tion— that  will  be  left  to  the  designers-by-assignment. 
One  is  more  likely  to  quickly  become  a  training  man¬ 
ager  responsible  for  hiring  and  overseeing  designers- 
by-assignment.  Do  graduate  programs  in  instructional 
design/systems/technology  properly  prepare  profes¬ 
sionals  for  such  positions?  At  best,  one  might  argue  that 
a  few  programs  include  some  emphasis  on  being  an 
effective  training  manager.  Even  fewer  focus  on  training 
graduates  to  create  tools  to  be  used  by  instructional 
designers— especially  by  designers-by-assignment. 

At  the  master’s  level,  the  emphasis  should  shift 
from  training  students  to  be  users  of  instructional  tech¬ 
nology  to  preparing  them  to  manage,  supervise,  and 


inspire  those  who  use  instructional  technology 
(designers-by-assignment) .  The  emphasis  for  master’s 
students  includes  a  detailed  study  of  empirically  veri¬ 
fied  theory  and  a  challenge  to  develop  technology- 
based,  learning-oriented  instructional  design  and 
development  tools  that  can  be  used  by  designers-by- 
assignment  and  nonspecialists 

What  constitutes  a  proper  dissertation  study?  Too 
many  projects  at  American  institutions  consist  of  a 
single  empirical  study  or  survey;  this  is  less  true  in 
European  institutions  which  typically  involve  doc¬ 
toral  students  in  a  series  of  studies  and  publications. 
A  doctoral  student  in  instructional  design  should  be 
able  to  identify,  modify,  and  develop  an  instructional 
design  theory  (this  corresponds  to  an  advanced 
instructional  design  competency  according  to  the 
International  Board  of  Standards  for  Training,  Per¬ 
formance,  and  Instruction;  see  Richey  et  al.,  2001). 
A  doctoral  student  should  conduct  extensive  product 
and  research  literature  reviews  related  to  the  theory 
of  interest.  In  addition,  a  doctoral  student  should 
conduct  additional  original  empirical  research  related 
to  the  theory  development.  Doctoral  students  might 
also  develop  tools  that  implement  the  theory  in  an 
appropriate  context  or  setting.  The  students  could 
then  demonstrate  the  use  of  these  tools  for  the  design 
of  instruction  and  evaluate  or  supervise  the  evaluation 
of  instructional  products  developed  by  the  use  of 
these  tools  in  a  field  setting. 

What  is  the  proper  study  of  instructional  design? 
If  we  conceive  of  instructional  design  as  that  discipline 
that  studies  instructional  technology,  then  we  might 
conclude  that  the  focus  of  instructional  design  ought 
to  be  on  tools  and  artifacts,  with  particular  emphasis 
placed  on  the  creation,  use,  and  impact  of  tools  and 
artifacts,  much  as  one  would  expect  to  find  in  an 
advanced  engineering  program.  The  goal  of  the  engi¬ 
neer  or  designer  is  to  design  useful  artifacts  and  tools, 
and  then  to  predict  their  performance  and  their  impact 
on  those  who  use  them  (Vincenti,  1990). 


CLOSING  WORDS 

In  closing,  we  hope  this  volume  helps  to  establish 
meaningful  and  useful  foundations  for  ongoing  and 
future  research  and  development  with  regard  to  edu¬ 
cational  communications  and  technologies.  We  hope; 
we  do  not  know.  We  are  grateful  for  the  contributions 
of  so  many  talented  researchers.  Working  on  this 
research  handbook  these  last  two-plus  years  has  been 
a  growing  experience  for  all  of  us.  Of  course,  we  had 
our  ideas  about  the  various  topics  discussed  herein, 
and  we  began  with  excellent  input  and  guidance  from 
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David  Jonassen.  The  excellent  response  to  the  AECT- 
hosted  survey  about  Handbook  usage  gave  us  a  sense 
that  we  had  undertaken  a  significant  enterprise  that 
was  of  keen  interest  to  our  community  of  researchers, 
practitioners,  teachers,  and  students. 
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Abstract  mathematical  knowledge :  Mathematics  as  a 
body  of  knowledge  that  may  be  experienced  as 
experientially  real. 

Activation  principle:  Learning  is  promoted  when  learn¬ 
ers  activate  relevant  cognitive  structures. 

Active  learning:  Proponents  of  experiential  learning 
often  refer  to  active  learning,  and  by  this  phrase 
they  refer  to  the  participatory  nature  of  experiential 
learning;  learners  take  an  active  role  in  their  own 
learning,  and  this  often  also  implies  that  learners 
have  to  take  some  responsibility  for  their  own 
learning  and  advancement  of  understanding. 

Adaptive  hypermedia  systems  (AHS):  Combining 
micro-adaptive  systems  and  hypermedia  systems 
to  provide  adaptive/adaptable,  hybrid  features  by 
presenting  learners  with  choices,  along  with  guid¬ 
ance. 

Adaptive  instructional  systems:  Any  forms  of  educa¬ 
tional  intervention  aimed  at  accommodating  indi¬ 
vidual  learner  differences. 

Adaptivity:  The  capability  exhibited  by  an  organic  or 
an  artificial  organism  to  alter  its  behavior  according 
to  the  environment.  In  the  context  of  an  instruc¬ 
tional  system,  this  capability  allows  the  system  to 
alter  its  behavior  according  to  learner  needs  and 
other  characteristics.  This  is  typically  represented 
within  a  learner  model. 

Analytical  science:  Within  this  perspective  researchers 
aim  to  develop,  test,  and  justify  theories;  see  also 
design  science  and  situated  science. 

Animated  pedagogical  agents:  Talking  heads  with 
speech,  facial  expressions,  and  gestures  that  imple¬ 
ment  pedagogical  strategies. 

Application  principle:  Learning  is  promoted  when 
learners  engage  in  the  application  of  their  newly 
acquired  knowledge  or  skill. 

Apprenticeship:  A  process  through  which  a  more  expe¬ 
rienced  person  assists  a  less  experienced  one  by 
way  of  demonstration,  support,  and  examples. 


Also,  a  traditional  learning  paradigm  in  which  a 
student  learns  by  working  with  a  master;  this 
method  of  education  was  practiced  widely  in  the 
Middle  Ages .  Also ,  a  skilled  practical  trade  through 
participation  in  practice. 

Aptitude-treatment  interactions  (ATI):  Adapting  spe¬ 
cific  instructional  procedures  and  strategies  to  spe¬ 
cific  learner  characteristics  (or  aptitudes). 

Articulation:  In  cognitive  apprenticeship,  verbalizing 
the  results  of  reflective  acts. 

Artifacts:  Designed  objects  or  systems,  including  those 
created  in  the  process  of  design  and  those  resulting 
from  the  act  of  design. 

Assessment:  The  systematic  observation  of  achieve¬ 
ment  using  any  of  a  number  of  formats,  including 
paper,  technology,  or  live  judgment. 

Assessment  criteria:  Describe  the  aspects  of  perfor¬ 
mance  that  will  be  assessed. 

Assessment  of  learning:  Measuring  learning  achieve¬ 
ment,  performance,  outcomes,  and  processes  by 
many  means. 

Assessment  standards:  Describe  the  quality  of  perfor¬ 
mance  on  each  of  those  aspects  that  can  be 
expected  of  participants  at  different  stages  (e.g., 
age,  grade)  based  on  a  participant’s  past  perfor¬ 
mance  (self-referenced) ,  peer  group  performance 
(norm-referenced),  or  an  objective  standard  (crite¬ 
rion-referenced)  . 

Asynchronous  learning:  Communication  between 
learners  and  instructors  that  does  not  take  place 
simultaneously  or  in  real  time;  an  example  of  asyn¬ 
chronous  learning  is  when  a  learner  engages  in  a 
self-paced,  self-service  learning  module  without 
communicating  with  another  person. 

Attention:  Arousal  and  intention  in  the  brain  which 
influence  an  individual’s  learning  processes;  with¬ 
out  active,  dynamic,  and  selective  attending  of 
environmental  stimuli,  it  follows  that  meaning  gen¬ 
eration  cannot  occur. 


*  This  glossary  of  terms  represents  a  consolidated  list  of  keywords  with  definitions  taken  from  the  individual  chapters  of  this  Handbook.  The  editors 
have  made  only  very  occasional  and  very  minor  changes  to  definitions  provided  by  authors.  Multiple  definitions  by  different  authors  are  indicated 
by  the  use  of  “Also”  within  a  definition. 
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Authenticity:  Learner-perceived  relations  between  the 
associated  practices  and  one’s  projected  or  envi¬ 
sioned  use  value  of  those  practices. 

Authoring  tool:  A  software  application  used  by  non¬ 
programmers  to  assemble  digital  media  files  into 
displays,  presentations,  and  interactive  exercises. 

Automated  instructional  design  (AID):  Leveraging 
technology  to  automate  the  instructional  design 
as  part  of  the  process  of  learning  content  devel¬ 
opment. 

Automated  knowledge:  About  how  to  do  something; 
with  repetition  it  operates  outside  of  conscious 
awareness  and  executes  much  faster  than  con¬ 
scious  processes. 

Baddeley’s  memory  model:  An  information-processing 
model  that  emphasizes  the  different  short-term 
memory  stores  for  visual  (the  sketchpad)  and  audi¬ 
tory  (the  phonological  loop)  information. 

Behaviorism:  An  objectivist  and  monist  perspective 
with  regard  to  individual  actions  and  decisions. 

Blended  learning  environment:  A  learning  environ¬ 
ment  that  combines  face-to-face  instruction  with 
technology-mediated  instruction. 

Blog:  From  the  term  “Web  log”;  originally  a  Web- 
based  diary  but  now  often  used  to  refer  to  the 
software  designed  to  support  it  that  presents  Web 
pages  in  a  reverse  chronological  order. 

Change  agentry:  The  activity  of  facilitating  change. 

Clinical  education:  Supervised  acquisition  of  clinical 
skills. 

Coaching:  In  cognitive  apprenticeship,  assisting  and 
supporting  learners’  cognitive  activities. 

Cognitive  abilities:  Abilities  to  perform  any  of  the 
functions  involved  in  cognition  whereby  cognition 
can  be  defined  as  the  mental  process  of  knowing, 
including  aspects  such  as  awareness,  perception, 
reasoning,  and  judgment. 

Cognitive  apprenticeship:  An  apprenticeship  process 
that  utilizes  cognitive  and  metacognitve  skills  and 
processes  to  guide  learning. 

Cognitive  constructivism:  A  form  of  realism  that  stress 
the  reorganization  of  mental  structures  of  an  indi¬ 
vidual  making  sense  of  the  world. 

Cognitive  load  theory:  A  cognitive  model  of  informa¬ 
tion  processing  that  emphasizes  a  conceptual  men¬ 
tal  workload  in  understanding  human  thought. 
Also,  an  instructional  design  theory  based  on  our 
knowledge  of  human  cognitive  architecture. 

Cognitive  models:  Descriptions  of  human  thought  pro¬ 
cesses  via  metaphorical  constructs  that  may  or 
may  not  represent  actual  biological  structures;  the 
value  of  any  model  is  judged  by  its  utility  in  rep¬ 
resenting  or  predicting  actual  thought,  not  by  the 
degree  of  accuracy  in  depicting  brain  structure. 


Cognitive  task  analysis:  Interview  and  observation 
protocols  for  extracting  implicit  and  explicit 
knowledge  from  experts  for  use  in  instruction  and 
expert  systems. 

Cognitive  training  model:  A  five-task  instructional 
design  model. 

Cognitivism:  An  objectivist  and  rationalist  perspective 
with  regard  to  individual  cognitive  structures. 

Collaboration:  Activity  involving  multiple  people 
developing  shared  meaning  while  working 
together  on  a  common  problem;  often  involves 
harmonious  cooperation  but  is  not  contingent 
upon  it. 

Collaborative  learning:  Students  working  together  to 
maximize  their  own  and  each  other’s  learning  (i.e., 
to  achieve  shared  learning  goals;  synonymous 
with  cooperative  learning  for  many). 

Collective  data  collection:  Obtaining  data  from  indi¬ 
vidual  group  members;  data  are  later  aggregated 
or  manipulated  into  a  representation  of  the  group 
as  a  whole. 

Community  of  practice:  A  group  of  people  bound  by 
participation  in  an  activity  common  to  them  all; 
may  be  formal  or  informal. 

Competency :  A  knowledge,  skill,  or  attitude  that 
enables  one  to  effectively  perform  the  activities  of 
a  given  occupation  or  function  to  the  standards 
expected  in  employment. 

Competency  development:  A  feature  of  the  holistic 
approach,  indicating  that  educational  programs 
should  be  aimed  at  the  development  of  competen¬ 
cies  rather  than  teaching  different  topics  in  differ¬ 
ent  courses. 

Competitive  learning:  Students  working  individually 
to  achieve  a  grade  or  recognition  attainable  by  or 
restricted  to  only  a  few. 

Complex:  Pertains  to  a  group  of  multiple  independent 
entities  with  interrelated  functions  seeking  a  com¬ 
mon  goal  through  adaptive  processes. 

Complex  performance:  Refers  to  real-world  activities 
that  require  the  integration  of  disparate  measure¬ 
ment  instrumentation  as  well  as  the  need  for  time- 
critical  experimental  control. 

Complex  tasks:  Tasks  where  performance  requires  the 
integrated  use  of  both  controlled  and  automated 
knowledge  to  perform  tasks  that  often  extend  over 
many  hours  or  days. 

Complicated:  Consisting  of  many  interconnecting 
parts  or  elements;  intricate;  involving  many  dif¬ 
ferent  and  confusing  aspects. 

Component  investigation:  The  identification  or  confir¬ 
mation  through  research  of  steps  (procedural  mod¬ 
els)  and  factors  (conceptual  models)  critical  to  the 
instructional  design  process. 
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Computer-mediated  communication  (CMC):  Commu¬ 
nication  between  two  or  more  individuals  with 
text -based  tools  such  as  e-mail,  instant  messaging, 
or  computer-based  conferencing  systems. 

Computer-supported  cooperative  learning:  The 
instructional  use  of  technology  combined  with  the 
use  of  cooperative  learning. 

Conceptual  model:  A  theoretically  based  explanation 
of  a  natural  phenomenon. 

Conditions-based  instruction:  Described  by  Gagne  as 
internal  and  external  conditions  of  learning;  states 
possessed  by  the  learner,  such  as  prior  knowledge, 
are  internal  conditions,  and  instructional  supports 
designed  to  promote  learning  are  external  conditions. 

Construction-integration  model:  A  cognitive  model  for 
understanding  the  processing  of  text;  this  model 
suggests  a  continual  multilevel  process  of  building 
and  confirming  a  cohesive  mental  model  from  a 
text  document  and  a  reader’s  prior  knowledge. 

Constructive  controversy:  Occurs  when  group  mem¬ 
bers  have  different  information,  perceptions,  opin¬ 
ions,  reasoning  processes,  theories,  and  conclu¬ 
sions,  and  they  must  reach  agreement  to  make 
progress  or  proceed. 

Constructivism:  A  point  of  view  related  to  cognitive 
development;  cognitive  development  depends  on 
a  child’s  actions  on  his  or  her  own  surroundings 
to  change  and  actively  learn  from  those  surround¬ 
ings  (Piaget)  and  the  strong  relationship  between 
a  child’s  surroundings  and  his  cognitive  develop¬ 
ment  (Vygotsky).  Also,  a  set  of  assumptions  about 
human  learning  emphasizing  the  central  role  of 
the  mind’s  active  construction  of  new  knowledge. 

Contextualism:  A  philosophical  worldview  in  which 
any  event  is  interpreted  as  an  ongoing  act  insepa¬ 
rable  from  its  current  and  historical  context  and 
in  which  a  radically  functional  approach  to  truth 
and  meaning  is  adopted;  the  root  metaphor  of  con¬ 
textualism  is  the  act  in  context,  and  the  truth  cri¬ 
terion  of  contextualism  is  successful  working  or 
effective  action. 

Cooperative  learning:  Students  working  together  to 
maximize  their  own  and  each  other’s  learning  (i.e., 
to  achieve  shared  learning  goals;  synonymous 
with  collaborative  learning  in  this  chapter). 

Core  reference:  A  reference  that  is  of  central  interest 
to  researchers  in  a  particular  area  and  generally 
considered  to  be  a  primary  source  of  data  or  infor¬ 
mation;  core  references  are  indicated  by  an  aster¬ 
isk  in  the  reference  section  of  each  chapter  in  this 
Handbook. 

Curriculum  design:  A  process  of  conceiving  a  plan  to 
define  a  set  of  courses  constituting  an  area  of  spe¬ 
cialization  that  supports  the  specified  learning  goal. 


Cybernetics:  System  theory  concerned  with  the  issues 
of  regulation,  order,  and  stability  that  face  us  in 
the  treatment  of  complex  systems  and  processes. 

Cycle  of  instruction:  The  activation-demonstration- 
application-integration  cycle  of  first  principles. 

Declarative  knowledge:  Knowledge  about  what  or 
why;  hierarchically  structured  and  formatted  as 
propositional,  episodic,  or  visuospatial  informa¬ 
tion  that  is  accessible  in  long-term  memory  and 
consciously  observable  in  working  memory. 

Demonstration  principle:  Learning  is  promoted  when 
learners  observe  a  demonstration  of  the  skills  to 
be  learned. 

Descriptive  accounts:  Analysis  of  naturally  occurring 
instances  of  human  activity. 

Descriptive  contextualism:  A  variant  of  contextualism 
that  has  as  its  primary  goal  an  understanding  of 
the  complexity  and  richness  of  a  whole  event 
through  an  appreciation  of  its  participants  and  fea¬ 
tures.  It  seeks  the  construction  of  knowledge  that 
is  specific,  personal,  ephemeral,  and  spatiotempo- 
rally  restricted,  like  a  historical  narrative. 

Design  and  development  research:  Research  that  seeks 
to  create  knowledge  grounded  in  data  systemati¬ 
cally  derived  from  practice  and  based  upon  a  sys¬ 
tematic  analysis  of  specific  cases. 

Design  and  development  team:  The  group  of  people 
and  skill  sets  involved  in  the  research-based  devel¬ 
opment  of  artifacts  and  systems  in  education. 

Design-in-the-large  (DIL):  Refers  to  the  goal  of 
changing  existing  situations  into  preferred  ones; 
see  design-in-the-small  (DIS). 

Design-in-the- small  (DIS):  Refers  to  the  design  of  a 
specific  innovation;  see  design-in-the-large  (DIL). 

Design  language:  A  set  of  abstractions  used  to  give 
structure,  properties,  and  texture  to  solutions  of 
design  problems;  designs  are  expressed  in  terms 
of  design  languages. 

Design  layer:  One  aspect  of  a  decomposed  design  prob¬ 
lem  that  can  be  approached  using  one  or  more 
design  languages;  a  subdomain  of  the  larger  prob¬ 
lem  to  which  one  or  more  design  languages  pertains. 

Design  principles:  Research-based  guidelines  for 
instructional  design;  design  principles  can  be 
articulated  at  different  grain  sizes:  Specific  prin¬ 
ciples  characterize  rationales  for  designing  spe¬ 
cific  features  in  a  learning  environment;  pragmatic 
principles  connect  rationales  behind  several  fea¬ 
tures;  meta-principles  synthesize  a  cluster  of  prag¬ 
matic  principles. 

Design  science:  Within  this  perspective  researchers 
aim  to  build  and  evaluate  artifacts  for  well-defined 
contexts  of  use;  see  analytical  science  and  situated 
science. 
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Design  theory:  The  concepts  and  principles  that  help 
to  develop  strategies  and  methods  for  designing. 

Development  model:  A  recommended  set  of  activities 
or  tasks  that  define  a  process  for  successful 
instructional  design. 

Direct  process  measure:  Continuous  elicitation  of  data 
from  the  beginning  to  end  of  a  group  process. 
Direct  process  measures  involve  videotaping, 
audiotaping,  direct  researcher  observation,  or  a 
combination  of  these  methods. 

Discovery  learning  environment:  Environment  that 
provides  students  with  a  set  of  events  or  data  that 
they  can  explore  independently  or  in  a  prestruc¬ 
tured  way  to  discover  the  regularities  in  these  data; 
a  rich  learning  environment  that  facilitates  chil¬ 
dren’s  own  constructive  endeavors  to  learning 
experiences. 

Distance  education:  An  educational  program  charac¬ 
terized  by  the  separation,  in  time  or  place,  between 
instructor  and  student  and  in  which  communica¬ 
tions  media  are  used  to  allow  interchange. 

Distributed  perspective:  An  analytic  approach  to 
understanding  human  activity  that  distributes 
agency  across  people  and  material  artifacts. 

Domain-specific  theories:  Theories  on  instruction  that 
are  tailored  to  a  specific  subject  matter  domain. 

Dual-coding  theory:  A  cognitive  model  of  information 
processing  that  emphasizes  the  unique  contribu¬ 
tions  of  verbal  and  visual  subsystems  in  under¬ 
standing  human  cognition. 

Dualism:  When  two  apparently  related  items  are 
treated  as  separate  and  distinct  (e.g.,  mind/body 
or  individual/environment). 

Dynamic  pretest:  A  test  delivered  online  that  can  lit¬ 
erally  adapt  courseware  to  learner  deficiencies 
identified  through  online  testing  and  scale 
courseware  to  match  specific  learner  needs. 

Educational  communications:  Forms,  means,  and 
methods  of  expressing  and  sharing  ideas,  informa¬ 
tion,  and  knowledge  to  support  learning  and 
instruction. 

Educational  technology  research:  Research  focus  on 
describing,  predicting,  understanding,  and  design¬ 
ing  effective  applications  of  technology  to  serve 
the  goals  of  education,  training,  and  performance 
support. 

Educational  technology:  The  disciplined  application 
of  scientific  principles  and  theoretical  knowledge 
to  support  and  enhance  human  learning  and  per¬ 
formance. 

Effect  size:  A  statistical  measure  of  the  difference 
between  the  mean  of  the  control  group  and  the 
mean  of  the  experimental  group  in  a  quantitative 
research  study. 


e-Learning  principles:  Prescriptive  principles  for 
designing  e-learning;  see  multimedia  learning 
principles. 

Electronic  performance  support  systems  (EPSS):  An 
enabler  of  work  tasks  delivered  by  electronic  tech¬ 
nology  provided  to  individuals  or  teams  at  the  time 
of  need  on  the  job;  typical  support  includes  pro¬ 
cedural  guidance  or  references  to  factual  informa¬ 
tion  needed  to  complete  tasks. 

Embodiments:  Tactile  or  visual  models  that  are  meant 
to  represent  mathematical  relationships  and  con¬ 
cepts  in  a  ready  apprehensible  form. 

Emergent  modeling:  A  dynamic  approach  to  modeling 
in  mathematics  education,  in  which  the  models 
that  the  students  use  develop  from  models  of  infor¬ 
mal  mathematical  activity  into  models  for  more 
formal  mathematical  reasoning. 

Empiricism:  An  epistemology  that  states  that  knowledge 
comes  from  experience  and  through  the  senses. 

Epistemology:  How  we  come  to  know  about  what  exists . 

Ergonomics  of  learning  environments :  The  applied  sci¬ 
ence  that  helps  illuminate  the  relationships 
between  a  learner  and  a  learnplace. 

Evaluation:  The  process  of  gathering  information 
about  the  merit  or  worth  of  a  program  for  the 
purpose  of  making  decisions  about  its  effective¬ 
ness  or  for  program  improvement. 

Experiment:  A  design  that  involves  the  comparison  of 
one  treatment  to  another,  using  two  or  more  dif¬ 
ferent  groups. 

Expertise  development:  Acquisition  of  knowledge  and 
skill  in  a  domain. 

Exploration:  In  cognitive  apprenticeship,  forming  and 
testing  a  personal  hypothesis  in  pursuit  of  learning. 

External  validity:  The  degree  to  which  the  results  of 
an  experiment  can  be  generalized  to  other  settings. 

Extraneous  load:  In  cognitive  load  theory,  the  work¬ 
load  component  associated  with  information  that 
is  not  directly  relevant  to  a  particular  content  area. 

Feedback:  Information  on  goal  attainment  designed  to 
help  workers,  teams,  or  functional  units  monitor 
and  evaluate  their  progress  in  achievement  of 
desired  accomplishments.  Feedback  may  be  quan¬ 
titative  or  qualitative;  it  may  or  may  not  include 
explanations  to  guide  performance;  it  may  be 
directed  to  individual,  team,  unit,  or  organizational 
levels;  and  it  may  be  provided  through  personal 
communications  or  through  impersonal  channels 
such  as  charts  and  graphs  posted  in  work  areas  or 
by  computer. 

Feedback  ( cybernetic  definition):  Output  of  a  system 
that  is  “fed  back”  to  the  controller  of  the  system 
as  an  input  signal  to  regulate  the  system  with 
regard  to  a  reference  value. 
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Feedback  (instructional  context ):  Post-response  infor¬ 
mation  that  is  provided  to  a  learner  to  inform  the 
learner  on  his  or  her  actual  state  of  learning  or 
performance. 

First  principles  of  instruction:  Five  principles  funda¬ 
mental  to  effective,  efficient,  and  engaging  learn¬ 
ing. 

Flexible  learning:  Learning  that  is  relatively  free  of 
logistical  and  educational  constraints. 

Formalism:  The  formal  structure  and  abstract  princi¬ 
ples  that  underlie  the  conceptual  framework  of  the 
content  area;  for  example,  the  concept  of  erosion 
is  a  formalism  in  science,  division  is  a  formalism 
in  mathematics,  and  metaphor  is  a  type  of  formal¬ 
ism  in  language  arts. 

Four-component  instructional  design  model  (4C/ID): 
An  instructional  design  model  that  distinguishes 
recurrent  from  non-recurrent  tasks  and  emphasizes 
whole-task  practice  on  complex  cognitive  skills. 

Functional  contextualism:  A  philosophy  of  science 
and  variant  of  contextualism  that  has  as  its  pri¬ 
mary  goal  the  prediction  and  influence  of  events 
with  precision,  scope,  and  depth  using  empiri¬ 
cally  based  concepts  and  rules.  It  seeks  the  con¬ 
struction  of  knowledge  that  is  general,  abstract, 
and  spatiotemporally  unrestricted,  like  a  scien¬ 
tific  principle. 

Gaming:  Taking  part  in  open-ended  activities  that  give 
opportunity  to  elicit  conceptual  learning. 

Generational  differences:  The  theory  that  people  born 
within  an  approximately  20-year  time  period  share 
a  common  set  of  characteristics  based  on  the  his¬ 
torical  experiences,  economic  and  social  condi¬ 
tions,  technological  advances,  and  other  societal 
changes  they  have  in  common;  the  term  first  came 
into  popularity  in  the  1960s  when  it  was  used  to 
distinguish  the  rebellious  Baby  Boomer  genera¬ 
tion  from  their  parents. 

Germane  load:  In  cognitive  load  theory,  the  workload 
component  associated  with  strategies  that  require 
processing,  but  in  doing  so  make  the  relevant  con¬ 
tent  more  accessible. 

Goal  setting  theory:  Guidelines  for  optimizing 
worker  or  team  performance  through  setting  of 
specific  and  difficult  goals.  Goal  setting  theory 
considers  the  effects  of  self-efficacy,  goal  com¬ 
mitment,  feedback,  and  incentives  on  goal  effec¬ 
tiveness. 

Group:  Two  or  more  individuals  working  together  to 
achieve  a  common  goal. 

Group-based  learning  (asynchronously):  A  mode  of 
learning  that  enables  individuals  to  learn  in  groups 
with  online  technologies  in  their  own  time,  at  their 
own  pace,  and  from  their  own  place. 


Group-based  learning  ( synchronously ):  A  mode  of 
learning  which  enables  individuals  to  learn  in 
groups  with  online  technologies  at  the  same  time 
and  at  the  same  pace  as  that  of  the  group  but  from 
their  own  place. 

Group  learning  process:  Actions  and  interactions  per¬ 
formed  by  group  members  during  the  group  learn¬ 
ing  task. 

Groupware  authoring:  Concurrent  process  of  creating 
learning  content  in  a  team  environment  with  mul¬ 
tiple  authors  often  serving  in  different  roles  such 
as  writers,  graphic  artists,  instructional  designers, 
etc. 

Hard  technologies:  Represent  devices  may  be  used  in 
adaptive  systems  to  capture  learner  information 
(e.g.,  eye-tracking  devices)  or  present  content  to 
a  learner  (e.g.,  tactile  tablet);  can  be  used  to  detect 
and  classify  learners’  performance  data  or  affec¬ 
tive  states  such  as  confusion,  frustration,  excite¬ 
ment,  disappointment,  boredom,  confidence,  con¬ 
tentment,  and  so  on. 

Hierarchical  hypertext:  A  hypertext  structure  in  which 
content  is  ordered  relative  to  the  concepts  pre¬ 
sented.  The  concepts  presented  on  a  screen  link  to 
superordinate  (more  general)  concepts,  or  subor¬ 
dinate  (more  specific)  concepts. 

Holistic  data  collection:  Obtaining  data  from  the  group 
as  a  whole.  As  this  type  of  data  collection  results 
in  a  representation  of  the  group  rather  than  indi¬ 
vidual  group  member,  it  is  not  necessary  to  aggre¬ 
gate  or  manipulate  data. 

Human  cognitive  architecture :  The  manner  in  which 
structures  and  functions  required  for  human  cog¬ 
nitive  processes  are  organized. 

Hybrid  learning  environment :  Alternative  term  for 
blended  learning  environment. 

Hypertext:  Text-based  informational  screens  that  are 
presented  using  a  computer.  Informational  screens 
are  connected  to  each  other  using  links. 

Hypertext/hypermedia:  Pages  of  text  and  other  media 
with  hot  spots  that  users  can  click  on  and  access 
other  pages. 

Idealism:  A  view  of  reality  as  mental,  implying  that 
the  world  is  not  separate  from  the  mind. 

IMS  learning  design:  A  formal  instructional  design 
language  that  is  used  to  specify  the  design  of  a 
teaching  and  learning  process  in  a  machine  inter¬ 
pretable  way. 

Incentives:  Tangible  and  social  rewards  intended  to 
optimize  performance  of  individuals  or  teams;  may 
include  money,  feedback,  and  social  recognition. 

Indirect  process  measure:  Discrete  measure  at  a  spe¬ 
cific  point  in  time  during  the  group  process; 
often  involves  multiple  points  of  data  collection. 
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Indirect  process  measures  may  measure  processes, 
outcomes,  products,  or  other  factors  related  to 
group  process. 

Information  processing:  The  modeling  of  sensory 
input  and  cognitive  transformations  as  a  series  of 
processing  stages. 

Informative  tutoring  feedback:  Multiple-try  feedback 
strategies  providing  elaborated  feedback  compo¬ 
nents  that  guide  the  learner  towards  successful 
task  completion  without  offering  immediately  the 
correct  response. 

In-program  learner  behaviors:  Measures  of  learner 
activities  and  outcomes  while  engaged  in  an 
instructional  task. 

Inquiry  learning:  Students  actively  learn  by  asking 
questions  and  interpreting  answers  or  by  formu¬ 
lating  and  testing  hypotheses.  Also,  an  “approach 
to  learning  that  involves  a  process  of  exploring  the 
natural  or  material  world,  and  that  leads  to  asking 
questions,  making  discoveries,  and  rigorously  test¬ 
ing  those  discoveries  in  the  search  for  new  under¬ 
standing”  (National  Science  Foundation). 

Instructional  design:  A  purposeful  activity  that  results 
in  a  combination  of  strategies,  activities  and/or 
resources  to  facilitate  learning.  Also,  creating 
blueprints  for  effective,  efficient,  and  engaging 
instruction.  Also,  the  systematic  process  of  ana¬ 
lyzing,  designing,  developing,  implementing,  and 
evaluating  instruction;  also  known  as  instructional 
systems  design. 

Instructional  design  model:  A  coherent  set  of  mostly 
prescriptive  theoretical  statements  on  the  appro¬ 
priateness  of  particular  instructional  approaches  or 
interventions. 

Instructional  designer:  A  person  with  the  knowledge 
and  skills  to  design  effective  instruction. 

Instructional  strategies:  The  prescribed  sequences  and 
methods  of  instruction  to  achieve  a  learning  objec¬ 
tive. 

Instrumentation:  Hardware  devices  used  to  assist  with 
the  process  of  data  acquisition  and  measurement. 

Integrated  curriculum:  A  curriculum  based  on  a 
whole-task  approach  aimed  at  the  integration  of 
supportive  contents  with  whole  tasks,  knowledge, 
skills,  and  attitudes  and  at  integrating  first-order 
skills  with  higher-order  skills. 

Integration  principle:  Learning  is  promoted  when 
learners  integrate  their  new  knowledge  into  their 
everyday  life. 

Intelligent  tutoring  systems:  Intelligent  computer  sys¬ 
tems  that  model  the  learner’s  knowledge  and  skills 
at  a  fine-grained  level  and  that  adaptively  respond. 

Interactive  learning  task:  Tasks  providing  multiple 
response  steps  or  tries  and  instructional  compo¬ 


nents  such  as  feedback,  guiding  questions, 
prompts,  simulation  facilities,  and  so  on. 

Interactive  multimedia:  The  use  of  more  than  one  form 
of  media  (such  as  text,  visuals,  video,  animation, 
and  audio)  in  a  way  in  which  a  user  has  a  great 
deal  of  control  over  the  choice  or  progress  of  the 
program. 

Interactive  simulation:  Learners  manipulate  compo¬ 
nents  and  parameters  of  a  complex  system  and 
observe  what  happens  when  system  output  is  gen¬ 
erated. 

Internal  validity:  The  degree  to  which  the  results  of  a 
study  can  be  attributed  to  the  treatment  rather  than 
extraneous  or  confounding  variables. 

Intrinsic  load:  In  cognitive  load  theory,  the  workload 
component  associated  with  a  particular  content 
area  and  its  level  of  complexity. 

Keyword:  A  word  or  phrase  that  is  critically  important 
to  a  topic  area  or  which  has  a  particular  definition 
important  for  understanding  issues  and  further  dis¬ 
cussion. 

Knowledge-based  system:  A  computer  program  that 
can  reason,  based  on  a  database  of  knowledge 
acquired  from  a  human  expert  (also  known  as  an 
expert  system). 

Knowledge  communities:  Technology  focused  in 
bringing  communities  of  people  together  to  both 
generate  and  share  knowledge  about  a  shared 
interest. 

Knowledge  generation:  Generation  of  understanding 
through  developing  relationships  between  and 
among  ideas. 

Knowledge  integration:  The  process  of  adding,  distin¬ 
guishing,  organizing,  and  evaluating  accounts  of 
phenomena,  situations,  and  abstractions. 

Knowledge  management:  A  collection  of  technologies 
and  organizational  processes  aimed  at  capturing, 
disseminating  and  archiving  performance  related 
knowledge  within  an  organization. 

Knowledge,  skills,  and  attitudes  (KSAs):  Used  in  the 
traditional  approach  for  specifying  team  compe¬ 
tencies;  recent  research  now  expands  the  skills 
beyond  those  specifically  attributed  to  the  team 
task  to  include  psychomotor  and  cognitive  com¬ 
petencies. 

Learner  model:  A  representation  of  the  learner  that 
is  maintained  by  an  adaptive  system.  Learner 
models  can  be  used  to  provide  personalized 
instruction  to  a  particular  individual  and  may 
include  cognitive  and  noncognitive  aspects  of  the 
learner.  Learner  models  have  been  used  in  many 
areas,  such  as  adaptive  educational  and  training 
systems,  help  systems,  recommender  systems, 
and  others. 
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Learning  content  management  system  (LCMS):  A 
multi-user  software  application  in  which  learning 
developers  can  create,  store,  reuse,  manage,  and 
deliver  digital  learning  content  from  a  central 
object  repository  (database). 

Learning  environment:  A  system  that  incorporates  a 
set  of  features  including  a  navigation  system. 
Learning  environments  can  deliver  curriculum  in 
any  topic  area. 

Learning  hierarchy:  Description  of  successively 
achievable  intellectual  skills,  each  stated  as  a  per¬ 
formance  class  in  which  achievement  of  a  super¬ 
ordinate  skill  is  in  part  dependent  on  the  internal 
condition  of  having  learned  necessary  subordinate 
skills. 

Learning  management  system  (LMS):  A  computer- 
based  system  to  support  or  replace  classroom- 
based  learning;  examples  include  Blackboard, 
Moodle,  Sakai,  and  WebCT. 

Learning  object:  A  digital  resource  that  can  be  reused 
to  mediate  learning. 

Learning  places  (learnplaces):  The  immediate  physi¬ 
cal  setting  for  someone’s  learning  activity,  includ¬ 
ing  the  tools  and  artifacts,  digital  and  material,  that 
come  to  hand. 

Learning  style:  There  is  no  single  agreed-upon  defini¬ 
tion  of  learning  style;  a  general  definition  is  that 
it  is  a  description  of  the  attitudes  and  behaviors 
that  determine  an  individual’s  preferred  way  of 
learning.  Also,  a  learning  style  involves  a  theoret¬ 
ical  model  that  proposes  that  individuals  have  pre¬ 
ferred  ways  in  which  they  learn;  learning  styles 
typically  involve  visual  representation,  vocal 
explanations,  and  practical  examples;  experiential 
learning  proposes  several  ways  in  which  an  edu¬ 
cator  can  accommodate  these  learning  styles, 
although  all  such  styles  have  a  bias  toward  prac¬ 
tical  expression  and  place  less  emphasis  on  theo¬ 
retical  expressions  of  learning. 

Lethal  adaptation:  An  alteration  to  an  innovation  that 
undermines  the  expected  benefits  of  the  innova¬ 
tion. 

Library  media  center:  A  physical  repository  of  instruc¬ 
tional  materials  and  technology  resources  found 
in  most  American  public  schools ,  usually  managed 
by  a  library  media  specialist. 

Linear  hypertext:  A  hypertext  structure  in  which  links 
allow  the  reader  to  move  forward  and  backward 
through  the  content,  as  if  turning  the  pages  of  a 
book. 

Log  files:  A  sequence  of  behavioral  data  stored  in  a 
permanent  file,  offering  time-  or  location-stamped 
records  of  human  choices  made  through  computer 
interfaces  with  learning  environments. 


Long-term  memory:  A  component  of  the  information¬ 
processing  model  of  cognition  that  represents 
information  stored,  presumably  for  the  life  of  an 
individual.  Also,  the  store  holding  all  knowledge 
acquired  during  the  processes  of  learning. 

Loose  coupling:  An  organizational  arrangement  char¬ 
acterized  by  significant  freedom  between  levels  in 
the  hierarchy. 

Macro-adaptive  systems:  Allowing  different  alterna¬ 
tives  for  choosing  instructional  goals,  curriculum 
content,  and  delivery  systems  by  grouping  stu¬ 
dents. 

Mathemagenic  methods:  Instructional  methods  that 
explicitly  aim  at  the  transfer  of  learning;  these 
methods  promote  learners  to  invest  effort  and  time 
in  the  development  of  general  or  abstract  cognitive 
schemas. 

Meaning  making:  The  process  of  connecting  new 
information  with  prior  knowledge,  affected  by 
one’s  intention,  motivation,  and  strategies 
employed. 

Memory:  The  mental  faculty  of  retaining  and  recalling 
past  experiences. 

Mental  model:  A  mental  representation  that  people  use 
to  organize  their  experience  about  themselves ,  oth¬ 
ers,  the  environment,  and  the  things  with  which 
they  interact;  its  functional  role  is  to  provide  pre¬ 
dictive  and  explanatory  power  for  understanding 
these  phenomena. 

Micro-adaptive  systems:  Diagnosing  the  learner’s  spe¬ 
cific  learning  needs  during  instruction  and  provid¬ 
ing  instructional  prescriptions  for  the  needs. 

Minimalist  principles:  Instructional  design  principles 
for  sparse  instruction. 

Mixed-methods  research:  Studies  that  combine  quali¬ 
tative  and  quantitative  data  collection  methods. 
Also,  studies  that  rely  on  quantitative  and  qualita¬ 
tive  as  well  as  other  methods  for  formulating 
research  questions,  collecting  and  analyzing  data, 
and  interpreting  findings. 

Model:  An  artifact  representative  of  a  real  object  or  of 
an  internal  interpretation  of  something  real,  often 
represented  on  a  computer  screen.  Also,  structured 
representation  of  a  system  in  terms  of  variables  or 
concepts  and  their  (quantitative  or  qualitative) 
relations  that  can  be  used  for  predicting  system 
behavior  by  means  of  simulations. 

Model  research:  Research  that  addresses  the  validity 
or  effectiveness  of  an  existing  or  newly  constructed 
development  model,  process,  or  technique. 

Model  validation:  Experiments,  quasi-experiments, 
interviews,  and  expert  reviews  that  are  used  for 
verification  and  evaluation  of  a  particular  design 
and  development  technique. 
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Modeling:  In  cognitive  apprenticeship,  demonstrating 
thought  processes.  Also,  the  process  of  creating 
simulations  as  a  means  for  learning. 

Motivation  processes:  Interest  and  attribution  are  two 
essential  and  linked  components  of  motivation 
processes;  they  are  activated  by  arousal  and  inten¬ 
tion  through  the  descending  reticular  activation 
system. 

Multimedia  learning  principles:  Likely  effects  of  text, 
animation,  audio,  and  graphics  on  learning. 

Mutual  adaptation:  The  coevolution  of  both  an  inno¬ 
vation  and  the  environment  in  which  it  is  imple¬ 
mented. 

Natural  information  processing  system:  The  proce¬ 
dures  by  which  natural  systems  such  as  human 
cognition  and  evolution  by  natural  selection  pro¬ 
cess  information. 

Naturalistic  evaluation:  An  evaluation  approach  that 
relies  on  qualitative  methodology  but  gives  eval¬ 
uators  freedom  to  choose  the  precise  method  used 
to  collect,  analyze,  and  interpret  their  data. 

Novice  developer:  A  person  who  has  little  or  no  train¬ 
ing  in  instructional  design  yet  has  the  responsibil¬ 
ity  of  creating  learning  content  using  authoring 
tools. 

Objectivism:  An  ontological  and  epistemological  view 
that  contends  that  reality  exists  outside  of  the  indi¬ 
vidual  and  consists  of  specific  entities. 

Online /offline  measures:  Online  measures  are  recorded 
during  task  performance,  offline  measures  are 
recorded  after  task  performance. 

Ontology:  A  shared  and  agreed-upon  explicit  formal 
representation  of  a  domain.  Also,  what  exists  in 
the  world. 

Open  educational  resource:  A  learning  object  that  can 
be  freely  used,  reused,  adapted,  and  shared. 

Open-ended  learning  environment :  Instructional 
projects  based,  to  varying  degrees,  on  constructiv¬ 
ist  principles  in  which  learners  self-discover  con¬ 
cepts  and  principles  while  engaged  in  authentic 
learning  tasks. 

Open  source  software  (OSS):  Software  that  has  OSI 
open  source  licenses  attached;  these  licenses  state 
that  the  source  code  of  a  program  should  always 
be  available  to  everyone  and  that  everyone  can 
change  the  source  code. 

Open  standard  (OS):  Commonly  agreed-upon  and 
published  specifications  of  the  conventions  used 
in  a  community  to  ensure  the  quality  and  interop¬ 
erability  of  products  and  services. 

Outcome-referenced  models:  Approaches  to  instruc¬ 
tional  design  in  which  (1)  consideration  of  the 
nature  of  the  learning  tasks  and  the  conditions 
required  to  support  them  are  central,  and  (2)  learn¬ 


ing  outcomes  are  categorized  to  represent  not  only 
qualitative  differences  in  the  acquired  capability 
(as  a  category  of  task  or  goal)  but  also  external 
conditions  that  support  learning  and  different 
learner  states;  the  latter  are  referred  to  as  internal 
conditions,  which  also  facilitate  learning. 

Participation  models  of  authenticity :  Models  that 
establish  a  sense  of  authenticity  by  engaging  learn¬ 
ers  in  the  authentic  practices  as  they  work  on  real- 
world  tasks  as  part  of  authentic  communities  and 
in  contexts  that  value  the  outcomes  of  those  tasks. 

Participatory  design:  A  user-centered  design  approach 
in  which  users  are  actively  involved  in  the  design 
process  of  a  system  or  product  that  addresses  their 
specific  needs. 

Part-task  models:  Instructional  models  that  apply  an 
atomistic  approach  in  which  complex  contents  and 
tasks  are  reduced  into  increasingly  simpler  ele¬ 
ments  until  a  level  where  the  distinct  elements  can 
be  taught  to  the  learners. 

Pebble-in-the-pond  instructional  design:  A  content- 
first  approach  to  designing  instruction. 

Performance:  Ability  of  student  or  trainee  to  demon¬ 
strate  accomplishment  in  either  a  very  specific 
domain  such  as  a  formal  examination,  across 
domains  and  subject  matters,  or  realistic  contexts. 

Performance  improvement:  An  approach  to  optimizing 
organizational  outcomes  that  uses  a  systemic  com¬ 
prehensive  methodology  to  define  and  resolve 
gaps  at  the  organizational,  process,  and  individual 
worker  levels. 

Perturbances:  Events  that  cause  a  small  disruption  in 
organizational  function;  useful  for  encouraging 
reflection  on  an  organization’s  purposes. 

Phonological  loop:  In  Baddeley’s  memory  model,  a 
short-term  memory  component  devoted  to  retain¬ 
ing  auditory  information. 

Positive  interdependence:  When  individuals  perceive 
that  they  can  reach  their  goals  if  and  only  if  the 
other  individuals  with  whom  they  are  coopera¬ 
tively  linked  also  reach  their  goals. 

Pragmatism:  The  view  that  knowledge  is  derived  from 
interaction  among  groups  of  individuals  and  the 
artifacts  in  their  environment,  which  together  cre¬ 
ate  a  reality. 

Precedent:  Artifacts  and  the  memory  of  designing 
them  or  examining  them. 

Prescriptive  accounts:  Designed  tools,  strategies,  and 
interventions. 

Problem-based  learning:  Acquiring  knowledge  as 
part  of  a  learner  group  by  analyzing  a  problem, 
studying  privately,  using  various  learning 
resources,  and  collectively  synthesizing  knowl¬ 
edge.  Also,  an  instructional  method  that  initiates 
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students’  learning  by  creating  a  need  to  solve  an 
authentic  problem;  during  the  problem-solving 
process,  students  construct  content  knowledge  and 
develop  problem-solving  skills  as  well  as  self- 
directed  learning  skills  while  working  toward  the 
solution  to  the  problem. 

Problem-centered  instruction:  See  task-centered 
instruction  principle  and  strategy. 

Problem  solving:  A  process  of  understanding  the  dis¬ 
crepancy  between  current  and  goal  states  of  a 
problem,  generating  and  testing  hypotheses  for  the 
causes  of  the  problem,  devising  solutions  to  the 
problem,  and  executing  the  solution  to  satisfy  the 
goal  state  of  the  problem. 

Process-tracing  techniques:  Records  performance  pro¬ 
cess  data  such  as  verbal  reports,  eye  movements, 
and  actions  that  can  be  used  to  make  inferences 
about  the  cognitive  processes  or  knowledge  under¬ 
lying  task  performance. 

Programmed  technologies:  Process-based  methods 
and  approaches  to  support  learning  and  instruc¬ 
tion,  often  represented  in  the  form  of  algorithms 
and  implemented  in  computer  software. 

Punctuated  equilibrium:  A  view  of  organizational 
change  characterized  by  long  periods  of  stasis  and 
short  periods  of  change. 

Qualitative  research:  Research  that  emphasizes  verbal 
descriptions  or  narrative  accounts  of  observed  or 
recorded  events;  sometimes  called  naturalistic. 
Also,  research  on  human  systems;  hallmarks 
include  researcher  as  instrument,  natural  settings, 
and  little  manipulation.  Also,  often  conceived  of 
as  more  traditional,  or  positivistic,  typified  by 
experimental  or  correlational  studies;  data  and 
findings  are  usually  represented  through  numbers 
and  results  of  statistical  tests. 

Quasi-experiment:  An  experimental  design  in  which 
participants  are  not  randomly  assigned  to  treat¬ 
ments. 

Rapid  prototyping:  A  user-centered  design  approach 
in  which  users  participate  in  a  rapid,  iterative 
series  of  tryout  and  revision  cycles  during  the 
design  of  a  system  or  a  product  until  an  acceptable 
version  is  created. 

Rationalism:  An  epistemological  view  where  reason  is 
the  principle  source  of  knowledge. 

Realism:  A  form  of  objectivism  that  assumes  that  there 
is  some  sort  of  reality  that  is  separate  from  the 
mind  and  that  knowing  involves  a  correspondence 
between  the  word  and  the  mind. 

Reasoning:  Formulating  and  substantiating  conclu¬ 
sions  based  on  given  information. 

Reculturing:  Changes  that  involve  examining  the 
assumptions  and  purposes  of  an  organization. 


Redundancy :  The  presentation  of  information  multiple 
times  either  in  the  same  or  in  different  forms;  the 
value  of  redundancy  in  communication  is  context 
dependent  and  debated. 

Reflection:  In  cognitive  apprenticeship,  self-analysis 
and  self-assessment.  Also,  a  key  element  in  the 
model  of  experiential  learning  where  the  learner 
has  some  inner  mental  consideration  of  the  con¬ 
sequences  of  the  actions  they  have  taken  in  prac¬ 
tical  learning;  there  are  some  implied  mental  mod¬ 
els  that  are  necessary  for  reflection  to  be  an 
effective  element  within  learning  in  the  experien¬ 
tial  approach. 

Relational  hypertext:  A  hypertext  structure  with  links 
allowing  the  reader  to  access  information  on  other 
screens  that  have  some  logical,  conceptual,  or 
hierarchical  connection  to  the  content  on  the  cur¬ 
rent  screen. 

Resource:  Media,  people,  places,  or  ideas  that  have  the 
potential  to  support  learning. 

Resource-based  learning:  The  use  and  application  of 
available  assets  to  support  varied  learning  needs 
across  contexts. 

Restructuring:  Changes  that  involve  organizational 
structure,  new  work  patterns,  or  new  functions. 

Scaffolding:  Process  through  which  individuals  are 
supported  in  identifying,  interpreting,  or  otherwise 
using  resources.  Also,  support  that  is  provided  to 
help  learners  reach  skill  levels  beyond  their  current 
abilities;  essential  to  scaffolding  is  the  fading  of 
the  support  inversely  to  the  learners’  acquisition 
of  the  skill  that  is  being  supported. 

Scaled  instructional  strategies:  Hypothesis  that  the 
application  of  first  principles  has  an  accumulating 
performance  effect  for  complex  skills. 

Schemata:  Schemata  are  data  structures  for  represent¬ 
ing  both  generic  and  specific  knowledge. 

Science  learning:  Understanding  concepts  related  to 
biology,  physics,  chemistry,  and  astronomy. 

Self-paced  learning  (offline):  A  mode  of  learning  that 
enables  individuals  to  study  with  portable  technol¬ 
ogies  in  their  own  time,  at  their  own  pace,  and 
from  their  own  place. 

Self-paced  learning  (online):  A  mode  of  learning  that 
enables  individuals  to  study  online  and  in  their 
own  time  at  their  own  pace  and  from  their  own 
place. 

Self-regulated  learning:  When  learners  set  their  own 
goals  for  learning  and  then  attempt  to  plan,  mon¬ 
itor,  regulate,  and  control  their  cognition,  motiva¬ 
tion,  behavior,  and  context. 

Self-regulation:  Active  participation  in  terms  of  behav¬ 
ior,  motivation  and  metacognition  in  one’s  own 
learning  process. 
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Sensory  memory:  A  component  of  the  information-pro¬ 
cessing  model  of  cognition  that  describes  the  initial 
input  of  information  (such  as  vision  or  hearing). 

Serious  games:  Games  that  help  students  learn  new 
content,  strategies,  and  skills  that  are  relevant  to 
academic  and  practical  subject  matter. 

Shared  mental  models:  Crucial  in  the  work  of  a  team, 
as  members  of  the  team  need  to  share  an  accurate 
model  of  the  problem  to  be  addressed  and  the 
associated  task  to  assess  the  problem,  determine 
their  own  roles  and  responsibilities,  and  coordi¬ 
nate  their  efforts  over  time. 

Short-term  memory:  A  component  of  the  information¬ 
processing  model  of  cognition  that  describes  a 
person’s  attention. 

Simulation:  A  working  representation  of  reality;  used 
in  training  to  represent  devices  and  process  and 
may  be  low  or  high  in  terms  of  physical  or  func¬ 
tional  fidelity.  Also,  an  executable  (runnable) 
model;  computer  software  that  allows  a  learner  to 
manipulate  variables  and  processes  and  observe 
results.  Also,  a  computer-based  model  of  a  natural 
process  or  phenomenon  that  reacts  to  changes  in 
values  of  input  variables  by  displaying  the  result¬ 
ing  values  of  output  variables. 

Simulation  models  of  authenticity :  Models  that  build 
on  the  assumption  that  classroom  activity  should 
be  made  to  resemble  as  much  as  possible  the  activ¬ 
ities  in  which  real-world  practitioners  engage. 

Situated  learning:  A  perspective  on  learning  that 
emphasizes  its  social  and  physical  context. 

Situated  science:  A  perspective  in  which  the  scientific 
inquiry  and  attitude  are  carried  into  (instead  of 
applied  to)  the  field  of  the  project  and  of  practice, 
modifying  each  other  over  time  as  the  project 
unfolds;  see  analytical  science  and  design  science. 

Situatedness:  The  context  or  constellation  of  influen¬ 
tial  events  and  elements  that  govern  and  shape 
human  life. 

Situational  embodiment:  When  to-be-leamed  content 
is  experienced  in  relation  to  a  particular  context 
of  use  that  provides  legitimacy  to  the  content  and 
student  actions,  a  meaningful  goal,  and  a  set  of 
actions  for  the  learner,  and  on  which  learner 
actions  have  some  consequence. 

Situativity  theory:  A  form  of  realism  that  stresses  an 
individual’s  direct  perception  of  events  and  phe¬ 
nomena. 

Socio-cultural/historicism:  A  relativist  perspective  that 
emphasizes  relations  and  processes  between  the 
individual  and  society. 

Soft-skills  simulation:  Simulations  designed  for  the  spe¬ 
cific  purpose  of  teaching  interpersonal  skills  such 
as  leadership,  coaching,  facilitating,  and  so  on. 


Soft  technologies:  Algorithms,  programs,  or  environ¬ 
ments  that  broaden  the  types  of  interaction 
between  students  and  computers;  for  example,  an 
adaptive  algorithm  may  be  employed  in  a  program 
that  selects  an  assessment  task  that  provides  the 
most  information  about  a  particular  learner  at  a 
particular  point  in  time. 

Software  features:  Specific  applications  of  technology 
intended  to  advance  learning;  features  include 
designed  artifacts  such  as  modeling  tools,  simula¬ 
tions,  micro- worlds,  visualizations,  collaboration 
tools,  reflection  prompts,  games,  and  embedded 
assessments. 

Storyboard:  A  document  that  details  and  specifies  on¬ 
screen  text,  narrative  scripts,  and  interaction  in  a 
paper-based  format  before  it  is  converted  into  an 
online  course. 

Structure-guidance-coaching-reflection  cycle: 
Instructional  assistance  within  the  activation- 
demonstration-application-reflection  cycle  of 
instruction. 

Student  modeling:  Student  models  store  information 
about  students  including  domain  competence  and 
individual  domain-independent  characteristics; 
student  modeling  is  the  process  of  building  and 
updating  the  student  model. 

Subject-matter  expert  (SME):  A  person  with  extensive 
experience  who  is  able  to  rapidly  and  successfully 
perform  a  class  of  tasks. 

Synthetic  learning  environments:  A  learning  environ¬ 
ment  characterized  in  terms  of  a  particular  tech¬ 
nology,  subject  matter,  learner  characteristics ,  and 
pedagogical  principles;  a  synthetic  experience,  as 
opposed  to  a  real-world  interaction  with  an  actual 
device  or  process,  is  created  for  the  learner 
through  a  simulation,  game,  or  other  technology. 

System:  An  interdependent  entity  that  is  responsive, 
systematic,  and  bound  by  the  open  or  closed  nature 
of  the  context  in  which  it  is  located. 

Systemic  change:  The  process  of  changing  a  system 
from  one  paradigm  to  another  by  applying  systems 
thinking  and  systems  theory. 

Systems  design:  The  process  for  determining  what 
characteristics  a  new  system  should  have. 

Systems  theory:  An  interdisciplinary  field  with  appli¬ 
cations  in  both  the  hard  and  soft  sciences;  it 
focuses  on  understanding  relationships  among 
components  comprising  the  whole. 

Task-centered  instruction  principle:  The  central  prin¬ 
ciple  that  learning  is  promoted  when  learners  are 
engaged  in  a  task-centered  approach. 

Task-centered  instructional  strategy:  Teaching  compo¬ 
nent  skills  in  the  context  of  a  progression  of  real- 
world  whole  tasks. 
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Task  complexity:  Can  be  defined  subjectively  (individ¬ 
ual  characteristics,  such  as  expertise  or  percep¬ 
tion),  objectively  (task  characteristics,  such  as 
multiple  solution  paths  or  goals),  or  as  an  interac¬ 
tion  (individual  and  task  characteristics). 

Team  evolution  and  maturation  (TEAM)  model:  Offers 
stages  that  have  been  observed  in  team  perfor¬ 
mance  that  take  into  account  time,  experience,  and 
training. 

Technology:  The  application  of  scientific  or  other  orga¬ 
nized  knowledge  to  practical  tasks. 

Technology-enhanced  learning  environments :  Tech¬ 
nology-based  learning  and  instructional  systems 
through  which  students  acquire  skills  or  knowl¬ 
edge,  usually  with  the  help  of  teachers  or  facili¬ 
tators,  learning  support  tools,  and  technological 
resources. 

Technology-oriented  research:  Research  that  aims  to 
develop  new  technological  knowledge,  methods, 
and  artifacts  to  change  the  world  as  it  exists  with 
the  final  aim  to  improve  the  way  we  live. 

Theoretical  foundation:  A  related  set  of  rules  and  prin¬ 
ciples  that  can  be  brought  to  bear  as  a  basis  for 
making  predictions  and  providing  explanations  for 
a  variety  of  phenomena. 

Theoretical  statement:  Formal  expression  about  the 
relationship  between  at  least  two  variables  or 
instantiations  of  variables. 

Theory:  An  integrated  and  internally  coherent  set  of 
theoretical  statements  that  provides  a  sufficient 
basis  for  empirical  research  in  which  these  state¬ 
ments  can  be  tested. 

Theory  development:  Systematic  effort  to  generate 
(coherent  sets  of)  theoretical  statements. 

Theory-oriented  research:  Directed  at  the  development 
of  theories  about  the  world  as  it  exists,  with  the 
final  aim  to  predict  or  understand  events. 

Tool  research:  The  research-based  design  and  devel¬ 
opment  of  products  and  tools  to  enhance  learn¬ 
ing. 

Tools:  Devices  that  aid  individuals  to  engage  and 
manipulate  resources  and  ideas. 

Topic-centered  instructional  strategy:  Teaching  com¬ 
ponent  skills  in  sequence  prior  to  their  application 
to  a  whole  task. 

Transactional  leader:  Exerts  influence  by  setting 
goals,  clarifying  desired  outcomes,  providing 
feedback,  and  exchanging  rewards  and  recognition 
for  accomplishments;  one  of  two  leadership  styles 
that  hold  the  potential  for  more  effective  team 
performance,  moving  away  from  a  traditional  top- 
down  directive  approach;  see  transformational 
leader. 


Transformational  leaders:  Exerts  influence  by  broad¬ 
ening  and  elevating  followers’  goals  and  providing 
them  with  the  confidence  to  go  beyond  minimally 
acceptable  expectations  specified  in  the  exchange; 
one  of  two  leadership  styles  that  hold  the  potential 
for  more  effective  team  performance,  moving 
away  from  a  traditional  top-down  directive 
approach;  see  transactional  leader. 

Uncertainty:  Nonlinear  patterns  that  are  usually  the 
result  of  unpredictable  interactions. 

Unit  of  analysis:  Boundaries  of  phenomena  of  interest. 

Usability:  The  ease  with  which  humans  can  use  a 
system  or  a  product  to  accomplish  their  goals  effi¬ 
ciently,  effectively,  and  with  satisfaction. 

User-centered  design:  A  design  philosophy  and 
approach  that  places  users  at  the  center  of  the 
design  process  from  the  stages  of  planning  and 
designing  the  system  requirements  to  implement¬ 
ing  and  testing  the  product. 

User  control:  The  need  for  the  reader  to  determine  the 
sequence  and  pacing  of  information  accessed  in  a 
hypertext. 

Validation  (validity):  The  process  and  outcome  of  col¬ 
lecting  and  interpreting  results  of  assessments  or 
measurement  so  that  inferences  from  findings  are 
warranted  by  evidence. 

Visual  sketchpad:  In  Baddeley’s  memory  model,  a 
short-term  memory  component  devoted  to  visual 
and  spatial  information. 

Visuals:  A  form  of  media  in  which  information  is  pre¬ 
sented  visually;  text  may  or  may  not  be  considered 
a  visual  form. 

Web  log  file:  A  data  file  residing  on  a  Web  server  that 
contains  a  record  of  all  visitors  to  the  site  hosted 
by  the  server,  where  they  came  from,  what  links 
they  clicked  on,  and  other  information. 

Whole-task  models:  Instructional  models  that  apply  a 
holistic  approach  in  which  complex  contents  and 
tasks  are  analyzed  in  coherence  and  taught  from 
their  simplest,  yet  meaningful  version  towards 
increasingly  more  complex  versions. 

Working  memory:  The  structure  that  processes  infor¬ 
mation  coming  from  either  the  environment  or 
long-term  memory  and  that  transfers  learned 
information  for  storage  in  long-term  memory. 

Zone  of  learnability:  In  the  construction-integration 
model  of  text  processing,  a  hypothesized  optimal 
overlap  between  a  text  document  and  a  reader’s 
prior  knowledge. 

Zone  of  proximal  development  (ZPD):  A  term  coined 
by  Vygotsky  to  describe  the  space  between  a 
learner’s  current  skill  level  and  the  next  skill  level 
which  the  learner  cannot  reach  without  assistance. 


827 


Author  Index 


A 

Aamodt,  A.,  553, 562 
Aarts,H„  49, 52 
Abedi,J„  602 
Abell,  M„  321, 325 
Abelson,  R.  A.,  554,  565 
Abelson,  R.  P.,  28 
Abernathy,  W.  J.,  620,  630 
Abma,  H.  J.,  523 

Abrami,  P.  C.,  263, 266 ,  655, 656, 
657,  658,  759 
Abrams,  D.,  658 
Abreu,  J.  M.,  585, 590, 597 
Absalom,  D.  J.,  271, 276 
Acee,  T.,  759 
Achilles,  C.M.,  494,  500 
Achtenhagen,  F.,  444, 454 
Ackoff,  R.  L„  242, 247,  628,  630, 
693,694,695,700, 701 
Acredolo,M.  W.,  276 
Adamo- Viliam,  N.,  94,  95 
Adams,  D.,  409, 418 
Adams,  J.  A.,  142, 445, 454 
Adams,  M.,  655,  656 
Adamski,  A.  J.,749,  757 
Adams-Webber,  J.  R.,  591 
Adcock,  A.,  222 
Addae,  J.,  500 
Addison,  R.M.,  514, 523 
Adir,  N„  159 
Adler,  P.  A.,  780,  800 
Admiraal,  W„  553, 562 
Agarwal,  R.,  243, 247 
Agnew,  N.  M.,  591 
Agodini,  R.,  612,  616 
Agostinelli,S., 411,415,478 
Aguiar,  A.  C.,  494, 500 
Ahuja,  M.  K.,  41 1 , 418 
Ainsworth,  S.,  223, 434, 437 


Airasian,  P.  W„  140,  770,  800 
Akers,  C.,  51 

Akhras.F.N.,  474, 477, 479 
A1  Naeme,  F.  F.  A.,  308, 313 
Alavi,M„  274,  780,  800 
Albacete,  P.,  138,740 
Albanese ,  M.  A . ,  487 , 488 , 490 , 497 , 
500,  561 ,562 
Albert,  D.,  138,740 
Albion,  P.R.,  498, 500 
Albrecht,  F„  475, 479 
Al-Diban,  S.,  53,  804,  815 
Alesandrini.K.,  114,  119,724 
Ales  si,  S.  M.,  26, 95,  330,  335,  336, 
342, 409,  412, 423, 461, 466, 
707  ,  777 

Aleven,  V.,  131, 139,740, 141,143, 
474, 478,  479, 480, 483,  554, 
555,556,562,733,  757 
Alexander,  C.,  147,  759, 255 
Alexander,  J.,  309, 313 
Alexander,  P.  A.,  S7, 280, 292,  554, 
562 

Alfonso,  B„  649,  654,  656 
Alibali,  M.,  314 
Al-Kodmany,  K.,  663,  668 
Allen,  D.E.,  500, 502 
Allen,  I.E.,  274 
Allen,  J.,  689,  690 
Allen,  M.  W„  178,  119,183 
Allen,  R.,  407, 47S,  487,  500 
Allen,  S„  275 
Alleyne,T.,  490, 500 
Almeida,  L.C.,  624,  630 
Almond,  R.  G.,  293,  604 
Alonso,  G.,  358, 363 
Altalib,  H„  521, 523 
Alupoaie,  S.,  364 
Alvarez,  F.  J.,  363 
Amandi,  A.,  314 


Ambrose,  D.,  56,  64 
Ambrosio,  J.,  244, 247 
American  Association  for  the 
Advancement  of  Science 
(AAAS),  458, 466 
American  Association  of  School 
Librarians  (AASL),  233,  234, 
239 ,239 

American  Educational  Research 
Association  (AERA),  596,  598, 
599, 600, 602 

American  Evaluation  Association 
(AEA),  775,  800 

American  Psychological  Association 
(APA),  15, 18,  602 
Amigues,  R.,  41 1, 415, 47S 
Amirault,  R.  J.,  767 
Amorim,  R.  R.,  362, 363 
Amundsen ,  C .  L . ,  261, 266 
An,  Y.-J.,  274,  669 
Anand,F„  473, 483 
Anand,  P.  G.,  722,  757 
Ancona,  D.  G.,  782,  800 
Anders,  E„  139,742 
Anderson,  A.,  409, 418, 628,  631 
Anderson,  C.,  6, 18 
Anderson,  D.,  209 
Anderson,  J.  A.,  37,  37 
Anderson,  J.R.,  26,  95,  127,  128, 
139,740, 747, 165,770,213, 
216,220,222,284,292,293, 
387,389,395,445,456,432, 
583,584,597,593,637,643, 
644,708, 777,784,785,300 
Anderson ,  L .  W . ,  1 30 , 1 36 , 740 , 262 , 
266 

Anderson ,  R .  B .,  43 , 52 
Anderson,  R.C.,  44,  57,  114,  118, 
119,723,  133,  137,  139,  740, 
142,143,  190,794,545,562 


829 


Author  Index 


Anderson,  R.  E.,  16, 75 
Anderson,  R.  M.,  501 
Anderson,  T.  M„  24, 28, 222,  264, 
266, 275,  358,  363, 422, 443, 
456,  804,  812,  815 
Anderson,  W.  L.,  474.  504 
Anderson-lnman,  L.,  202, 206, 208, 
209 

Andino-Rifon,  L.,  644 
Andre,  E.,  222 

Andre,  T„  126, 140,  340, 344,  708, 
711 

Andrews,  D.  H„  322, 325,  454, 446, 
685 ,690,  707,777,741,757 
Andrews,  H.  F.,  565 
Andrews,  K.,  292 
Andriessen,  J.,  421 ,411 , 480 
Angel,  S.,  255 

Angeli,  C„  437, 489, 496,  500,  802 
Angell,  D.,  190,794 
Anglim,  J.,  802 

Anglin,  G.  J.,  26,  721 , 723,  757,  760 

Angulo,  C .,292 

Annett,  J.,  127, 140, 445, 454 

Anson,  R.,  410, 427 

Antepohl.W.,  490, 500 

Anthony,  J.  L.,  543, 562 

Anthony,  K.,  688,  690 

Antikiewicz,  ML,  644 

Anzai,  Y.,  46,  57 

Apostolopoulos,  P.,  466 

Appelman,  R.,  665,  668,  686,  690 

Arbaugh,  J.  B.,  275 

Archer,  W.,  222, 275, 422,  804 

Arends,  R.  I.,  487,  500 

Arfield,  J.,  257 

ARIADNE,  349,  352 

Arid,  A.,  108 

Arisman,  M.,  687,  690 

Armatas ,  C . ,  5 3 1 ,534 

Armour-Thomas,  E.,  436 ,437 

Armsby,  P.,  326 

Arnone,  M.,  678,  684 

Aronson,  J.  E.,  344 

Aroyo,  L.,  363,  481 

Arriaza,  G.,  622,  623,  624,  630 

Arroyo,  I.,  288,  290 ,292 

Arter.J.  A.,  784,  800 

Artzt,  A.  F„  436,  437 

Arvaja,  M.,  438 

Aschbacher,  P.  R.,  602 

Asche,  S.  E.,  230 

Asensio-Perez,  J.  I.,  360, 363, 364 

Ash,  D.,  108 


Ash,  M.G.,  445, 454 
Ash,  M.  J.,  81 
Ashcraft,  M.H.,  49, 57 
Ashley,  K.  D„  554,  555,  556, 562 
Ashman,  H.,  313, 482 
Aspray,W„  242, 248 
Aspy,C.B„  487, 500 
Aspy.D.N.,  487,  500 
Association  for  Contextual 
Behavioral  Science,  64 
Association  for  Educational 
Communications  and 
Technology  (AECT),  31 ,32, 
233,234 ,239,  689 ,690 
Astleitner,  H.,  48, 57 
Atkinson,  C.  R.,  473, 480 
Atkinson,  R.  C.,  41 , 57 
Atkinson,  R.  K„  217, 220,  377,  381 
Atkinson,  R.  L.,  87,  95 
Atlas,  R.S.,  592 
Attali,  Y.,599 ,602 
Atwood,  M.E  .,222,380 
Aubin,  C.,  684 
Auburn,  T.,  252, 255 
August,  G.  J.,  114,724 
Austin,  J.,  63,  64 

Ausubel,  D.  P„  26,  77,  78,  80,  213, 
220,  706,  777 
Avery,  S.,  527, 534 
Avgeriou,  P.,  476, 480 
Aviles,  K„  272, 274 
Avissar,  G.,  622,  625,  627,  630 
Aviv,  R„  780,  800 
Avolio,  B.  J.,  658 
Avon,  J.,  427 
Avouris,  N.,  335, 342 
Awad,  E.  M„  242 ,247 
Axelrod,  R„  334, 342 
Ayres,  P.,  375, 380,  758,  785,  800, 
815 

Azambuja  Silveira,  R.,  28 1 , 292 
Azevedo,  R.,  121,723,  130,740, 
201 , 202,  204,  207,  208, 209, 
215,220,336,343,478,450 

B 

Baarda,  M.,  565 
Baars,  B.  J.,  52 
Back,  H.  P.,445, 455 
Badash,  L.,  650,  656 
Baddeley ,  A.  D.,  41 , 57 , 88,  94,  96, 
376,  380 
Badler,  N.,  222 


Baglio,  C.,  65 
Bahar,  M.,  308, 373 
Bailey,  B„  8,  521, 523 
Bailey,  J.  H„  503 
Bailey,  K.  V.,  75 
Baird,  D.,  687,  690 
Bajraktarevic,  N.,  307, 373 
Baker,  C„  403, 475 
Baker.E.L.,  321, 325, 597, 598, 599, 
600,  601, 602,  602,  603,  604, 
609, 676 

Baker,  L.,  214, 220 
Baker,  M.  J„  414, 475 
Baker,  R„  743, 453 
Baker-Ward,  L„  565 
Bakker,  A.  B.,  53 
Baldwin,  C.Y.,  635,  643 
Bales,  R.F.,  780, 500 
Ball,  K„  327 

Banathy,  B.  H„  26,  30, 32, 620, 621 , 
630,  692,  693,  694,  695,  696, 
707 

Bandura,  A.,  50,  57,  134,  740,  264, 
266,406,419,656,  652 
Bangert-Drowns,  R.  L.,  127,  130, 
131, 740,  795 
Bankes,  S.,  343 
Bannan-Ritland ,  B .,  35 1 , 352 
Bannert,  M.,  707  ,  777 
Bannon,  L.,  660,  665 
Barab,  S.  A„  69,  70,  72,  75,  79,  SO, 
57,52,98,99, 100,  106,705, 
119,  121,723, 146,  158,759, 
254,255,746,757 
Barabasi,  A.  L.,  630 
Barak,  M„  153,  759, 487, 500,  651, 
656 

Barbas,  A.,  507 

Barbosa,  S.,  457 

Barclay,  M.  W„  175,  753, 353 

Bareiss,  R.,  454 

Baresi,  L.,  635,  643 

Bargh,  J.  A.,  583,  585,  590, 597 

Barker,  L.M.,  77,  57 

Barker,  T.  A.,  566 

Barley,  S.,  411, 422 

Barnard,  A.,  487,  533, 500, 534 

Barnard,  J.,  677 

Barnard,  Y.  F.,  505 

Barndollar,  K.,  597 

Barnes,  R.,  252, 255 

Barnes,  S.,  262, 266 

Barnes-Holmes,  D.,  62,  64,  65 

Barnett,  J.E.,  114,  115,  116,723 


830 


Author  Index 


Barnett,  M.,  327 
Barnett,  M.  G.,  108, 159 
Barowy,  W.,  341 , 342 
Barrett,  C.,  535 

Barron,  B„  98,  100, 108,  165,  166, 
169, 770,529,  554,650,  656 
Barron,  L.,  656 
Barrowcliff,  D.,  254,  255, 256 
Barrows,  H.S.,  101,708,486,487, 
488,492,493,496,498,500, 
507, 562, 540, 561, 641, 643 
Barry,  D„  652,  653,  656 
Barry,  W.E.,  502 
Barson,  J.,  13, 18 
Bartlett,  F„  372,  373, 380 
Barton,  D„  227,  640,  643 
Barton,  K.,  397 
Barton,  M.  D.,  230 
Barwise,  D.,  57 
Basnet,  K.,  690 

Bass,  B.M.,  648,  651, 652, 656 
Bass,  K.M.,  160 
Bass,  L.,  635,  643 
Bassok,  M.,  480 

Bastiaens,  T.  J.,  522, 523, 563, 802 
Bates,  A.,  17, 18,  386,  395,  609,  616 
Bateson,  G,  168,  770 
Battista,  M„  410, 413,479, 421 
Battistich,  V.,  779,  800 
Bauer,  J.,  96 
Bauer,  M.,  294 

Baumgartner,  E.,  152,759, 167,770, 
433 ,439 

Baumgartner,  L..  438 
Bavelier,  D.,  320, 326 
Baveta,  E.  G.,  326 
Baxter,  J.,  420 

Baylor,  A.  L.,  215, 217,  218, 221, 
223,243,247, 324,525,417, 
419,  722,  759,  809,  815 
Bazeley,  P.,  788,  800 
Bazerman,  C.,  535,  640,  641 , 643 
Beadouin-Lafon,  M.,  662, 668 
Beal,  C.  R.,  218, 222, 292, 293 
Bean,  T.  W„  432, 437 
Beane,  W„  412, 479 
Beaufort,  A.,  435, 437 
Beaumont,  I.,  475, 480 
Becerra-Fernandez,  I.,  242, 247 
Beck,  A.,  431, 437 
Beck,  I.  L„  544,  545, 562 
Beck, J„ 243, 248 

Becker,  H„  164,  168,  170,770,410, 
418,479 


Becker,  W.C.,  63, 64 
Beckwith,  D.,  273, 274 
Beckwith,  R.,  454 
BECTA,  358, 363 
Bednar,  A.  K.,  15,78,71,74,87, 
530,534 
Bednarz,  N.,  65 
Beebe,  T.J.,  229, 230 
Beers,  P.  J.,  777 
Beheshti,  J.,  240 

Beissner,  K.  L.,  1 19,  121,723,452, 
454 

Bejar,  1. 1.,  599,  601, 602 

Beke,N  .,256 

Belch,  G„  631 

Beldarrain,  Y.,  264,  266 

Bell,  B„  452,  454, 456,  674,  684 

Bell,  H.H.,  322, 325 

Bell,  J„  760 

Bell,  M„  228, 230 

Bell,P„  146, 149, 158,759,760, 165, 

770.777.413.479.434.437, 
466, 503 

Bell,  T„  309, 373 
Bell,  W„  228, 230 
Belland,  J.  C.,  27 
Bellisimo,  Y„  503, 504 
Bellman,  B.L.,  775, 800 
Belovics,  R.,  680,  684 
Benavidez,  D.  A.,  65 
Bench-Capon,  T.  J.  M.,  554, 562, 565 
Bender,  T„  138,742 
Benford,S„  255, 256 
Beni,  G„  94,  95 
Bennett,  C.,  500 

Bennett,  R.  E„  599,  600,  601, 602 
Benton,  S.L.,  389, 397 
Bera,S„  340, 342, 735,  759 
Berdel,  R.  L.,  759 

Bereiter,  C.,  71, 72, 74, 87, 104, 105, 
708,770,  146,  154,  760,  166, 
777,214,219,227,224,237, 

229.387.395.422.434.437, 
439,  546,  563 

Bereiter,  K.,  422 
Berg,  R.,  344 
Bergan,  J.  R.,  388,  395 
Berger,  C„  409, 479, 420 
Berger,  R.  C.,  592 
Bergman,  R.,  742,  760 
Bergquist,  M.,  357, 363 
Berkson,  L„  490, 507, 561 , 563 
Berlanga,  A.,  358, 363 
Berliner,  D.  C„  472, 480,  729,  757 


Berio,  D.K.,  11,78 
Berlyne,  D.  E.,  42, 49, 57 
Berman,  P.,  624,  625, 629, 630 
Berman,  T.  R„  327, 439, 456 
Bernard,  C„  192,  794 
Bernard,  E.  M.,  502 
Bernard,  R.M.,  115,  116,724,  130, 
740,261,262,263,264,266, 
267,  759 

Bernath,  U.,  261, 266 
Berndt,  T.,  412, 479 
Berners-Lee,  T„  244, 248,  358, 363 
Bernhardt,  P.,  690 
Berry,  F.  M.,  56,  64 
Bers,M„  334,  342 
Bertalanffy,  L.  V.,  694,  707 
Berzins,  J.  I.,  56,  64 
Beswick.N.,  526,  534 
Betrancourt,  M.,  54,  96, 224 
Betsch,  T.,  584, 597 
Beukhof.G.,  391, 395 
Bewley,  W.  L.,  603 
Beyer,  H„  662,  668 
Beynon-Davies,  P.,  257 
Bhola,  D.  S.,  604 
Biancarosa,  G.,  280, 294 
Bianco,  M.,  275 
Bias,  R.G.,  662,  668 
Bichard,  S.L.,  527, 534 
Bichelmeyer,  B.  A.,  17, 19,  86, 96, 
664,665,666 ,668,670,  685, 
686, 690 

Bickman,  L.,  607,  676 
Bickmore,  T.,  292 
Biddle,  W.B.,  119,723 
Bielaczyc,  K.,  759,  323,325 
Biemiller,  A.,  544,  563 
Biggs,  J.,  445, 454 
Biglan,  A.,  56,  58,  59,  62,  63, 64 
Biklen,S.K„  771, 775,  800 
Bilal,  D„  235, 239 
Billet,  S.,  430, 437 
Billsus,  D.,  294 
Bilodeau,  E.  A„  126,  127,740 
Binder,  C„  63, 64,  65,  193,794 
Biocca,  F.,  41 1,479 
Birenbaum,  M.,  130,  740,  598,  601, 
602,767,783,800 
Birt,  A.  T.,  795 
Bishop,  M.J.,  93,  94,  95,  96 
Bisoux,  T.,  272, 274 
Biswas,  G.,  285, 292, 293 
Bjorck,  U.,  498,  507 
Bjork,  R.,  319, 327,  767,  784,  800 


831 


Author  Index 


Bjorn,  M„  343 
Blackmore,  D.,  565 
Blake,  R.L.,  488, 490, 495, 501, 505, 
561 ,565 

Blake,  S.B.,  386, 395 
Blakeslee,  S.,  387, 396 
Blasco,  M.  T.,  363 
Blasi,L„  649,  654,  656 
Blat,J„  364 
Blaustein,  J.  J.,  195 
Blessing,  S.B.,  223,  584,  591 
Blick,  C„  701 

Bliss,  J.,  252, 256,  331,343 
Blobitt,  W.  E.,  141 
Block,  J.  H„  262, 266, 471 , 480 
Blommaert,  J.  M.,  502 
Bloom,  B.S.,  26, 126, 127,214,227, 
140, 266, 262,473 , 480, 636, 643 
Blue,  A.  V„  490, 507, 505 
Blumberg,  P.,  492, 501 
Blumenfeld,  P.  C„  103, 108, 109, 
159,160,631,654,656,657 
Blunder  J.  G.,  50, 57 
Blumschein,  P.,  53,  804 
Bobrow,  D.  G.,  87,  96 
Bodemer,  D.  R„  95, 96, 339, 340, 342 
Boden.D.,  252, 256 
Bodilly,S.  J„  623,  630 
Bodker,  S.,  662,  668 
Boecker,  D.,  482 

Boekaerts,  M.,  121,725,  127,  128, 
133, 140 
Boerger,  A.,  209 
Bogaart,  N.  C.  R.,  775,  800 
Bogdan,  R.  C.,  771, 775,  800 
Bohlin,  R„  322 ,326 
Bolchini,  D.,  635,  643 
Boling,  E„  662,  665,  666,  668, 670, 
690 

Bolzan,  N.,  487, 507 
Bond,  N.,  229, 237 
Bonk,  C.  J.,  407, 415, 419, 429, 431 , 
433 , 434, 437, 439,610,616,802 
Bonn,  K„  112,723 
Bonnett,  C.,  431, 437 
Booch,  G„  360,  363 
Boonthum,  C.,  223 
Boot,  E.  W„  637,  643 
Boot,  R„  252, 256 
Booth,  C„  669 
Booth,  K„  420 
Booth  Sweeney,  L.,  463, 466 
Boozer,  R.  F.,  399 
Boren,  M.  T„  787,  800 


Borg,  W.  R„  726, 757, 768,769,773, 

115.800 

Borko,  H„  622,  625,  626,  630 
Borman,  G.  D.,  720,  757 
Borner,  K„  737,  757 
Bortolussi,  D.,  669 
Borzak,  L.,  35,  36, 37 
Bos,  N„  760 

Boshuizen,  H.  P.  A.,  423, 504,  554, 
563,564,565 
Bottorff,  J.,  803 

Botturi,  L„  394, 395,  638,  639,  643, 
686,  690 

Boud.D.,  37, 487, 507 
Bouhuijs,  P.  A.  J.,  504 
Bouillion,  L.,  109 
Boulaoude,  S.,  118,723 
Boulter,  C.  J.,  46,  57 
Bourdeau,  J.,  386, 395 
Bousiou,  D.,  53 
Boutwell,  R.  C.,  473, 482 
Bouwense,  E.,  565 
Bouzeghoub,  A.,  352 
Bovair,  S.,  46, 52 
Bowdish,  B.  E.,  498, 507 
Bower,  G.H.,  308, 373 
Bowers,  C.  A.,  3 19,  320,  321 , 325, 

326. 327. 649. 656. 657. 180.800 
Bowers,  J.,  565 

Boyatzis,  R.  E.,  326 
Boyd,  D.,  245 ,248 
Boyle,  C.F .,292,643 
Boyle,  T.,  271, 274 
Brache,  A.  P.,  28,  512, 524 
Brackbill,  Y„  137,747 
Braden,  R.  A.,  91, 96 
Bradley,  C.,274 
Bradshaw,  J.  M.,  581, 597 
Bradsher,  M.,  105, 108 
Brady,  A.,  353 

Brailsford,  T.  J„  373, 480, 482 
Brameld,  T.,  385,  395 
Brancati,  F.  L.,  566 
Branch,  Jr„  W.  T„  559,  561 , 563 
Branch,  R.M.,  25, 27,  31, 32,  194, 
195,665,669,101,712,140, 
741.742,743,758 
Brand,  S„  635, 642,  644 
Bransford,  J.  D.,  26, 44, 51,53 , 98, 
103,  70S,  770,  146,  150,  154, 
159,160, 165,770,754,212, 
214,227,292,293,319,325, 
534,583,593,650,656,657 
Bransford,  J.  P.,  1 14,  724 


Bransford,  T.  D.,  105,  70S 
Branson,  R.  K.,  13,  79 
Brashears,  T.,  43, 57 
Braswell,  R.,  265, 266 
Braun,  C.  C„  800 
Bravo,  C„  464, 465, 466 
Brayton,  R.,  639,  644 
Bredeweg,  B.,  282, 292 
Bredo,E„  57,  64,14,81 
Breedin,S.D„  583, 592 
Breman,  J.,  668 

Brennan,  S.  E„  219, 227, 226, 230 
Brenton,  H.,  263, 266 
Brescia,  W„  229, 230 
Brethower,  D.  M.,  26 
BretzkeH.,  281, 292 
Breuleux,  A.,  240 
Brewer,  E.  K„  635,  638,  642,  644 
Brewer,  W.F.,  131, 747, 550,  565, 
566,583,597 
Breyer,  F.  J.,  604 
Bridgeman,  B.,  800 
Bridges,  E.M.,  487, 498,  507 
Briggs,  G.E.,  443, 454, 455 
Briggs,  L.  J„  27,  81,  191,794,389, 
395, 396, 446, 455, 472,  481, 
592,  649,  657 
Briggs,  P.,  339,  342 
Briggs-Myers,  I.,  309,  310, 373 
Brill,  D.G.,  603 
Brindley,  J.E.,  261, 266 
Brinkerhoff,  J.  D.,  203, 208 
Brinkmann,  J.,  597 
British  Information  Services,  10,19 
Britt,  ML,  507 

Britton,  B.K.,  392, 395,  772 
Britton,  J.,  404, 479 
Broadbent,  D.  E.,  87,  96 
Brocklehurst,  N.,  805 
Broderick,  W.  A.,  191,797 
Brody,  H„  218, 227 
Brody,  P„  723 
Bromme,  R.,  556, 563 
Bron,  A.,  624,  630 
Bronander,  K.  A.,  244, 248 
Brookfield,  S.  D.,  35, 36, 37, 262,266 
Brooks,  C„  288,  289, 293, 294 
Brooks,  D.,  136, 141 ,302,669,101 , 
711 

Brooks,  L.,  438 
Brooks,  R.  A.,  285, 292 
Brooks,  T„  526, 534 
Brooks-Gunn,  J.,  564 
Brophy,  S„  770,  760,  770, 184 


832 


Author  Index 


Brosnan,  M.,  321, 325 
Brouns,  F.  M.  R.,  365 
Brown,  A.  L.,  26, 44, 57, 99, 101, 
102, 104 ,108,109, 114,725, 
146, 147,759, 167,770,227, 
228,250, 323,525,427,432, 
457, 458,  530,  554,  734,  757 
Brown,  B.R.,  522, 524 
Brown,  E„  307,  575, 475, 476,  480 
Brown,  F.  J„  134,747 
Brown,  G„  271, 274 
Brown,  J.  S„  81, 98,  99,  709,  759, 
796, 427, 457, 454, 454, 459, 
466,531,554,644 
Brown,  M.,  565 
Brown,  S.J.,  324,  525,  777 
Brown,  S.  L„  620,  628,  629,  650 
Brown,  S.  W„  202, 209, 575,  757 
Brownstein,  A.  J.,  62,  65 
Bruchmuller,  K.,  96 
Brudner,  H.  J.,487 
Bmer.J.T.,  146,  759 
Bruner,  J.  S„  79,  24. 27,  73,  78,  81, 
105,709, 165,770,777,340, 
342,439 

Burning,  R.,  132,  742 
Briininghaus,  S.,  554,  556, 562 
Briinken,  R.,  785,  800 
Brash,  T.,  335, 342, 415, 479, 520, 523 
Brusilovsky,  P.,  220, 227,  281, 286, 
292,358,565,475,476,477, 
480,481,483 
Bryant,  B.K.,  228, 250 
Bryant,  P.,  549, 564 
Brynmor-Jones  Committee,  19 
Bucciarelli,  L.  L.,  640,  642,  644 
Buccino,  G.,  381 
Buch.E.E.,  755,  757 
Buchannan,  B.,  641, 644 
Buck,  P.,  564 
Buckendahl,  C.  W .,604 
Buckley  ,B.C„ 46, 57, 462, 466 
Buckman,  L.,  427 
Buckmaster,  A.,  438 
Buehl,  M.  M.,  592 
Buhanan,  C.  W„  184 
Buhanist,  P.,  658 
Bull,  B.  L.,  114, 116,725 
Bull.S.,  295 
Bullot,B„  280, 294 
Bulte,  J.  A.,  505 
Bumstead,  J.  F.,  6,  79 
Bundy,  C.,  805 
Bunzo,  M.,  225, 482 


Burbules,N.C„  60,  64 
Burdett,  K.,  805 
Burgess,  S.  R.,  566 
Burgett,  J.  E.,  505 
Burke,  R.,  247, 248 
Burke,  W.  W„  620, 650 
Burkhardt,H„  746,  757 
Burleson,  W„  219, 227 
Burns,  B.  R.,  472,  480 
Burns,  H.  L„  192,794 
Burstein,  J.  C.,  216, 227, 599,  600, 
601,604 

Burton,  A.  M.,  592 
Burton,  J.  K„  12,  79,  62,  64,  73, 87, 
115,  116,725,796, 392,595, 
530, 534 

Burton,  R.  R.,  466 
Butler,  D.L.,  127,  128,  131,  133, 
134,747 

Buttrey,  T.,  731, 735,  767 
Buxton,  W„  282, 292 
Byrne,  M.D.,  307, 575 
Bystrom,  K.,  783,  801 

c 

Cadiz,  J.  J.,  <507 
Cadwell.D.,  487, 505 
Caeiro-Rodriguez,  M.,  639,  644 
Caffarella,  E.  P.,621, 630 
Cagiltay,  K.,  676 
Cahen,  L.  S.,  472, 480 
Caine,  G.,  624, 650,  696,707 
Caine,  R.  N„  696,  707 
Calamaio,  M.  E.,  389,  596 
Calandra,B„  203, 209 
Calder,  J.,  677 
Calderon,  P.,  504 
Calderwood,  R.,  584, 595 
Caldwell,  D.F.,  782,  800 
Callison,  D„  234,  235, 238, 239 
Calvi.L,  416,480,481 
Calvo.R.,  228, 250 
Cambre,  M.  A.,  8,  79,  755,  757 
Cambron-McCabe,  N.,  707 
Cameron ,  B .,  2 1 8 , 227 ,  32 1 , 525, 
550, 565 

Cameron,  T.,  498,  507 
Cammack,  D.,  759 
Camp,  P.  J.,  777, 505 
Campbell,  C.B.,  504 
Campbell,  D.  J.,  807 
Campbell,  D.  T„  613, 676,  758, 783, 
805 


Campbell,  F.  A.,  543, 564 
Campbell,  V.  N.,  794 
Campbell-Kelly,  ML,  242, 248 
Campion,  L.  E.,  79 
Campione,  J.  C.,  104, 108,323,325, 
438,530,534,624,630 
Campo,  M.,  314 
Camps,  J.,  787,  807 
Campsall,  J.  M.,  805 
Canas,  J.  J.,  209 
Cancelli,  A.  A.,  395 
Canfield,  A.  M„  393, 395 
Cannon-Bowers,  J.  A.,  3 19, 320, 323, 
525,526, 527, 589,648,653, 

656,  776,  781, 801,803,  804 
Cantlon.D.  E,  432,458 
Capaldi.E.  J.,  60,  64 

Caplow,  J.  A.  H.,  492, 493, 494, 507 , 
502 

Cappa,  S.,  381 
Capps,  C.G.,  192,794 
Capra,  F.,  630 
Carbonell.J.R.,  641, 644 
Card,  S„  215, 227 
Cardinali,  F.,  482 

Carey,  J.  O.,  141,159, 195,  657,  711 
Carey,  L„  747,  759,  795, 472, 487, 

657,  777,742,753,758 
Carey,  R.,  28 
Carley,K.M„  41 1,478 
Carlson,  H.,  409, 479 
Carmean,  C.,  577 

Carmi,  M.,  759 
Carmona,  C.,  294 
Carnevale,  A.,  776,  807 
Carney,  K.  E„  336,  337,  338,  340, 
342,344 

Carnine.D.  W„  63, 64, 65, 115,118, 
725,409,422 
Carnoy,M.,  11, 79 
Carr,  A.  A.,  650,  657, 

Carr,  B„  294 

Carr,  C.S.,  554,  555, 565,  707,  759, 
802 

Carr,  J.E.,  63,  64 
Carr,  W.  J.,  794 

Carr-Chellman,  A.  A.,  174, 784, 620, 

624. 650. 661 . 662. 663.667 .668 
Carrier,  C.  A.,  409, 410, 413,419, 

472,476,480,732,735,758 
Carroll,  J.B.,  127,  747, 262, 266 
Carroll,  J.  M„  478, 480,  662,  663, 

664. 668 
Carson,  H.,  478 


833 


Author  Index 


Carteaux,  B .,108 
Carter,  B.  J.,  669 
Carter,  J„  115,  116,724 
Carter,  K.,  553, 563 
Cartier,  J.L.,  322 ,325,327 
Cartwright,  G.  P.,  761 
Carugati,  F.,  564 
Caruso,  J.  L.,  388, 395 
Carver,  C.  A.,  312, 313 
Casati,  F.,  363 
Cascallar,  A.,  784,  801 
Cascallar,  E.,  784,  801 , 805 
Case,  S.M.,  505 
Casey,  C.,  429,  437 
Casey,  J.,  760 
Casillas,  A.,  601, 602,  604 
Caspi,  A.,  95,  96 
Cassell,  J.,  222,  286,  292 
Castelli,  C.,  202, 208 
Cate,  C.,  96, 222 
Cates,  W.M.,  93,  94,  95,  96 
Catrambone,  R.,  95,  96,  802 
Cavalier,  F.  J.,  722,  758 
Cavalier,  J.  C.,  722,  758 
Cavanaugh,  M.  A.,  50, 52 
Cawley,  P.,487, 507 
Centre  for  the  Study  of  Learning  and 
Performance  (CSLP),  655,  657 
Cerna,  N.,  657 
Chadha,  R.,  183 
Chalk,  P.,274 
Chalmers,  A.  F.,  553, 563 
Chambers,  B.,  658 
Chambers,  J.  C .,658 
Champy,  J.,  699,  700,  701 
Chan,  A.  L„  742 
Chan,  C.K.K.,  225 
Chan,  L„  593 

Chandler,  P„  138,  747, 214. 222, 
224, 374,  376,  380, 381,455, 
723, 758,7X3,801,815 
Chao,  C.,  456 
Chao,  C.  I. ,  390 ,395 
Chao,  C.-J.,  580,  584,  585,590, 597 
Chapman,  B„  673,  678,  681, 684 
Chapman,  P.,  805 
Chapman,  S.,  223 
Charness,  N.,  786,  801 
Chase,  C.  A.,  756,  758 
Chase,  C.  I.,  769,  801 
Chase,  D.H.,  134,743 
Chase,  N„  521, 522, 523 
Chase,  W.G.,  371, 380 
Chauvin,  S.  W.,  507 


Cheak,  A.  M„  603 
Checkland, P.,694,  701 
Chen,  H.,  771 , 801 
Chen,  M.,  280, 292 
Chen,  S.  Y.,  307, 374 
Cheng,  P.  C.-H.,  245, 248 
Chen-Troester,  J.,  397 
Chepegin,  V.,  363, 481 
Cherkes-Julkowski,  M.,  81, 108 
Chernick,  R.,  412, 479 
Chernobilsky,  E.,  760 
Chester,  E.,  300,  302 
Cheung,  A.,  658 
Cheung,  W„  40,  50, 52 
Chi,  M.T.H.,  96,  131,747,214, 
227,319,325,371,375,380, 
458,466,478,480, 553,554, 
563,580,583,592,600,603 
Chiang,  I.  R„  652,  657 
Chickering ,  A.,  609,  611, 676 
Childress,  M.D.,  265, 266 
Chin,  G.,  668 

Chinn,  C.  A.,  131, 747, 583, 597 
Chipman,  P.,  222 

Chipman,  S.  F„  578,  579,  580,  581 , 
591,593 

Chiseri,  M.  H„  795 
Chissom,B.S„  771, 803 
Chiu,  J.L.,  760, 467 
Chiu,  M„  227 

Chodorow,  M„  601 , 602,  604 

Choi,  I„  434, 437 

Chou,  C„  756,  758 

Chow,  S.M.K.,  496, 507 

Chrispeels,  J.  H„  492, 507 , 503 

Christ,  D„  669 

Christensen,  C.,  699,  707 

Christensen,  D.,  423 

Christensen,  D.  L.,  344, 474, 483 

Christensen,  L.  B.,  766,  801 

Christian,  S.,  629,  637 

Christie,  B.,  422 

Christie,  M.,  466 

Chu,C„  280, 292 

Chudowsky,  N„  604 

Chularut,  P„  120,  121,723 

Chun,  J„  630 

Chung  J„  732,  758 

Chung,  G.  K.  W.  K„  599,  600,  601. 

602,  603,  604,  802,  803 
Chung,  J.C.C.,  507 
Chung,  J.-S.,  496, 503 
Chung,  W.,  759 
Churchill,  D„  351, 353 


Cialdini.R.B.,  621, 630 
Cissell,  G„  327 

Clancey,  W.  J„  243, 248,  641 , 644 
Clarebout,  G„  707,  772 
Clariana,  R.  B„  130,  137,  138,  747, 
309,373,330,342,722,  758 
Clark,  D„  147,  150,  151,759,760, 
175,783,535 

Clark,  H.  H„  219, 227, 226, 230, 420 
Clark,  J.  M„  253, 256,  307,  373 
Clark,  K.B.,  635,  643 
Clark,  R.  C„  175, 178, 783, 522, 523, 
684,707,708,777 
Clark,  R.  E„  44, 52,  783, 212, 227, 
226, 230,  265, 266,  323,  325, 
374, 380,  386, 395, 443, 454, 
456,523,579,597,579,580, 
581,582,583,586,587,588, 

590.591.592.593.655.657, 
706,709,777,772,726,728, 
731,732, 758,875 

Clark,  R.  W„  767 
Clarke,  J .,256,466 
Clarke,  T„  721, 758 
Clauser,  B.  E„  599,  600,  601, 603, 
604 

Clayton,  J.S.,  11,79 
Clayton,  S.,  602 
Clement,  A.,  663,  667,  668,  669 
Clement,  J„  458, 466 
Clements,  D„  410,  413, 479, 427 
Clements,  P.,  643 
Cline,  F„  800 
Cline,  H.F.,  462, 467 
Cline,  R.  W„  676,  684 
Clyman,  S.  G.,  603 
Cobb,  P„  69, 72, 74, 75, 87,98,  709, 
546,548,563,565,573,575, 
746,  758 
Cobb,  T„  61, 64 
Cocchiarella,  M.  J„  710,  773 
Cochran-Smith,  M.,  625, 630 
Cockayne,  S.,  409, 479 
Cocking,  R.  R.,  26,  708,  759,  770, 
221,325 
Code,  S.,  802 
Coffey,  J.  W„  580, 592 
Coffield .  F„  302 

Cognition  and  Technology  Group  at 
Vanderbilt  (CTGV),  79, 87, 99, 
100,709, 146, 149,759,319, 
322,323,325,460,466,487, 

507.529.534.650.657,  733, 
734,  758 


834 


Author  Index 


Cohen,  A.  J„  603,  616 

Cohen,  A.  B„  49, 53 

Cohen,  D.,  407,  422 

Cohen,  I  .,292,294 

Cohen, J„ 586, 592, 663,  668 

Cohen,  M.  J.,  563 

Cohen,  P.  A.,  795 

Cohen,  W.  W„  293 

Coiro.J.,  759 

Colazzo,  L.,  208 

Cole,  B.K.  ,195 

Cole,  K„  520,  523 

Cole,  M.,  165,  167,170,171 

Cole,  P.,384, 391, 399 

Cole,  R„  217, 221 

Colella,  V.,  341, 342 

Coleman,  L.  T.,  388, 395 

Coles,  C„  557,  563 

Coles,  L.,  240 

Coller,  L.,  467 

Collier,  J.,  115,801 

Collier,  M„  775, 801 

Collies,  B„  126 ,141 

Collins,  A.  M.,  27, 19,81,99,  101, 

108.109,  147,  150,  158,759, 
161,170,230, 323,525,427, 
436, 437, 454,531,534,513, 

575.631.641.644.645.109, 
711 

Collins,  H.M.,  585, 592 
Collins,  M.,  227, 230 
Collis,  B„  177, 183, 252, 256,  270, 
213,274,275 
Colliver,  J.  A.,  490,  507 
Colucci,S„  348,  352 
Comber,  C.,  252, 256 
Committee  on  Scientific  Principles 
for  Education  Research,  761 
Compton,  L.  S.,  65 
Conati,  C„  218, 227, 279,  280, 284, 
285,287,288,291,292,295 
Conchas,  G„  280, 292 
Condly,S.J„  518, 525 
Conejo,  R.,294 
Confrey,  J.,  758 
Conklin,  J.,  215, 227 
Conlan,  O.,  555 
Connell,  J.,  55 
Conrad,  D.,  264, 266 
Conrad,  F.  G.,  643 
Constantinou,  C.  P.,  467 
Contamines,  J.,  645 
Converse,  S.  A.,  658 
Conway,  J.,  498, 501 


Conway,  T.  L.,  231,565 
Cook,  J.  O.,  190,794,257 
Cook,  K.,  610,  613,  676 
Cook,  R.,  290, 292 
Cook,  T.D.,  607,  612, 676 
Cooke, N.J., 579, 580, 583, 589, 590, 
592,180,181,186,801,804 
Cooper,  D.  W.,  475, 480 
Cooper,  G.  A.,  371, 373, 374,  375, 
318,380,381 
Cooper,  J.  O.,  796 
Cooper,  M.,  381 
Cope,  P.,430, 435, 437 
Corbett,  A.  T„  139,740,  747,213, 
220,221,292,631,643,644 
Corbin,  J.  M„  772,  805 
Corcho,  O.,  248 

Corcoran,  T.,  622, 623,625,626, 630 
Cordeiro,  P.  A.,  487, 507 
Corno,  L„  1 14,  121 , 123, 470, 472, 
480 

Cornu,  B„  768,  801 
Corry,  M.  D.,  41 1 , 423,  662,  665, 
668,  750,751,756,  758 
Cortese,  J.,  208 
Coscarelli,  W.  C.  C„  28 
Cosen,  M„  409,  479 
Cotten,  S.,  229, 257 
Cottrell,  D.M.,  271, 275 
Coulson,  J.  E.,  190,795,796 
Coulson,  R.  L„  209, 270, 224, 492, 
491,501,657,658,713 
Counts,  E.,  527, 534 
Courtenay,  B.,  438 
Cousins,  K.C.,  528,  534 
Cowan,  N.,  373, 380 
Cox,  D„  409, 479 
Craig,  S.D.,  50, 52, 218,  219, 227, 
280,292,809,815 
Craik,  F.  I.  M.,  40, 52,  87,  96,  214, 
227,392,595 
Craik,  K.,  9,  14,79 
Crandall,  B .,  580, 58 1 , 584, 585,589, 
592 

Craner,  J.,  676 
Crawford,  C.,  574, 575 
Creemers,  B.  P.  M.,  457 
Cress,  U.,  203, 208 
Creswell,  J.  W„  719,  754,  758 
Crevier,  F.,  684 
Crippen,  K.  J„  136,747 
Crismond,  D„  777, 461, 466,  503 
Cristea,  A.  I.,  471 . 475, 476, 477, 
480,  481 


Crombag,  H.  F.  M.,  553,  554,  563 
Crombez,  G.,  592 
Cromley,  J.  G„  121,  123,207,208, 
215,220 

Cronbach,  J.  L.,  471, 472, 487 
Cronbach,  L.  J„  607,  676,  731,  758 
Crone-Todd,D.E„  431, 438 
Cronje,  J.,  756,  758 
Crook,  C„  254, 255, 256, 403, 41 1 , 
414,415,479 

Crooks,  S.  M„  507, 773,  801,802 
Crooks,  T.,  603 
Cross,  D.  R.,  603 
Cross,  J.,  270, 275 
Cross,  N.,  686,  687,  690, 690 
Crowder,  N.W.,  473, 487 
Crowston,  K.,  357, 363 
Cruikshank,  K.  A.,  740 
Csikszentmihalyi,  M.,  50,  52,  709, 
219,227,324,525 
Cuban,  L„  9,  79, 212, 227, 226, 250, 
403,479, 620,624 ,630,  655, 
657, 131,758 
Cuevas,  H.  M.,  657 
Cuisamano,  M.,  504 
Cumming,  G.,  286, 292 
Cummings,  J.  A.,  610,  676 
Cuneo,  C.,  115,801 
Cunningham,  D.  J.,  15, 18, 19,27, 
12,14,81,534 

Cunningham,  W.  G„  487,  507 
Cunnington,  J.  P.  W.,  506 
Curry,  J„  201, 202 ,208,644 
Curtis,  R.  V.,  391 , 398 
Cusella,  L.  P„  133,747 
Custers,  E.  J.  F.  M.,  553, 563 
Custers,  R.,  49, 52 
Cuthbertson.  P .,437 
Cuyar,  C.,  668 
Czarnecki,  K.,  635,  644 

D 

D’Addono,  B.,  302 
D’Anjou,  P.,648,  649, 657 
D’eidio-Caston,  M.,  503 
d’lnverno,  M.,  243, 248 
D’Mello,  S.  K.,  219, 227,222 
Dabbagh,  N.  H„  229, 230, 352, 493, 
501 

Daft,  R„  423 
Dale,  E„  9,  79 

Dalton,  D„  747, 409, 413, 479 
Daly,  J„  230 


835 


Author  Index 


Dalziel,  J.,  358, 363 
Damasio,  A.  R.,  219, 227 
Damon,  W.,  109 
Daniel,  J.S.,  17, 19,  261, 266 
Danna,  M.,  381 

Dansereau,  D.  F.,  420,  422 ,  603 
Darabi,  A.  A.,  55,430, 437 , 520, 523, 
760 

Darby,  J.,  256 
Darringer,  J.,  639,  644 
Darwazeh,  A.  N.,  390,  398 
Das,  S.,  247 
Davenport,  E.,  245, 248 
Davenport,  T.,  96,  242, 248 
Davidsen,  P.  I.,  343,  815 
Davidson,  G.  V„  758 
Davidson-Shivers,  G.  V.,  204,  205, 
209,226,230 

Davies,  I.,  189,  795,  389, 396 
Davies,  J.,  526, 534 
Davies,  R.  C.,  662,  668 
Davies,  R.  K.,  239 
Davis,  B.,  30, 32 
Davis,  D.  L.,  504 

Davis, E.  A.,  156, 151,159,160,437, 
466,  467,  658 

Davis,  J„  411 ,419, 434, 435, 437 
Davis,  M„  115,  116,723 
Davis,  R.,  243, 249 
Davis,  W.K.,  507 
Davlin,  D.,  747 
Davydov,  V.  V„  27 
Dawes,  R.  M.,  580, 592 
Dawson,  D.,  353 
Dawson,  E.,  501 
Daytner,  G.,  437 
Daytner,  K.,  437 
DCMI,  352 
de  Assis,  P.,  477, 481 
de  Boer,  W„  252 ,256 
De  Bra,  P„  358, 363, 470, 475, 476, 
411,481 

de  Buisonje,  C.  D.,  505 
De  Corte,  E.,  709,  777 
de  Croock,  M.  B.  M.,  190,797, 381, 
456, 593 

De  Grave,  W.  S„  494, 495, 507 
De  Groot,  A.,  371, 380 
De  Groot,  E.  V.,  121,724 
de  Jager,B.,  429, 430, 437 
de  Jong,  T.,  203, 208, 215, 224,  308, 
375,331,335,339,341,342, 
344, 458, 459, 460, 461, 463, 
465 ,466,467 


de  Kleer,  J.,  466 

de  la  Rocha,  O.L.,  164,770 

de  la  Rosa,  J.L.,  292, 374 

de  la  Teja,  I.,  27, 364,  645,  815 

de  Leeuw,  N.,  227 

de  Souza  e  Silva,  A.  A.,  603 

De  Vera,  J.  ML  8,  79 

De  Vicente,  A.,  219, 227, 478, 481 

de  Vries,  M.,  504 

de  Wijkersloot,  J.  L.,  563 

Dean,  C.  D„  492, 493,  507, 510 

Dean,  P.  J„  27, 523 

Deary,  I.  J.,  312,  373 

DeBacker,  T.  K„  120,  121,723 

Debard,  R.D.,  299, 303 

DeBoer,  W„  747 

Debons,  A.,  242, 248 

DeBra,  P.,  743 

DeCecco,  J.P.,  12,79 

Deci,  E.  L.,  324 ,325,327 

Dede,  C.,  318,  322 ,325,466,  735, 

758 

Dees,  R.,  409, 479 

Defude,  B.,  352 

DeGroot,  A.,  319, 325 

Deimann,  M.,  215, 227 

Dekkers,  A.  C.,  52 

Del  Man,  H„  639,  644 

del  Soldato,  T.,  478, 481 

Del  Val,D„  224 

Delacruz,  G.  C.,  603 

Delaney,  H.,  397 

Delgado,  F.,  504 

DeLisi,R„  549,  563 

Delisle,  R„  487, 507 

DelMarcelle  M.,  760 

Delucchi,  K.,  800 

Demetriadis,  S.,  773,  801 

DeMeulle,  L„  773 

Dempsey,  B.,  357,  363 

Dempsey,  J.  V„ 28,  132,  137,747, 

194.796.341.344.740.760 
DeNisi,  A.,  127,  134,  742,  515,  516, 

517,522,523,524 
Dennen,  V.  P.,433, 434, 437 
Denning,  P.,  246, 248 
Dennis,  R.  A.,  604 
Dennis,  S.,  223 
Denzin,  N.  K.,  61 , 64 
Deo,  A„  247 

Derntl,  M.,  361, 363,  639,  643,  644 
Derry,  S.  J„  69,  72,  73,  74,  75,  81, 

154. 155.759.760 

Des  Marchais,  J.  E.,  495, 507 


Design-Based  Research  Collective, 
746, 158, 159,  573, 575,  745, 
746,  756,  758 
Desimone,  L.,  280, 292 
Desmarais,  M.  C.,  521, 522, 523 
Dessinger,  J.  C.,  399 
Detambel,  M.  H.,  190,  795 
Deterding,  R.,  503 
Detweiler,  M.  C„  444, 45 1 , 455,  456 
Deubel,  P.,  396 
Deutsch,  M.,  404,  406, 479 
Deutsch,  W.,  188,795 
Devlin-Scherer,  R.,  230 
Dewey,  J„  23, 24, 27,  34, 37, 59,  64, 

81.99. 101. 108.709.385.396, 
410, 481, 1U,  801 

DHEW,  31, 32 
Di  Noia,T„  352 
Di  Sciascio,  E.,  352 
DiCintio,  M.,  54 

Dick,  W„  61 , 64, 126, 747, 158, 759, 

191. 194.795.392.394.396, 
472,437,649,  657,707,709, 

777.742.753.753 
Dickey,  M„  319,  325 
Dickinson,  T.  L.,  653 
Didier.E.K.,  234,  237, 239 
Dijkstra,  S.,  20,  23,  174,  733,  734, 

386,396,399,640,  644,645, 
706,  772,  773,  774,  307 
Dillenbourg,  P„  219, 227, 404, 479 
Dillon,  A.,  205,208,  215, 227 
Dills,  C.R.,  706, 27,  772 
Dimitracopoulou,  A.,  335, 342, 462, 
466 

Dimitriadis,  Y.  A.,  363, 364 
Dimitrova,  D.,  423 
Dionne,  G.  B.,  603 
Dionne,  J.  P.,787,  788,  805 
Dirkx,  J.  M.,  630 
diSessa,  A.,  146,  759,  753 
Distlehorst,  L.  H.,  490, 507 
DiVesta,  F.  J„  1 14,  1 19,  723,  724, 

472.437.731. 753 
DiVesta,  F.T.,  723 
Doane,  W.  E.  J.,  463 
Dobbs,  G„  575 

Dochy,  F„  486, 488,  490, 491 , 502, 
505,583,592,767,783,300, 
305 

Doctorow,  M.,  1 14,  1 15,  723 
Dodds,  P.,213,  214.  220, 227 
Dodero,  J.  M.,  365 
Dodge,  B„  528, 534,  710,  772 


836 


Author  Index 


Dodge,  T.,  708 
Dods.R.F.,  487, 489, 502 
Doering,  A.,  265, 266, 416, 417,  419 
Doerr,  H.  M.,  46, 52,  330, 343 
Doignon,  J.  P.,  213, 221 
Doiron,  R.,  526,  534 
Dolezalek,  H.,  16,79 
Dolmans,  D.  H.  J.  M„  492, 496, 497, 
501,502,503,505 
Domizi,  D.,  535 
Donahue,  M.,  105, 109 
Donahue,  P.,  564 
Donahue,  V.  M.,  191,795 
Donaldson,  J.  F„  493, 494, 501,502 
Donaldson,  R.,  487, 502 
Donini,  F.  M.,  352 
Donnelly,  M.  B„  501,503, 505 
Donovan,  M.  S.  E.,  654,  657 
Dooley,  K„  629 ,630 
Doom,  T.,  343 
Doorman,  L.  ML,  548,  563 
Doostdar,  A.,  227, 230 
Dori,  Y.  J.,  152,  153,  154,  759, 487, 
500 

Dornan,  T„  557,  558,  559,  561 , 563, 
564 

Dorner,  D„  27,  308,  313,  809,  815 
Dorr,  H.,  781, 802 
Dorsey,  L.  T„  520, 524,  665, 666, 
668,  669,  709,  772 
Dorval,  B.,  565 
Doucet,  M.  D.,  490, 502 
Dougher,  M.  J.,  59,  63,  64 
Douglas,  I.,  245 ,248,249 
Douglass,  M.  H.,  239 
Dowling,  C„  271, 275 
Downes,  S.,  347,  350, 352,  526,  534 
Downing,  S.  M.,  770,  801 
Drake,  L„  641 ,644 
Draper,  R.  C.,  592 
Drayton,  B.,  110 
Dreher,  D.,  343 
DresehM.,  134,747 
Dreyfus,  H.  L.,  27 
Dreyfus,  S.E.,  27, 444,455 
Driscoll,  D.,  52 

Driscoll,  M.  P.,15,79,24,27,60,61, 
64,69,71,73,77,87, 104,709, 
747, 166,770,  270,275,574. 
575,707,775,768,807 
Drucker,  P.  F.,  243, 248 
Druckman,  D.,  407, 420 
Druin,  A.,  343,  662,  669 
Drukker,  J.,  504, 564 


Drummond- Young,  M.,  497, 502 
du  Boulay,  B„  342,  478, 481,482 
Dubois,  A.,  628,  630 
Dubrovsky,  V.,  422 
Duch,  B.  J.,  487, 495, 496, 497, 500, 
502,  505 

Duchastel,  P.  C.,  393, 396,  71 1 , 772 
Duchowski,  A.  T.,  786,  788,  807 
Dueber,  W„  708 

Duffy,  F.  M.,  624,  630,  664,  670, 
695,697,698,707 
Duffy,  T.M.,  15,18,19,27,  69,70, 
72, 73, 74,75, 79, 80, 80, 87, 82, 
98,99,  100,  101,  106,708,709, 
770, 326,534,616 
Dufner.D.,  274, 274 
Dufresne,  R.  J.,  554, 565 
Duguid.P.,  708, 457, 554 
Duguid,  S.,  454 
Duitama,  F.,  348, 352 
Dumais,  B.,  507 
Dumais,  S.,  216, 227 
Dumas,  J.  S.,  662,  665,  669 
Dunbar,  S.  B.,  604 
Duncan,  J.,  392,  396 
Duncan,  K.D.,  191,795 
Duncan,  R.  G.,  467, 658 
Dunn,  S.,  321, 525 
Dunn,  T.G.,  389,  596 
Dunser,  A.,  204,  205, 208 
Dunwoody,  S.,  208 
Durnin.R.,  409, 410,  423 
Dutton,  J.,  707 
Duval,  E„  220, 227, 555 
Dweck.C.S.,  48,  52,  219, 227 
Dwyer,  C.  A.,  214, 227, 389, 396 
Dwyer,  D.J.,  323, 525, 403, 410, 420 
Dwyer,  F„  114,  115,  118,724,218, 
227,321,525,389,596,550, 
563,123,759 
Dwyer,  H.,  807, 804 
Dwyer,  J.,  492, 502 
Dyck,  J.L.,  50, 52 
Dyer,  L.,  409, 410, 412, 420 
Dynarski,  M.,  676 
Dziuban,  C.  D„  271 , 272, 273,  275 

E 

Eagle,  C.  J.,  502 
Eakin,  D.,  327 
Early,  S.,  49, 52, 592,  777 
Ebbinghaus,  H.,  41 , 52 
Eccles.D.  W.,780,807 


Ecclestone,  K.,  302 
Eckblad.G.,  49, 52 
Eco,U„  27 

Edelson,  D.  C.,  146,  153,759 
Edelson,  E.,  422 
Edens,  K„  487, 502 
Edwards,  C.  C.,  231 
Edwards,  J.,  327 
Edwards,  R.,  253, 256 
Eelen,  P.,  592 

Egan,  D.  E„  3 19, 325,  37 1 , 380 
Eggan.G  .,223,482 
Eggemeier,  R.  T.,  584, 592 
Ehn,  P.,662,  663, 668,  669 
Ehrenberg,  A.,  51, 57 
Ehrmann,  S .  C.,  609,  61 1 , 676 
Eigen,  L.D.,  190,795 
Einhorn.H.,  585, 592 
Eisenberg,  M.  B .,  235 , 239, 344, 530, 
534 

Eisenhardt,  K.  M„  620, 628,629 ,630 
Eisenhart,  M.,  720,  758 
Eisensteadt,  R.  S.,  491 , 502 
Eisner,  E.W.,  608,  61 1,676 
Eklund,  J.,  476, 480, 487 
Ekman,  M.,  528, 534 
Ekman,  P.,  280, 292 
Elbinger.L.,  650,  657 
Elen,!.,  453, 455, 456, 590, 591, 592, 
707,708,709,710,772,768, 
802 

Elio,  R.,  443, 455 
Elliott,  A.  J.,  48,  52 
Elliott,  M.  J.,  437 
Elliott,  S.P.,257 
Ellis,  A.,  270, 275 
Ellis,  D„  245, 248 
Ellis,  H.C.,  445, 455 
Ellis,  R.,  228, 230,  321,325 
Ellsworth ,  J .  B . ,  620 ,  62 1 , 622 ,  630 
Ellul,  J.,  649,  657 
Elmore,  J.,  603 
Elshafei.D.L.,  495, 502 
Ely,  D.,  28,  620 ,630 
Elzubeir,  M.  A.,  558, 563 
Emery,  C.  D„  742,  743,  750,  756, 
760 

Emurian,  H.,  320, 325 
Ender,  P.,  423 
Endicott,  J.,  524 
Engel,  C.,  502 
Engelberg,  M.,  631 
Engelhart,  M.  D.,  26 
Engelmann,  M.  D.,  190,  795 


837 


Author  Index 


Engelmann,  S.,  63,  64 
Engestrom,  Y„  254, 256,  528, 535 
English,  J.,  409, 479 
English,  R.E.,  391, 394,  396,452, 
455 

Engstrom,  M.  E.,  527, 534 
Enkenberg,  J.,  427, 437 
Epstein,  J.  A.,  324, 325 
Epstein,  R.  M.,  561 , 563 
Eraut,  M.,  415, 420 
Erickson,  G.,  229, 231 
Ericsson,  K.  A.,  44,  52,  242,  245, 
248,373,380 , 579,580,581. 
583,592,766,773,779,780, 
784,785,786,787,788 ,801 
Eriksson,  H.,  503 
Erkens,  G„  414,  420,  788,  801 
Ertl.B  .,231 

Ertmer,  P.  A.,  27,  69,  71 , 73,  80, 81, 
323,327,  649,657 
Eshleman,  J.  W.,  796 
Espey,  L.,  774,  801 
Esquembre,  F.,  342 
Estavan,  D.  P.,  195, 196 
Esterhuysen,  P.  W.,  631 
Estes,  F„  443, 454,  579,  583,  586, 
587,590,592,707,777 
Etelapelto,  A.,  438 
Etrl,B  .,423 

Eustace,  B.  W.,  388, 396 
Eva,  K„  326 
Evans,  C„  338,  339, 342 
Evans,  J.L.,  191,795 
Eveland,  W.  P.,  203, 208 
Evenson,  D.  H.,  487, 502 
Evenson,  S.,  635, 645 
Everitt,  N„  70,  81 
Evidence-Based  Intervention  Work 
Group  (EBIWG),  621, 623,  637 
Eylon,  B.-S.,  158,760 

F 

Fabregat,  R.,  312, 373 

Fabricius,  W.,  373 

Fairbanks,  M.  M„  544,  545, 565 

Fajardo,  I.,209 

Falk,  D.,  409, 419 

Falmagne,  J.  C.,  213, 227 

Fan,  J.,  438 

Fan,  X.,  280, 292 

Fan,  Y.D.,  325 

Fano,  A.,  456 

Faraj,S.  A.,  651, 652, 657 


Farmer,  J.  A.,  192,  795, 399,  438 
Farr.M  .,96,325,603 
Farr,  R„  109 
Fasse,  B.,  171,503 
Faulkner,  R.R.,  628,  637 
Fay,  A.  L„  340, 342, 461, 467 
Fazio,  F.,  381 
Federal  Communications 
Commission,  528, 534 
Federico,  P.,410, 420,470, 471, 472, 
413,481 
Feld,  J.  K„  395 
Feldenkrais,  M.,  444, 455 
Felder,  R.  M„  307,  309,  3 1 1 , 3 12, 
373,  805 

Feldhusen,  J.  F.,  191,795 
Feldman,  J.,  341 ,342 
Feldon,D.F„  584,  591, 592 
Feletti,  G„  487, 495, 501,502 
Feller,  J„  357, 363 
Feltovich,  L,  159 
Feltovich.  P„  209, 270, 224,  655, 
657,  773 

Feltovich,  R.  P.,  658 

Ferguson,  C.,  270, 276 

Ferguson,  M.  J.,  583,  590, 597 

Fernald.P.  S„  192,795 

Fernandez,  C.,  365 

Fernandez-Lopez,  M.,  248 

Fernandez-Manjon,  B.,  347, 352 

Ferster,  C.  D.,  190,  795 

Feuer,  M.  J.,  613,  676, 720,  758,  761 

Feuerlein,  I.,  342 

Fiddler,  M.B.,  493, 502 

Fidel,  R.,  235, 239 

Fiedler,  K„  597 

Fielding,  M„  622,  637 

Fielding,  R.  T.,  364 

Fields,  D.  C.,  28, 575,  815 

Figl.K.,  643 

Fiksdahl-King,  I.,  255 

Filipczak,  B.,  303 

Finarelli,  M.  G.,  105,709 

Finch,  P.M.,  490,  502 

Findeli,  A.,  648,  649,  655,  657 

Fine,  D.,  421 

Finegan,  A.,  660, 669 

Finger,  S.,  645 

Fink,  D„  745,  759 

Fink,  J.,  281, 289, 292, 475, 481 

Fink,  L.D.  ,803 

Finkelstein,  M.,  491 , 495, 503 

Finkle,  S.,  101,709 

Finn,  A.,  270, 275 


Finn,  J.  D.,  9,  79 
Finnegan,  Jr.,  J.  R.,  637 
Fiore,  S.  M.,  651, 653,  657,  776, 804 
Fischer,  F„  740,  167,  168,770,237, 
423,438,480,  757 
Fischer,  G.,  415,481 
Fischer,  O.,  523 
Fischer,  S.,  483 
Fiset,  J.  V.,  523 
Fish.  J.,  686,  687,  690 
Fisher,  A.,  70,  81 
Fisher,  K„  256 
Fisher,  T.,  373 

Fishman,  B.  J.,  709,  146,  759, 326, 
411,420, 622,623,624,625, 
626,627,637,654,656,657 
Fisk,  A.  D„  584, 592 
Fitts,  P.M.,  126,747 
Fitzgerald,  B.,  357, 363 
Fitzgerald,  P.  J.,  482 
Fitz-Gibbon.  C.  T„  803 
Fjermestad,  J.,  658 
Flagg,  B.  N.,  669 
Flaherty,  P.,5 14, 524 
Flak,  E.  A.,  565 
Flanagan,  J.  C.,  471, 481 
Flannery,  M.  E.,  504 
Flavell,  J.  H.,  114,  123,309,314 
Fleming,  M.,  9,  79,  707  ,  772,  723, 
758 

Fleming,  V.,  373 
Fletcher,  B.,  409, 420 
Fletcher,  J.  D„  213,  214,  220,  221, 
224, 483 

Fletcher-Flinn,  C.,  408, 420 
Floden,  R.  E.,  57,  58,  60,  61 , 66,  69, 
72,  74,  75,  82 
Flood,  R.L.,  694,  707 
Florer,  F.,  792,  793,  801 
Flores,  F.,28,  635,  641, 645 
Floyd,  R.G.,  223 
Fodor,  J.,  70,  81 ,  98,  709 
Fogarty,  R.,  487,  502 
Fokkema,  S.  D.,  27 
Foltz,  P.  W„  96,  216, 222, 223, 248, 
603 

Fontaine,  M.,  248 

Forbes-Riley,  K.,  219, 223,  280, 293 

Forbus,  K„  282, 292,  337,  341 , 342 

Ford,  D.H.,  24, 27 

Ford,  K.  M„  597 

Ford,  N„  307, 374 

Ford,  P.,252, 256 

Forer,  D.  C.,  293 


838 


Author  Index 


Forrester,  J.  W.,  461, 466 
Foshay,  R.,  189, 795 
Foshay,  W.  R„  174,  178,  180, 183, 
673,682,684,749,760, 

Foster,  E.,  522,  524 
Fowles,  M.  E„  360,  363,  604,  637, 
644 

Fowles-Winkler,  A.,  293 
Fowlkes,  J.E.,  323, 325 
Fox,  E.  J.,  57,  58,  59,  62,  63,  64,  64, 
65,  570,  574,  575 
Fox,  H.  C.,  313 
Foxon,M  .,28,575,815 
Francis,  D.  J.,  543, 562 
Francis,  J.  B.,  570, 575 
Franklin,  S„  44,  52, 227 
Franz,  T.,  325 
Fraser,  W.,  37, 37 
Frederiksen,  J.  R.,  146,  158, 161 , 
217,224,282,292,459,460, 
462,463,465 ,467,  468,484 
Fredrick,  F.  D.,  62,  65 
Fredricks,  J.,  160 

Free  Software  Foundation,  357, 364 
Freebody,  P.,  545, 562 
Freedman,  D.,  230 ,230 
Freeh,  V„  364 
Freeman,  M.,  602 
Freidus,  H.,  622,  631 
French,  J.R.  P.,621, 628,  637 
Frensch,  P.  A.,  786,  802 
Fretz,E„  760, 467,  658 
Freudenthal,  H.,  547, 563 
Frick, T.W.,  177,783,274,274,394, 
398,  662,  665,  668,  669,  670, 
758 

Fried,  R.  G„  604 
Friedland,R„  252, 256 
Friedman,  A.,  46, 52 
Friedman,  S.  F.,  27 
Friedman,  T.  F.,  736,  758 
Friend,  R„  119,723 
Friesen,  347,  350,  351, 352 
FrizelhS.,  255, 256 
Froschl,C„  281, 282, 292 
Fry,  E.  B.,  191,796 
Fry,  R„  34,  35, 37 
FSF-DEF,  357, 364 
Fu,  W.-T.,  788,  807 
Fuks,H„  242,  245, 248 
Fullan,  ML,  622,  623,  624,  625,  626, 
637 

Fullick,  P.,373 
Funk,  J.B.,  416, 420 


Furst,  E.  J.,  26 
Fussell,  S.  R.,  780,  807 

G 

Gabbard,  R„  205, 208,  215, 227 
Gabbay,  J.,  245, 248 
Gabbitas,  B.,  536 
Gabriel,  R.  P.,  644 
Gadde,L.E„  628,  630 
Gaffney,  T„  604 
Gage,  N.L.,  676 
Gagne,  E„  387, 396 
Gagne,  R.  M„  25, 27,  31 , 32,  77,  87, 
96, 174, 180,783, 191, 194,795, 
213,222,306,374,384,386, 
387,388,389,394,395,396, 
397,  399, 446, 455, 472, 487, 
530,534,583,592,636,644, 
649,657, 678,684,707,710, 
772 

Gaimari,  R.,  224 
Gainer,  F.,  807 
Gal,  S„  770 

Galanter,  E.  H.,  190,795 
Galbraith,  J.  K„  5,  79 
Galbraith,  P.,  437 
Galinsky,M„  321, 325 
Gall,  J.  P„  96,  201, 202, 208,  757 
Gall,M.  D.,  757, 768, 769, 773, 775, 
800 

Gallagher,  S.  A.,  486, 487, 489, 490, 
491.497,502,505 
Gallese,  V.,  387 

Gallimore,  R.,  427, 428, 432, 437, 
439 

Gallini,  J.  K„  43, 52 
Gallivan,  M.  J.,  357, 364 
Gallwey.W.T.,  442, 455 
Galvao,  J.R.,  318,  326 
Gamblian,  V.,  585, 592 
Gamma,  E.,  147,  759,  790, 807 
Gamson,  Z.,  609, 676 
Ganesan,  R.,  573, 575 
Ganeshan,  R.,  294 
Garcia,  C.E.,  95, 96 
Garcia,  F.  J.,  358, 363 
Garcia,  P.,312, 374 
Garcia,  R.,  364 

Garcla-Barrios,  V.  M.,  285, 292 
Gardner,  A.,  535 
Gardner,  D.F.,  309,  374 
Gardner,  H.  E„  71, 73,  81,96,  98, 
106, 709 


Garfield.  J.,  73,  87 
Garfinkel,  H„  780,  807 
Garfinkel,  R.  S„  565 
Garg,  A.,  321, 326 
Garhart.C.,  410, 420 
Garibaldi,  A.,  421 
Garland,  N.J.,  487,  502 
Garofalo,  J.,374 
Garrett.  S.,  87,  708 
Garrison,  D.  R„  219, 222,  270,  271 , 
274, 275, 422,  804 
Garrison,  J.,  65,  69,  79,  87 
Garrison,  S.,  708, 534 
Garstens,  H.  F.,  396 
Garton,  F.,  423 

Garzotto,  F„  471, 475, 476,  481,643 
Gasevic,  D.,  353,  364 
GaBner,  K.,  343 
Gaver,M.,  239 
Gaver,W.W„  262, 266 
Gavurin,  E.  L,  191,795 
Gearhart,  M.,  420 
Gee,  J.  P„  105,  709, 169,  770,  212, 
218,222,302,303,318,320, 
321,326,710,772 
Geer,  C.H,  229, 230 
Geis,G.L„  509, 524 
Gemeny ,  B .  A . ,  325 
General  Medical  Council,  556,  557, 
560,563 

Gentner,  D„  27,  322, 326, 458, 466 
Georgoudi,  M.,  59,  66 
Gerace,  W.  J.,  563 
Gergen,  K.  J.,  59,  65 
Gerhard,  M„  324, 326 
Gerjets,  P„  201, 208,  785,  802 
Gernsbacher,  M.  A.,  222 
Gertner,  A.,  292 

Gery,  G„  510, 519,  520, 524,  655, 
657, 665,669 
Getsie.R.F.,  389,  396 
GhaziriH.M.,  242, 247 
Ghelani,  T.,  535 

Gholson,  B„  52, 227,222, 292,  875 
Giarratano,  J.  C.,  242, 248 
Gibbons,  A.  S„  330,  342, 346, 347, 
353,356,364,635,643,636, 
637,638,639,642,644,645, 
690,  708,  772 

Gibbons,  N.J.,  338,  339, 342 
Gibson,  G.  V.,  784 
Gibson,  I.  W.,  498,  500 
Gibson,  J.  J.,  87, 98,  709,  254, 256, 
262,266 


839 


Author  Index 


Giere,  R.  N.,  331, 542 
Gifford,  E.  V.,  59,  61, 62,  65 
Gifford,  R.,  599,  600,  658 
Gijbels,  D.,  488, 496,  502, 505 
Gijlers,  H.,  339, 342 
Gijselaers,  W.  H„  487,  496,  502, 
506 

Gilabert,  R  .,96 

Gilbert,  N„  166,770,334,345,361, 
364 

Gilbert,  P.  K„  229, 230 
Gilbert,  T„  27,  512, 524 
Gilder,  J„  335,  343 
Gilding,  T.,  256 
Giles,  C.L.,  527, 535 
Gilkison,  A.,  495,  502 
Gilley,  J.  W„  630 
Gilliam,  S.,  222 
Gillis,  P.  D.,  760 
Gillon,S.M„  302,  303 
Gilmartin,  K„  311,381 
Gioia,  I.,  109 
Giouvanakis,  T.,  53 
Gist,  M.E.,  321, 326 
Gitomer,D.H.,601 604 
Glanz,K„  502 
Glaser,  B.G.,  80,  81 
Glaser,  R.  D„  12, 19, 27,  96,  174, 
183,  190,195,196,197,  280, 
287,  292, 294,  325, 326, 327, 
380,319,459,460,467, 470, 
471, 472, 480, 481,  530, 534, 
580,583,589,592,598,603, 
604,131,758 
Glasgow,  N.A.,  487, 502 
Glass,  G.  V.,  396 
Glazer,E.M„  428, 437 
Gleason,  M.  E.,  552, 563 
Glick,  A.,  238, 240 
Glickman,  A.  S.,  658 
Globerson,  T.,  761 
Gluck,  K.  A.,  294 
Glynn,  S.M.,  707,  712 
Gobert,J.D„  458, 466 
Gobet.F.,  319, 326 
Godfrey,  J.M.,  275 
Goel,V„  687,  690,  756,  758 
Goeters,  K.,  321 , 326 
Goettl,B.  P.,451, 455 
Goetz,  E.T.,  77,  78,  81, 96 
Goetz,  J.  P„  768,  771, 772,  774,  802 
Goetzfried,  L„  732,  758 
Gogoulou,  A.,  141 
Goin,  L.,  656 


Golan-Duncan  R..  160 
Golas,  K.  C.,  32 
Golbeck.S.L.,  549, 563 
Gold,  N„  503 
Goldbeck,R.  A.,  194 
Goldberg,  N.S.,  390, 396 
Goldberg,  S.,  650,  657 
Goldenberg,  C„  623,  626,  631 
Goldhaber,G.M„  628,  631 
Goldman,  E.,  656 
Goldman,  S.  R„  53, 108, 222, 534, 
656,  658 

Goldschmidt,  B.,  686,  690 
Goldstein,  I.,  294 
Goldstein,  L.S.,  230, 230 
Goliber.M.,  252, 256 
Gollwitzer,  P.  M.,  591 
Gomes,  M.  R.,  326 
Gomez,  L„  109, 411, 420, 422 
Gomez-Perez,  A.,  246, 248 
Gonschorek.  M.,  475, 481 
Gonzalez,  A.,  247 
Gonzalez,  G.,  281, 292 
Goodenough,  D.,  37 
Goodman,  B.,  224 
Goodman,  N.,  687, 690 
Goodman,  P.  H.,  248 
Goodrum,  D.  A„  524, 661 , 662, 665, 
668,  669,  712 

Goodson,  L.  A.,  446, 454,  685,  690, 
707,777,741,757 
Goodwin,  C.,  634,  644 
Goodwin,  G.  F.,  803 
Goodyear,  P.,  253,  254,  255, 256, 
109,712,111,802 
Goos,M„  436, 437 
Gopalakrishnan,  M.,  760 
Gordin,  D„  759,  760 
Gordon,  A.,  108 
Gordon,  D.  L.,  564 
Gordon,  I.  J.,  772 

Gordon,  J„  660,  669,  685, 690,  710 
Gorsky,  P.,  96 
Goswami,  U.,  788,  802 
Gott.S.P.,  592 
Gottdenker,  J.,  341 , 343,  344 
Gouli,  E.,  139,747 
Gowin,  D.  B.,  600,  604 
Grabinger,  R.  S.,  81,396 
Grabowski,  B.  L.,  27,  112, 113, 120, 
723,306,307,312,374,321, 
326,  690,  815 
Grace-Farfaglia,  P.,  5 1 , 52 
Graen,G.B.,  444, 455 


Graesser,  A.  C„  204,  206,  207, 208, 
214,216,217,218,219,227, 
222, 223, 224, 285, 292, 437, 
433,  803,  815 
Graetz,  K.,  252, 256 
Graf,  E.  A.,  294 
Graf,  S„  312,  313, 374 
Graff,  M„  204,  205, 206, 209,  272, 
275 

Graham,  C.  R„  269,  270,  271 , 273, 
274,275,276,536,610,676 
Graham,  G.,  227, 237 
Graham-Pole,  J.,  444, 455 
Granott,  N.,  552, 563 
Grant,  J.  M.,  327 
Gratch,  J„  218, 222 
Grattan,  J.,  535 
Gravatt,  B.,  408, 420 
Gravemeijer,  K.,  303, 547, 563, 565, 
767 

Graves,  M.  F.,  544, 563 
Gray,  J„  777, 503,  635,  644 
Gray,  P.,  669 
Gray,  S.H.,  731, 758 
Gray,  W.D.,  215, 222 
Gredler,M.E„  321,326 
Green,  C„  320, 326 
Green,  K.  C„  273, 275 
Green,  N.  C„  265, 266 
Green,  P„  183 
Green,  R.  H„  592 
Greenbaum,  J.,  662,  669 
Greene,  B.  A.,  709,  529,  532, 535 
Greene,  H.C.,  229, 230 
Greenfield,  P.  M„  237 
Greenhill,  L.  P.,  755,  758 
Greeno,  J.  G.,  69,  71 , 73,  74,  75,  78, 
80,  81, 98,  709,  165,770,230, 
254, 256 

Greer,  J.  E„  281 , 284, 287, 288, 289, 
290 ,293,294 
Greer,  R.  D.,  63,  65 
Gregory,  J.,  263, 267 
Gregory,  P.,  669 
Gregory,  T.,  106,  709 
Gretchell-Reiter,  K„  581, 584,  585, 
589,592 

Griffiths,  D„  362, 364 
Grigg,  W„  564 

Grigoriadou,  M.,  747,312, 374 
Grimm,  V„  334, 343 
Grinter,  R.  E„  245, 248 
Groen,  G.  J.,  504 
Groh.S.E.,  502 


840 


Author  Index 


Grol,  R.  P.  T.  M.,  802 
Gropper,  G.  L„  191 ,195 
Gros,  B.,  676,  684, 707,  712 
GroBe,  C.,  381 
GroBler,  A.,  331 , 343 
Grosslight,  L.,  344 
Grotzer,  T.  A.,  443, 455 
Group,  P.,  759 
Gruber,  T.  R„  599, 603 
Grudin,  J.,  662,  669 
Guba,  E.  G„  71, 72,  81,608,  611, 
616,719,758,  771,802 
Guhe,  M.,  219, 222 
Guinn,  C.,  326 

Gulia,  M„  245,  246, 249, 423 
Gulikers,  J.  T.  M„  556,563, 784, 802 
Gunawardena,  C.  N.,  264, 266, 411, 
420 

Gunawardena,  L.,  219, 222 
Gunstone,  R.  F„  339,  341 , 343, 344 
Gunzenhaeuser,  R.,  482 
Gupta,  T.S.,  498, 502 
Gur,  B.,  183 
Gurtner  J.,  418, 421 
Guskey,T.R„  608,  61 1,676 
Gustafson,  J.  L.,  669 
Gustafson,  K.  L„  25, 27,  194, 195, 
524,670,707,712,740,741, 
742,743,754,  758,  761 
Guthrie,  J.  T.,  96, 208 
Giitl,C.,  292 
Gutman,  M.,  121, 123 
Guzdial,M.,  102,108, 109, 160,434, 
437,  439, 535 
Guzzo,R.  A.,776 ,802 
Gyles,  N.,  275 

H 

Haag,  B.  B.,  120,725 
Habermas,  J.,  694,  701 
Hacioglu,  K.,  227 
Hackbarth,  S.,  262, 266 
Hacsi,  T.  A.,  617 
Haderlie,  S.,  208 
Hadfield,  J.,  563 
Hadley,  W.  H„  222, 293 
Hadwin,  A.  F„  330, 343 
Haertel,  G.,  604,  616 
Hafter,  A.,  604 
Hagen,  J.  D.,  420 
Hagler,  P„  728,  758 
Haider,  H„  786,  802 
Haig,  C.,  514,  523 


Haigh,  S.,  614,  617 
Hair,  E.  C,  438 

Hiikkinen,  P„  14, 19, 256, 438,  708, 
772 

Haladyna,  T.  M.,  770,  801 
Hall,  A.  K.,  473, 481 
Hall,  E.M.,  302, 583, 592 
Hall,  G.E.,  621, 637, 656,  657 
Hall,  H.,  245, 248 
Hall,  R.  J.,  164,  166,  169,  777, 782, 
804 

Hall,  W„  373 
Hallinger,  P.,487, 507 
Halloun,  I.,  465, 466 
Hamalainen.  S.,  421 
Hambleton,  R.  K.,  784,  802 
Hamel,  L„  604 
Hamerlynck,  L.  A.,  62,  66 
Hamill,L.B„  229, 230 
Hamilton,  D.,  321, 327 
Hamilton,  J.  D.,  557, 563 
Hamilton,  R.  J.,  389, 396 
Hamm,  M.,  418 
Hammer,  D.,  343 
Hammer,  M.,  699,  700,  701 
Hammrich,  P.  L.,  742 
Hanani,  U.,  477, 481 
Hancock,  T.E.,  138,141,142 
Hand,  T„  575 
Hanesian,  H.,  220 
Hanges,  P.  J.,  804 
Hanna,  S.  E.,  524 
Hannafin,  J.,  89,  90,  95, 96, 419 
Hannafin,  K.  D.,  137,141 
Hannafin,  K.  M„  526, 528,529, 530, 
531.535 

Hannafin, M.  J.,  62,63 ,65, 141 , 347, 
353,389,392,394,396,399, 
409,410,411,412,420,428, 
437,535,536,707,712,719, 
727,730,732,733 ,758,759, 
761 

Hannafin,  R.  D„  717,  718,  730,  732, 
755,  759 

Hannay,N.B„  356, 364 
Hannon,  B.,  334, 343 
Hannum,  W .H.,27, 141 , 592,  644, 
712 

Hansell,  M.  H.,  308, 373 
Hansen,  D.  N.,  473, 481 
Hansen,  E.  G.,  286, 293,  294,  294 
Hansen,  L.,  668,  758 
Hanson,  K.,  109 
Hara,N„  437,  780 ,802 


Harabagiu,  S.  M.,  217, 222 
Harackiewicz,  J.  M.,  135,  138,743, 
324, 325 

Harasim,  L..  608,  617 
Harasym,  P.  H.,  502 
Harden,  R.  M„  562,  563 
Hardwick,  C.S.,  24. 27 
Hardwidge,  B.,  799,  802 
Harel,  G.,  656 
Harel,  I.,  461, 466 
Hargreaves,  A.,  745,  759 
Harik,P  .,603 
Haring,  T.  G.,  62,  66 
Harlen,  W.,  112,723 
Harless,  J.H.,  524 
Harmon,  T.  C.,  603 
Harp,  D„  527,  535 
Harp,  S.F.,  96 
Harris,  N.D.C.,  396 
Harris,  P.L..  43, 52,  270,  275 
Harris,  R.,  342 
Harris,  T„  331, 343 
Harrison,  K.  A.,  514, 524 
Harrison,  P.  A.,  230 
Harrison,  R.,  256 
Hart,  R.S.,  472, 487 
Harter,  D.,  292 

Hartley,  J.,  189,  190,  191,795,389, 
396,  707,  772 
Hartley,  K.,  237 
Hartman,  J.,  272,  273, 275 
Harvard,  B.C.,  503 
Harverty,  L.  A.,  308, 374 
Hasan,  B„  321, 326 
Hasselbring,  T.  S.,  325,  656 
Hatala,  M.,  267 
Hathorn,L.G„  264, 266 
Hativa,  A.,  410, 420 
Hativa,N„  732,  759 
Haugsjaa,  E.,  248 
Havelock,  R.  G.,  27 
Hawkins,  J.,  387, 396 
Hawkins,  Jr„C.H„  521, 524 
Hawkins,  S„  403, 412, 415, 420 
Hay,  K.  E.,  99,  70S,  759 
Hayakawa,  S.  I.,  11, 79 
Haycock,  C.  A.,  526, 535 
Hayes,  L.  J.,  65 

Hayes,  S.  C.,  56,  57,  58,  59,  61, 62, 
63,64 ,64,65 
Haynes,  B.  C.,  222 
Haynes,  R.  B.,  524 
Hays,  T.A.,  95, 96, 498 
Haythornwaite,  C.,  423 


841 


Author  Index 


Hayward,  L.M.,  431, 438 
Hazlewood,  W.  R.,  757 
Heap,  N.,  535 
Heard,  S.,  565 
Hebbeler,  K.,  543, 563 
Hecht,  S.  A.,  566 

Hedberg,  J.  G.,  273, 275,  570,  573, 
575,  649,  652,  654, 657,  658 
Hede,  A.,  40, 48, 52 
Heer,  J.,  245, 248 
Heffernan,  N.  T.,  138 ,747 
Heffner,  T.  S„  803 
Hegarty,M.,  46, 52, 95, 96, 155,759, 
215,222 

Heider,  K.  G.,  775,  802 
Heift,  T.,  138, 141 
Heinich,  R.,  262, 266,  707,  772 
Heinzmann,  J.,  286, 294 
Heller,  S.,  687,  690 
Helm,  R.,  759,  801 
Helokunnas,  T.,  498, 502 
Hemstreet,  S.,  506 
Henderlong,  J.,  219, 223 
Henderson,  T.  L„  274 ,  555, 563 
Hendler,  J.,  248, 363 
Hendricks,  C.  C„  430,  438 
Hendriks,  M.,  456 
Hendriksen,  A.  H.  M.,  803 
Henze,  N„  286, 293, 475, 477, 478, 
481,482 

Herbsleb,  J.D .,364 
Herdman,  C.  M.,  805 
Heredia,  M.,  363 

Herl,  H.  E„  598,  600,  603,  604,  781, 
802,803 

Herman,  G„  392, 396 
Herman,  J.  L„  599,  602,  609,  676 
Hermans,  D.,  584, 592 
Hermans,  H.,  364,  365 
Hernandez,  N.,  349, 353 
Hernandez-Leo,  D.,  362, 364 
Hernandez-Serrano,  J.,  102, 109 
Herrala.J.,  498, 502 
Herrenkohl,  L.,  777 
Herring,  S.C.,  227, 237 
Herrington,  J.,  146,  759,  777, 265, , 
266, 434, 438,  760 
Herscovitz,  O.,  159 
Hersh,R.H„  300, 303 
Hertzog,  N„  550,  552,  564 
Herzig,S„  490,  500 
Herzog,  C.,  475, 481 
Heseltine,  R.,  256 
Hessinger,  J.,  800 


Hestenes,  D.,  458, 462, 466 
Hevner,  A.  R.,  648,  649,  657 
Hewatt,  R.,  803 
Hewes,  G.,  757 
Hewitt,  J.,  219, 222 
Heycox,  K.,  487, 507 
Hick,  T.  L.,  468 
Hicken.S.,  732,  759 
Hickey,  A.  E,  191,795 
Hickey,  D.T.,  461, 466 
Higginbotham,  T.,  327 
Higgins,  L.,  487, 502 
Higgins,  N„  772,  773,  774,  802 
Higson,  V.,  758 
Hilbert,  S.,  505 
Hilgard,  E.  R.,  308, 373 
Hill,  C.,  224 

Hill,  J.  R.,  63 , 65, 353, 526, 528, 529, 
530,535,685,690 
Hill,  W.H.,  26 
Hillelsohn,  M.  J.,  483 
Hiltz,S.R.,410, 411,420,658 
Hinds,  P.  J.,  592 
Hinton,  G.  E.,  53 
Hipps,G„  655,  656,  657 
Hiroyuki,  M.,  777 
Hitch,  G.  J.,  41, 57 
Hively,  W„  598,  599,  601, 603 
Hlynka,  D.,  27 

Hmelo,  C.  E„  341 , 343, 461 , 466, 
486, 487, 491, 502,  506, 65S 
Hmelo-Silver,  C.  E„  154,  759,  760, 
336,343,486,488,502 
Ho,  F.,  677 
Ho,  P.,  139,742 
Hoadley,  C.  M„  759,  760,  770 
Hoban,  Jr.,  C  .F.,  9,  79,  726,  759 
Hobbs,  D„  326 
Hobma,  S.  O.,  784,  802 
Hockenberry,  M.,  747 
Hockings,  P.,  775,  802 
Hodgins.H.  W„  526, 535 
Hodgins,  W„  227,  346,  347,  353 
Hodgkinson,  K.,  257 
Hodgson,  V.,  252, 256 
Hoersch,  A.  L„  487,  505 
Hofer,  B.  K.,  147,  760 
Hofer,  M.  J.,  222 
Hoffman,  B„  498,  503 
Hoffman,  E.  J.,  650,  651 , 657 
Hoffman,  P.,585, 592 
Hoffman,  R.  R„  580,  581 , 582, 592, 
657 

Hofman,R„  204, 209 


Hofmann,  C„  334,  344 
Hogan,  D„  552, 565 
Hogan,  K.,  465, 466 
Hogan,  L„  105,  70S 
Hohl,H.,  475, 482 
Hokansen,  B.,  690 
Holbrook,  J.,  777, 503 
Holder,  J.  K.,  239 
Holding,  D.  H„  127, 137,747 
Holdredge,  T.  S.,  395 
Holdrinet,  R.  S.  G.,  505 
Hollan.J.,  641, 644 
Hollan.J.D.,  459,  466 
Holland,  C.M.,  66 
Holland,  G.  J.,  473, 482 
Holland,  J.G.,  190,  191,795 
Hollands,  J.G.,  290, 294 
Holmberg,  B.,  261, 266 
Holmes,  A.,  495, 535 
Holmes,  D.  B.,  503 
Holt,  D„  534 

Holton,  D.L .,343,466,502 
Holtzblatt,  K„  662,  668 
Holubec,  E.,  427 
Holum,  A.,  759 
Holyoak,  K.  J.,  553, 564 
Homans,  G.  C.,  406, 420 
Homme,  L.  E.,  795 
Homquist,  S.,  759 
Honebein,  P.  C.,  322, 326 
Honey,  M.,  676 

Honey,  P„  306,  309,  3 10,  3 1 1 , 374 
Honeycutt,  L.,  226, 230 
Hoogveld,  A.  W.  M„  453, 455 
Hooper,  S.,  1 14,  723, 407, 409, 410, 
411,412,479,420,422,689, 
690 

Hoover,  S.  P.,494, 500 

Hopkins,  C.  J.,  239 

Hoppe,  H.U.,  335, 343 

Horber,E.,  775, 802 

Hord,  S.  M.,  621, 637, 656,  657 

Horn,  R.E.,  389,  393, 396 

Horney,M.  A.,  202, 209 

Horning,  J.,  248 

Horvath,  J.  A.,  580, 593 

Horwitz,  P„  330,  338,  341 , 343, 466 

Hosie,  L.,  563 

Hoska.D.M.,  134,747 

Hosokawa,  M.  C.,  501 

Houle,  C.,  35,  36,  37 

House,  R.,  439 

Hovland,  C.  I.,  8,79 

Howard,  D.  M.,  566 


842 


Author  Index 


Howard,  K .,592,711 
Howard,  R.  A.,  373 
Howe,  N„  296,  297,  298,  299,  300, 
302 ,303 

Howison,  J.,  357, 363 
Hsi.S.,  146, 147, 149, 152, 156, 157, 
160, 170 

Hsieh,  P„  719,  720,  723,  724,  725, 
759 

Hsieh,  Y .,759 

Hsu,  C.,  350, 353,  639,  644 

Hu,  X.,  222 

Huang,  C.,  409, 411, 412, 420 
Hubal,  R.,  326 
Hubbard,  D.  C.,  141 
Hubbard,  L„  523 
Huber,  B„  520 ,524 
Huberman,  A.  M„  775,  781, 803 
Huberman,  M.,  109 
Hubscher,R.,  209,  255, 256 
Hucker,  S.,  96 
Hudson,  L.,  310, 314 
Hudson,  P„  431, 438 
Huebner,  T.  A.,  617 
Huff,  K.  L.,  599,  600,  603 
Hughes,  A.  S.,  487, 504 
Hughes,  J„  650,  657 
Hughes,  J.E.,  190,795 
Hughes,  J.  P.,  1 09 
Huglin,  L„  524 
Hui,  C.,  444, 455 
Hulbert,  L„  658 
Hulshof,C.D„  736,  759 
Hult,  R.  E.,  115,  116,723 
Humbert,  J.,  271, 275, 276 
Hume,  D„  809,575 
Hummel,  H.  G.  K„  555, 564 
Hummel,  J.  H.,  62,  65 
Hundert,  E.  M.,  494,  506 
Hundhausen,  C.  D.,  414, 423 
Hung,  D„  500,  505 
Hung,  W„  497, 498, 503 
Hunt,  D.  L.,  521 , 524 
Hunt,  E.,  214, 222,  654,  657 
Hunt,  N.,  322, 326 
Hunt,  S.K.,  237 
Hunter,  B„  105,709 
Huot,B.,  528,  533, 535 
Hursh,  D.  E.,  192,  795 
Husek,T.R„  598,  604 
Husic,  F.  T„  160,  389, 396, 467 
Hutchins,  C.  L„  621, 637,  695,  701 
Hutchins,  E.  L„  164,  166,  168,  777, 
254, 256, 466,  644,  734,  759 


Hutchins,  R.  M.,  385, 396 
Hutchinson,  C.,  504 
Huth,  K„  130,  132,  136,  137,  138, 
139,742 

Huttenlocher ,  D . ,  4 1 1 , 41 9 
Hutton,  D.,  264, 266 
Hylkema,N„  504, 564 
Hythecker,  V.  I.,  409, 420, 422 
Hyysalo,  J.,  803 

I 

Ice,  K„  767 

Ifenthaler,D.,  53,  781, 502 
Iiyoshi,T.,  529,  535 
Ikemoto,  G.,  630 
Im,Y„  275 
IMS,  349, 353 
Inagaki,  S.,  435 
Ingelman-Sundberg,  M.,  503 
Inglis,  A.,  386, 396 
Ingram,  A.  L.,  264, 266 
Inkpen,K„  409, 420 
Inman,  T.  F.,  245 
Inn,  M.  C.,  760 

Institute  of  Electrical  and  Electronics 
Engineers  (IEEE),  347,  349, 353 
Instructional  Systems  Technology 
Department,  659, 690 
International  Federation  of  Library 
Associations  and  Institutions 
(IFLA),  234, 240 
Ioannidou,  A.,  336, 344 
lokospentaki,  K.,  566 
Ip,  A.,  267 

Irlbeck,S.,  570,  572, 575 
Isenberg,  R.,  403,  420 
Isensee,  S.,  670 
Ishikawa,S„  759, 255 
Isken,  J.,  637 
Issenberg,  S.  B.,  562, 564 
Ito.M.,  167, 169,777 

J 

Jackendoff,  R.,  641 , 644 
Jackson,  C.,  705 
Jackson,  G.  T.,  222, 224 
Jackson,  J.  C.,  341, 343 
Jackson,  P.,  242, 248 
Jackson,  S.,  152, 760, 343, 462, 466, 
734,  759 

Jackson,  T.,  215, 222 
Jacob,  E„  229, 230 


Jacobs,  A.  E.J.  P.,499,  503 
Jacobsen,  M.,  487,  503 
Jacobson,  I.,  363 

Jacobson,  M.  J.,  200, 209, 270, 224, 
255,  655,  773 
Jacobson,  R.,  96 
Jacoby,  L.  L.,  40, 52 
Jaegar,  R.  M.,  502 
James,  C.M.,  603, 599 
James,  W.,  58,  65 
James,  W.B.,  309, 374 
Jameson,  A.,  282, 289,  290, 291 , 293 
Jamieson,  P.,  252, 256 
Janicki,  T.  C.,  472, 452 
Janneck,  C.  D.,  657 
Jansen,  H.,  503 
Jansen,  R.  G.,  503 
Janssen-Noordman,  A.  M.  B.,  453, 
455 

Januszewski,  A.,  5,  79,  640,  644 
Janzen,  K.,  256 
Jarvela,S.,  415, 420, 435 
Jarvelin,  K.,  783,  507 
Jarvis,  P.,  36, 37 
Jaspers,  J.,  414,  420 
Jean  Piaget  Archives  Foundation,  36, 
37 

Jean,  P.,  507 
Jebbett,  L.,  505 
Jeffrey,  L.,  374 
Jeffries,  R„  371, 350 
Jehng,J.C„  200, 209 
Jekielek,  S.  M.,  430, 435 
Jelfs,  A.,  529,  532, 535 
Jelsma,  O.,  456,  505 
Jenkins,  F.,  602 

Jenlink,  P.  M„  30, 32, 621 , 623,624, 
625,627,637,695,696,697, 
707 

Jensen,  F.,  312,374 
Jensen,  M.,  227, 230, 242, 245 
Jentsch,  F.  G„  321,323 ,326,327, 
800 

Jeong ,  A .  C .,  229 , 230, 246 , 245 , 780 , 
802 

Jermann,  P.,  479, 452, 453 
Jetton,  T.  L.,  230, 230 
Jewett,  D.,  527,  534 
Ji,  Q.,  222 
Jiang,  B.,  504 
Jirasko,  M.,  204,  205, 205 
Jochems,  W.  M.  G.,  212, 222, 267, 
427,455,707,772 
Joeng,  H.,  747 


843 


Author  Index 


John,  B.E.,  222 
Johnsen,  L.,  524 
Johnsey,  A.,  123 
Johnson,  A.,  564 
Johnson,  C.,  685,  690 
Johnson,  D.  L.,  527, 535 
Johnson,  D.  W„  37, 57,  219, 222, 
250,323,526,389,596,404, 
405,406,407,408,412,413, 
417,420,427,425,777,802 
Johnson,  E.  J.,  595,  804 
Johnson,  F„  37, 57, 406,420 
Johnson,  K„  63, 65,  192,  795,  658 
Johnson,  L.,  604,  641 , 644 
Johnson,  M.,  224,  690 
Johnson,  P.  E.,  585, 592 
Johnson,  R.,  159, 321 , 326, 404,405, 
406,407,408,412,413,417, 
420,421,423,801 
Johnson ,  77.  L.,  678,  684 
Johnson,  R.  T„  219, 222, 250,  389, 
596,  802 

Johnson,  S.,  169,777 
Johnson,  T.  E„  765,  767,  776,  111, 
778,781,782,805 
Johnson,  W.  L„  217,  218, 222, 285, 
281,293,294 

Johnson-Laird,  P.  N„  9, 74, 45, 79, 52 
Johnston,  J.  H.,  323, 526 
Jona,M.Y„  456,  649,  658 
Jonassen,  D.  H.,  15,79,27,  60,61, 
62, 69, 65, 70, 7 1 , 73, 74, 80, 87 , 
99, 102, 709, 725, 126, 127, 128, 
136,747,201,209,212,222, 
306,307,312,574,321,322, 
326,331,345,351,355,392, 
393, 396, 434, 438, 472, 480, 
488,499,507,503,530,535, 
574,575,579,580,581,583, 
592,  636,  644,  660,  669,  706, 
707,709,772,809,810,875 
Jones,  A.,  358,  364,  677 
Jones,  B.F.,  487, 503 
Jones,  D„  570,  571, 572,  574,  575 
Jones,  E.  K„  113,801,802,  804 
Jones,  H.L..  98,  190,795 
Jones,  M.  K„  392, 397,  666,  669 
Jones,  N.,  270, 275 
Jones,  R.,  274 

Jones ,  T .  S .,  665 , 669, 740 , 74 1 , 743 , 
753,  759 
Joosten,  T.,  275 

Joosten-Ten  Brinke,  D.,  361, 362, 
364 


Jordan,  H.  M„  272, 276 
Jordan,  P.,  224 
Jorgensen,  A.,  657,  658 
Jorgensen,  D.  L.,  780,  802 
Jorgensen,  N.,  357, 364 
Jorm,  A.  F„  543, 564 
Joseph.  D.,  759,  770 
Joseph.  R„  693,  707 
Joshi,S.,  334, 344 
Jost,K.L„  493, 503 
Jovanovic,  J.,  353 
Joy,  M„  353 
Joyce,  B.,  795 
Jubert,  A.,  245, 248 
Judy,  J.  E„  280,  292 
Juelich,  C.,  342 
Juge,F„  275 

Jules-Rosette,  B.,  775,  800 
Jung,  C„  310, 374 
Jung,  D.I.,  652,  657 
Junod,  A.,  505 
Jupp,V„  766,  804 
Jurafsky,  D.,  216, 222 
Just,  M.  A.,  46,  52 
Juszkiewicz,  P.  J.,  604 

K 

Kaariainen,  J.,  803 
Kabassi,  K.,  281 , 293 
Kafai,  Y.B.,  461, 466 
Kagan,  J.,  543,  564 
Kahai,  S.  S.,  658 
Kahle,B„  526, 535 
Kain,D.L.,  487, 503 
Kaiser,  W.  J.,  603 
Kakabadse,  A.,  627,  637 
Kakabadse,  N.  K.,  627,  637 
Kaleta,R„  273, 275 
Kali,  Y„  146,  147,  152,  153,  154, 
158,760 

Kallison,  J.  M„  388, 396 
Kalweit,  C.  M.,  758 
Kalyuga,  S„  214, 222, 308, 374, 377, 
380,810,875 
Kamin,C.S„  498,  503 
Kammeyer-Mueller,  J.  D.,  435, 438 
Kane,  M„  599,  603 
Kangassalo,  M.,  550, 564 
Kanselaar,  G.,  565 
Kantor,R.,  708, 534 
Kantor,R.J„  453, 455 
Kanuka,H„  264, 266 
Kanuta.H.,  270,  271, 275 


Kaplan,  C.  A.,  41, 54 
Kaplan,  R„  245, 249 
Kapoor,  A.,  280, 293 
Kappers,  A.  M.  L.,  455 
Kaptelinin,  V.,  167,777 
Karagiannidis,  C.,  470, 482 
Karampiperis,  P.,  220, 224 
Kardash.C.  A.,507 
Kareev,  Y„  412, 427 
Kareken,D.  A.,  584, 592 
Karlan,  J.  W.,  105,709 
Karlsson,  M.,  343 
Karp,  C.,  395 
Kashihara,  A.,  308, 374 
Kasowitz,  A.,  386,  393, 396,  676, 
684 

Kasper,  H.,  502 
Kasvi,J.J„  520,  524 
Kato,  H.,  167,777 
Katsionis,  G.,  224 
Katz,  E„  50,  57 

Katz,  1.  R„  598,  599,  600,  601, 603 
Katzenbach,  J.  R.,  65 1 , 657 
Katzir,T„  788,  802 
Kauffman,  D.F.,  207, 209 
Kaufman,  D.  M.,  492, 495, 502, 503 
Kaufman,  R.,  27 

Kay ,  J. , 284 , 289 , 290 , 292 , 293 , 475 , 
482 

Kayama,  M.,  476, 482 
Kaye,  A.,  264, 267 
Kaye,  B.K.,  51, 52 
Kayes,  D.C.,  780,  782,  805 
Kays,  E„  570,  571 , 572,  574, 575 
Kazanas,  H.  C.,  514, 524,  707,  773 
Kazman,  R.,  643 
Kearsley,  G.,  320, 326,  755,  759 
Keating,  T.,  759 
Keefe,  J.  W„  309, 374 
Keegan,  D.,  261 , 266 
Keeler,  C„  410, 427 
Keeps,  E.  J.,  28 
Keisler,  E.  R„  191,795 
Keizer,  P.  K.,  502 
Keller,  B.H.,  390, 397 
Keller,  F.  S„  63, 65,  191,  192,795, 
262,266,471,482 
Keller,  J.M.,  32,  135,  747, 215, 227 
Keller,  T.,  330, 344 
Kelley,  C.M.,  40, 52 
Kelley,  H„  406, 427 
Kellner,  D.,  526, 535 
Kelly,  A.,  602,  746,  759 
Kelson,  A.,  487, 507 


844 


Author  Index 


Kemp,  J.,  66,  194, 195,  707,  712 , 
760 

Kempler,  T.  M.,  108 
Kendall,  S„  222 
Kendra,  K.  A,  628,  631 
Kennedy,  C.,  263, 267 
Kenny,  R„  115,723 
Kermani,  H.,  267 
Kern,  D.E.,  566 
Kerr,  S.T.,  742,  759 
Kessler,  G.  D.,  657 
Kester,  L.,  456 

Ketelaar,  H.  W.  E.  R„  775,  800 
Ketelhut  ,  D .  J .,  46 1 , 466 
Kett,  A.  L.,  184 
Kettebekov,S.,  280, 293 
Khalili,  A.,  408, 427 
Khan,  B.H.,  27,  252 ,256 
Khan,  S.,  228, 230 
Kibby,  M.,  418 
Kieras,  D.  E.,  46, 52 
Kiesler,S.,  41 1,427, 422 
Kiewra,K.  A.,  41 1,397 
Kiili,K.,  50,  52 
Kim,  C.  H„  644 

Kim,  H„  222, 223, 498, 503,  530, 
531,535,  759 
Kim,  J„  535,  600,  603 
Kim,  M.,  535 

Kim,  Y.,  6,79,217,218,227,503, 
722,  759,  809,  815 
Kinder,  C„  115,  118,723 
Kinder,  D.,  63, 65 
Kindfield,  A.  C.  H.,  466 
King,  A.,  1 15,  723, 217, 222, 409, 
427 

King,  K„  407, 415, 479 
King,  N„  563 
Kinlaw,  C.  S.,  657 
Kinlaw,  D.  C.,  657 
Kinshuk,  286,  293,  308,  309,  312, 
374 

Kintsch,  E.,  216, 222 
Kintsch,  W„  27, 44,  52,  89,  91 , 92, 
96,200,206,209,223, 373,380, 
583,592,708,772 
Kinzer,  C.  K.,  53, 325, 504, 656,  759 
Kinzie,  M.  B.,  216, 222,  731, 732, 
759 

Kirk, E.  P.,49, 57 
Kirk,  J.J.,  680,  684 
Kirkley,  J.R.,270 ,275 
Kirkley,S.E.,  270,275 
Kirkpatrick,  D.  L.,  608,  61 1 , 677 


Kirschner,  P.  A.,  167,777, 176, 179, 
182,  183, 183,184,267,  372, 
374.  380, 421,423, 444, 451, 
455, 456, 563, 564,  583,  587, 
589,591,593,709,772,742, 
743,756,759, 774,802,809, 

810.875 

Kirshner,  D„  75, 80,  87, 98,  709 
Kirtman,  L„  622,  624,  626,  627, 637 
Kitchener,  R.,  62,  63,  66 
Klabbers,  J.  H.  G.,  648,  649,  650, 
656, 657 

Klahr,  D„  176,  783, 374,  372,  380, 
465,466 

Klausmeier,  H.  J.,  471, 482 
Klawe.M.,  420 
Kleer,  A.  A .,466 
Klein,  D.  C.  D„  600,  603,  604 
Klein,  G.  A.,  308, 375, 592, 582, 583, 
584, 593 

Klein,  H.  J.,  435, 438 
Klein,  J.  D.,  24, 27, 95, 96, 203, 208, 
210,231,398,514,524,713, 
717,718,722,748,755,758, 

760. 773.807.802.813.875 
Klein,  J.P.,  759 

Klein,  M.,  550,  552,  564 
Kleiner,  A.,  707 
Klimoski.R.,  781. 802 
Kline,  A.  E„  489, 493, 505 
Klivington,  K.  A.,  27 
Klopfer,  E.,  416, 427 
Kluger,  A.N.,  127,  134,742,515, 
516,517,522,523,524 
Knabel.O.B.,  203, 208 
Knapczyk,  D.,  523 
Kneebone,  R.,  263, 266 
Knight,  A.  B„  803 
Knight,  C.,  362, 364 
Knight,  D„  653,  657 
Knoll,  J.  W„  493, 502 
Knowles,  M.S.,  559, 564 
Knowlton,  D.  C.,  7,  79 
Knowlton,  J.,  728,  758 
Knudsen,  J.,  334, 343,  668 
Ko,S.,  770,  802 

Kobsa,  A.,  281, 282,  284,  289,  290, 
292,293,481 
Koch,  N.,  361 ,364,479 
Koch,  S.,  270, 506 
Koedinger,  K.  R.,  133, 136, 137, 138, 
140,141,142,143,216,220,222, 
284,292,293,374,474,  479,480, 
482, 483, 602, 603, 637, 643, 644 


Koenderink,  J.  J.,  455 
Koestner,  R.,  325 
Koffka,  K.,  407, 427 
Kogut.B.,  242, 248 
Koh.K.,  553, 564 
Kohnstamm,  G.  A.,  552, 564 
Kolb,  A.,  37 

Kolb,  D.  A.,  33,  34,  35,  36, 37,  309, 
310,374,319,320,326 
Kollick,  P.,246, 248 
Kolodner,  J.  L.,  101,  102,  103,709, 
150,  151,760, 165,  168,777, 
343, 432, 434, 438, 466, 487, 
502, 503,  530, 535,  654,  657 
Kolodner,  K.,  566 
Komis,  V.,  335, 342,  466 
Koper,  E.  J.  R„  361 , 362, 364, 365, 
564 

Koper,  R„  222, 364,  639,  644,  772 
Korndle,  H.,  742 
Kornell,N„  219, 223 
Koroghlanian,  C.  M.,  95,  96, 208 
Kort,B„  219, 223 
Korzybski,  A.,  1 1 , 79,  330, 343 
Koschmann,  T.,  27, 166,  777, 264, 
266,  776,  802 
Koskela,  J.,  803 
Kostama,  H.,  658 

Koszalka,  T.  A.,  573, 575,  756,  759, 

781,805,809,811,875 
Koul,  R.,  137,  138,747 
Kourilsky,  M.,  118,723 
Kozma,R.  B.,27, 212,215, 223, 226, 
230,  265, 267, 343,  655,  657, 
727,729,732,733,734,  759 
Kozol,  J.,637 

Krajcik,  J.  S.,  103,  708,  709, 146, 
151,159,160,466,467, 482, 

631 , 654, 655 , 656, 657, 658, 759 
Kramarski,  B.,  121, 723 
Krammer,  H.  P.  M.,  190,  797 
Krathwohl,  D.  R.,  740 
Kraut,  R.  E„  807 
Krechmer,  K.,  358, 364 
Kreijns,  K.,  777, 264, 267, 41 1 , 427 
Kreisman,  N.,  507 
Krentler.K.  A.,  228, 230 
Kresh,E„  398 
Kriz.S.,  96, 222 
Krug,  D.,  743 
Krug,  S.,  661, 669 
Krynock.K.,  487, 503 
KuT.W.,  208 
Ku,  H„  722,  759 


845 


Author  Index 


Kucan,  L.,  562 

Kuehlmann,  A.,  639,  644 

Kuh,  E.,  639,  644 

Kuhlthau,  C.  C„  235, 240 

Kuhn,  D.K.,  554,  555, 564 

Kuhn,  G.  J„  491, 503 

Kuhn,  S.,  661, 662,  663,  665, 669 

Kuiper,  W.,  761 

Kukich,K„  604 

Kukulska-Hulme ,  A.,  262, 263, 267, 
364 

Kulhavey,  R.  W„  114,118, 123 
Kulhavy,  R.  W„  127,  132, 133, 135, 
137, 138,740, 141,142,143 , 
214 ,223 

Kulik,  A.  J„  473, 482 
Kulik,  C.  C.,  133,  137, 140, 142 
Kulik,  C.  L.  C.,  192,795 
Kulik,  J.  A.,  140,  742,  795, 265, 267 
Kuliko  wich ,  J .  M . ,  82 , 200 , 202 , 203 , 
209,  734,  759 

Kummerfeld,  J.  R„  415,482 
Kumpf,  D.,  506 
Kumpulainen,  K.,  549, 564 
Kuno,H„  363 
Kurland,  L„  459 ,466 
Kurtz,  S.,  487, 503 
Kurtz  dos  Santos,  A.,  462,466 
Kushner,  E.  J.,  239 
Kushniruk,  A.,  617 
Kutay.C.,  139,742 
Kuusela,  H.,  787,  802 
Kuzuoka,  H„  777 
Kwong,  K.,  364 
Kyng,M.,662 ,668,669 

L 

Labinowics,  E.,  546,  564 
Lacefield-Parachini,  N.,  631 
Lachter,  G.  D.,  795 
Lacroix,  G.,  658 
Lagowski,  J.  J.,  430, 437,  439 
Laham,  D„  96, 223, 248,  603 
Lahav,  O.,  160 
Lai,  K.-W.,  625,  626,631 
Laird,  J.,  243, 248 
Lajoie,  S.  P„  27, 223, 294, 429, 438, 
482 

Lakoff,  G.,  60,  65,  71 ,  <57 
Lalik.R.M.,  725,  395 
Lama,  M.,  363 
Lamb,  B.,  350, 353 
Lambdin-Kroll,  D.,  314 


Lamberski,  R.  J.,  723,  759 
Lambert,  B.,  353 
Lambiotte,  J.,  420, 422 
Lambiotte,  J.  G.,  600,  603 
Lamon,  M.,  422 
Lampert,  M.,  323, 326 
Lancaster,  L.  C.,  296,  297,  300, 302, 
303 

Lance,  K.  C.,  234,  235,  236,  237, 
238,240 

Land,  S.  M.,  27,  392, 396, 434, 437, 
438,  529,532,535,707,733, 
772,  759 

Landa,L.N„  393, 397, 446,  455, 
413,482 

Landauer,  T .  K . ,  96,  2 1 6 , 223 ,  246 , 
248,  599,  603 
Landes,  S.,  742 
Landow,  G.,  202, 209 
Lane,  D.M.,  592 
Lane,  J.,  321, 326 
Lane,  N„  325 

Lane,  S.,  622,  623,  624,  631 
Lane,  W.D.,  373 
Laney.J.D.,  119, 121,724 
Lang,  A.,  43, 52 

Langan-Fox,  J.,  780,  781, 782,  783, 
802 

Langdon,  D.,  27 
Langer,  A.  M.,  430,  438 
Langer,  P.,  396 
Langfield-Smith,  K.,  802 
Langille,  D.  B.,  502 
Large,  A.,  235, 240 
Larochelle,  M.,  60, 65 
Larose,  R.,  54 
Larson,  C.,  420, 422 
Laseau,  P.,  686,  687,  690 
Lashley,  K.  S.,  7,79 
Lassila,  O.,  248,  363 
Laster.S.,  270, 275 
Latchem,  C.,  252, 256 
Latham,  G.  P.,  323, 326,  510,  51 1, 
515,516,524 

Latour,  B.,  10,11,81, 164, 166, 168, 
777 

Latour,  L,  364 
Laurillard,  D.,  16,  79 
Lauten,  G.  N.,  105,709 
LaVancher,  C.,  227 
Lave,  J., 27, 69,75,79, 82,98, 101, 
104, 106,709,770, 165, 168, 169, 
171 ,254,256,264,267,421 ,438, 
631,644,108,712,134,760 


Law,  J.,  30, 32 
Law,  M.  P.,665,  669 
Lawless,  C.  J.,  774,  802 
Lawless,  K.  A.,  105,  200,  202,  203, 
208, 209,  644 

Lawrence,  N„  622, 623 , 625 , 626, 630 
Lawrence,  R.,  218,  223 
Lawrence,  S.,  527,  535 
Lawson,  B.,  687, 690 
Lawson,  D.  W.,  795 
Lax,  L.R.,  434, 438 
Lay,  E„  462 ,467 
Layng.T.  V.,  63,  65 
Lazonder,  A.  W.,  344,  467 
Le  Maistre,  C.,  742,  759 
le  May,  A.,  245, 248 
Leader,  L.  F.,  804 
Leahy,  W.,  378,  380 
Learning  Technology  Center,  The, 
650, 658 

Lebiere,  C.,  284, 292, 583, 584, 597, 
785, 800 

Lebow,  D.,  61, 65 
Leclair,  R.,  523 
Lecocq,  C.,  352 

LeCompte,  M.  D.,  61, 65,  768,  771 , 
772, 774, 802 
Lee,  C„  563 

Lee,  D.,  130,  747, 722,  758 
Lee,  E.Y.C.,  219, 223 
Lee,  H.-S.,  760,  767, 467 
Lee,  J.,  293, 47 1 , 413,482,491 ,503, 
616 

Lee,  J.  J.,  600, 603 
Lee,  J.K.,  203, 209 
Lee,  J.-Y.,  580,  586,  593 
Lee,  M.,  215, 223, 413, 427 
Lee,  M.  J.,  203, 209,  279, 293 
Lee,  O.,  275 
Lee,  R.L.,  593 
Lee,  V.E.,  543, 564 
Lee,  V.L.,  62,  65 
Lee,  W.W.,  707,  709,  772 
Lee,  Y.,  119,  121,724 
Lee-Chai,  A.,597 
Leelawong,  K.,  282, 293 
Lefoe,G.,  273, 275 
Legenza,  A.,  723 
Lego  Group,  334,  343 
LeGrand  Brandt,  B.,  427, 438 
Legree,  P.  J.,  760 
Lehmann,  A.  C.,  766,  784, 801 
Lehrer,  R.,  322, 326,  33 1 , 343, 458, 
466,  758 


846 


Author  Index 


Lehtinen,  E.,  415, 427 
Leighton,  C.,  520, 524 
Leinonen,  T.,  347,  353 
Leith,  G.  O.  M„  190,795 
Lemke,  E.,  412, 479 
Lemon,  M.,  208 
Lempp,H„  559, 564 
Lengel,  R.,  423 
Lentz,  R.,  43, 52 
Leonard,  M.,  364,  645 
Leone,  A.  J.,  160 
Leont’ev,  A.  N.,  25, 27 
Lepper,  M.  R„  218,  219, 223, 418, 
421 

Lerch,  F.  1.,  807 
Lerner,  J.,  357, 364 
Lesgold,  A.  M.,  24, 27,  216, 223, 
319,526,415,427 ,429,438, 
478, 482 

Lesh,  R„  46, 52,  330,  343,  781 , 802 
Lesser,  E.,  244, 248 
Lester,  F.,  309, 314 
Lester,  J.,  222,  644 
Lester,  J.  C.,  293,  478, 482 
Letondal,  C.,  667 ,669 
Leu,  Jr.,  D.J.,  736,  759 
Leutner,  D.,  415,482,  800 
Levenson,  W.  B.,  8,  79 
Levenstein,  I.  B.,  223 
Levesque,  D.  O.,  629,  631 
Levie,  W.  H.,  9, 79. 43, 52, 707,  772, 
723,  758 
Levin,  D.,  616 
Levin,  H.M.,  590, 593 
Levin,  J.,  412, 427 
Levin,  J.  R„  389, 597, 719, 720, 721 , 
727,  759,  761 
Levine,  A.,  347,  350, 555 
Levine,  J.  M.,  28, 267,  776,  802 
Levine,  L.  J.,  49, 54 
Levita,  I.,  427 
Levitin,  D.  J.,  93, 96 
Levonian,  E.,  726,  759 
Levy,  E.  I.,  44, 53 
Levy,  M.,  263 ,267 
Levy,  R.,  604 
Levy,  S.,  30 ,32 
Lewin,  K„  620,  631 
Lewis,  D.,  684 
Lewis,  E.,  629,  631 
Lewis,  E.L.,  755,  759 
Lewis,  L.,  18, 20 
Lewis,  M„  292,480 
Lewis,  N.  J.,  270,  275 


Lewis,  R.,  256 
Lewis,  R.  L.,  44,  54 
Lewis,  S.,  423 
Leyser,  Y.,  630 
Li,  E.  Y„  52 
Li,  M„  604 

Li,  Z.,  392 ,397,  665 ,669 
Liang,  T.  P.,  344 
Liaw,F.-R.,  564 
Liben.L.S.,  552, 565 
Liber,  O.,  351, 353 
Lieberman,  D.  A.,  325 
Lieux,  E.  M.,  489, 492, 493, 503 
Light,  P.,254,  255, 256 
Lim,  B.,  616 
Lim,J„  453, 455 
Lin,  C.,  204,  205, 209 
Lin,  J.,  409, 427 
Lin,  J.-Y.,  465, 466 
Lin,  T„  286, 293,  308,  309,  312, 314 
Lin,  X.,  770,  760 
Lincoln,  Y.S.,  61. 64,  71. 72,  81. 
771.802 

Lindahl,  L„  554, 564 
Lindamood,  C.  H.,  544,  564 
Lindamood,  P.  C.,  544, 564,  565 
Lindberg,  M.  A.,  43, 52 
Linden,  M„  115,  119,724 
Lindheim,  E.,  803 
Lindsley,  O.  R„  192,  193,  194,  795, 
796 

Lindvall,  C.  M„  470, 471 , 484 
Linebarger,  D.L.,  95,  96 
Linebarger,  J.  M.,  654,  657 
Lingard.L.,  779,  780,  802 
Link,  M.,  321, 326 
Linke,  R.  D.,  388, 397 
Linn,  M.,  760 

Linn,  M.  C.,  146, 147, 148, 149, 150, 
151.  155,  156,  157,  158,759, 
160,161,396,434,437,  459, 
461.465 ,466,467, 487,503, 
533,535 

Linn,  R.  L„  597,  601, 602,  604 
Linser,  R.,  267 
Lintern.G.,  443, 451, 456 
Linton,  F.,  224 
Linzmayer,  O.,  650,  657 
Lipnack,  J.,  651, 657 
Lippert,  L.  R.,  237 
Lippincott,  J.,  524 
Lipsky ,  M.,  621 , 637 
Litman,  D.  J.,  219, 223,  280, 293 
Little,  P.,507 


Littleton,  K„  708  ,  772 
Littlewood,  S.,  561, 564 
Liu,  H„  427 

Liu,  M„  340, 342, 506,  735,  759 
Liu,  T.C.,  427, 438 
Livingston,  G.,  503 
Ljungberg,  J.,  357, 363 
Llamas-Nistal,  M.,  644 
Llorens,  S.,  53 

Locke,  E.  A.,  323, 326,  510,511, 
515,516,524 

Lockee,  B.,  12,79,  190,796 
Lockhart,  R.  S„  40,  52,  87,  96,  214. 
22,392,395 

Lockwood,  F„  252, 256,  262, 267 
Loftin,  R.  B.,  325 
Loftus,  E.  F.,  48,  52 
Logan,  G.D.,  319,  326 
Logie,  R.H.,  44, 52 
Lohman,  D.  F.,  583, 593 
Lohman,  M.  C.,  491, 495, 503 
Lohner,  S.,  335,  337, 343, 458, 462, 
463,465 ,467 
Lohr.L.L.,  92,  96 
Lombardo,  T.  J.,  70,  71, 72,  75,  82 
Lonigan,  C.  J.,  542, 564 
Lonsdale,  M.,  235,  236,  231,240 
Looi.C.,  216,  219, 223 
Loong.D.H.  W„  431, 438 
Lopez,  B.,  292 
Lorenzo,  G.,  270, 275 
Losada,  M.,  779,  780,  802 
Lou,  Y„  728,  729,  759 
Louca,  L.,  337,  338, 343 
Louwerse,  M.  M.,  222 
Love,  W„  409, 427 
Lovelace,  J.  J.,  244, 248 
Low,  R.,  375, 380 
Lowe,  C.  A .,222,534 
Lowe,  R.  K„  43, 48, 52,  214, 223 
Lowther,  D.  L.,  773,  760 
Lowyck,  J.,  14,  79, 256,  709  ,  772, 
768, 802 

LSA  Research  Group,  222 
Lu,  S.,  222 

Lucas,  K.F.,  430, 438 
Lucas,  T.,  707 
Luck,  M.,  243, 248 
Luckin,  R.,  478, 481,482 
Lukas,  J.,  138,740 
Luke,  R.,  666,  667, 669 
Lukosch,  S.,  248 

Lumsdaine,  A.  A.,  12,  79,  190,  794, 
196 


847 


Author  Index 


Lund,  E.,  137, 745 
Lundgren-Cayrol,  K.,  264, 266, 364, 
645 

Lundy,  D.  H.,  45 1 , 455 
Luppino,  G.,  109 
Luria,  A.,  112,724 
Luthans,  F„  510, 517, 518, 522, 523, 
524 

Lyddon,W.  J„  57,  65 
Lyman,  P.,  527, 535 
Lynch,  C„  143,224,294 
Lyons,  A.  S.,  556, 564 
Lyons,  C.,  684 
Lysaught,  J.  P.,  196 
Lytinen,  S.  L.,  343 

M 

Ma,  J.,  227 

MacArthur,  C.,  322, 326 

Maccoby,  N.,  196 

Macdonald,  J.,  532, 535 

MacDonald-Ross,  M.,  196 

Maceli,  J.,  505 

MacFarlane,  S.,  669 

MacGregor,  D.,  593 

Machin,  M.  A.,  450, 455 

Machiraju,  V.,  363 

Mackay,H„  257 

Mackay,  N„  7 1 ,  S2 

Mackay,  W.  E„  662,  667,  668,  669 

MacKenzie,  N.,  17, 19 

Mackey,  M.,  533, 535 

MacLeod,  S.M.,  504 

MacPherson,  K.  A.,  327, 456 

Madden,  N.  A.,  655,  658 

Maddison,  S.,  229, 237 

Maddux,  C.,  527, 535 

Madey,G„  357,  364 

Madigan,  S.  D.,  631 

Madsen,  K.  H„  668 

Mager,  R.  F.,  27 

Magliano,  J.  P.,  223,  787,  803 

Magliaro,  S.  G„  79,  64,  81,534 

Maguire,  M.,  661 , 669 

Maher,  M.  A„  229, 230 

Maher,  M.  W„  276 

Maier,  F.,  331 , 343 

Mainemelis,  C.,  326 

Maiorano,  S.  J.,  222 

Maitland,  B„  487, 503 

Majer,  K.,  409, 422 

Maki,  H.  R.,  413,482 


Maki,S.  W„  473, 482 
MaKinster,  J.  G.,  108 
Makitalo,  K.,  434, 438 
Malikowski,  S .,437 
Malkonen,  E.,  421 
Malone,  T.,  218, 223 
Maloor,  P.,  224 
Malott.R.  W.,63 ,65 
Malouf,  D.,  322, 326 
Manchester,  W.,  302, 303 
Mancuso,  J.  C.,  60,  65 
Manderveld,  J.  M.,  362, 364 
Mandin,  H.,  502 

Mandinach,  E.  B.,  462, 467, 609, 617 

Mandl,  H.,  167,  168, 170,423 

Mandler,  G„  49, 52, 445 ,455 

Mandler,  J.  M.,  445, 455 

Manlove.S.,  344, 464, 467 

Mann,  K.  V.,  492, 503, 565 

Mannes,  B.,  206, 209 

Manos,  K.,  224 

Mantovani,  G .,327 

Manz,C.C.,652 ,657,658 

Mao,  J„  522 ,524 

Maor,D.,  229, 230 

March,  S.T .,657 

Marcone,  S.,  391, 397 

Marcus,  G.,  325 

Maredia,  N.,  563 

Margaritis,  M.,  342 

Margaryan,  A.,  Ill ,  183,275 

Margetson,  D.,  493, 503 

Margolis,  M.  J.,  600,  603,  604 

Margolis,  S.  A.,  564 

Marineau,  J.,  222 

Marino,  O.,  364,  645 

Mark,  M.  A .,222,293 

Marker,  A.,  5 14, 524 

Marker,  R.  W„  481 

Markle,  D.  G.,  26 

Markless,  S„  237, 240 

Markowotz,  N.,  137, 142 

Marks,  C.B.,  725 

Marlino,  M.  R.,  398 

Marra,  R.  M.,  229, 257 

Marsella,  S.,  293 

Marsh.  H„  427 

Marshall,  J.M.,  390, 397 

Marston,  L.,  484 

Martenson,  D.,  490, 503 

Martin,  B„  223,  410, 427 

Martin,  C.  A„  296,  300, 303 

Martin,  D.,  482 


Martin,  E.,  342 
Martin,  G.  L.,  63,  65 
Martin,  J.,  504 
Martin,  J.  H„  216,  222 
Martin,  K.  J„  492, 494, 501,503, 
555 ,563 

Martindale,  T.,  527,  535 
Martinez,  A.,  363 
Martinez,  G  .,96 
Martinez,  M.  E.,  603 
Martlnez-Mones,  A.,  483 
Martins,  P.  G.,  326 
Marton,  F.,  255, 256 
Marx,  R.  W„  108, 159,160, 631 , 656, 
657,  658 

Marzano,  R.  J„  174,  176,755 
Marzo,  J.  L.,  575, 574 
Masie,E„  270, 275 
Maskarinec,  G.  G.,  498,  506 
Mason,  J.B.,  132,742 
Mason,  R„  226, 231,  252, 256,  264, 
267, 535 

Massa,  L.,  321, 326 
Massimo,  V.  S„  228, 257 
Mastaglio,  T.,  481 
Masters,  J.,  433, 438 
Mathan,  S.  A.,  133,  137,742 
Mather,  A.,  535 
Mathewson,  J.  H .,96 
Mathieu,  J.  E„  327,  111,  781, 803 
Matson,  J.  L.,  63, 65 
Matsuda.N.,  284, 293 
Mattaini ,  M .  A . ,  63 , 65 
Matthews,  H.  R.,  276 
Matthews,  M.  R.,  60,  65, 327 
Maudsley,  G„  493, 494, 503 
Maughan,  C.,  556,  564 
Maupin,  F.,  586, 593 
Mautner,  T.,  60,  65 
Maxwell,  N.  L„  487,  496, 503, 504 
Mayadas,  F.,  211,275 
Maycunich  Gilley,  A.,  630 
Mayer,  J.  A.,  631 

Mayer,  R.  E„  42, 43, 52,  89,  90,  91, 
95 ,96,  114,  115,  118, 121,724, 
139,740,  742,  176,  178,  179, 
755,754,214,217,218,225, 
254 ,256,  308,309,574,321, 
324 ,326,  327,  335,339,340, 
342, 343,  372, 380, 381,  389, 
597,595,459,461,465,467, 
598,599,600,602,707,708, 
709  ,  777  ,  772  ,  775 


848 


Author  Index 


Mayes,  T.,  475 
Maymon,  T.,  656 
Mayo,  P.,494,  503 
Mayor,  J.  R.,  396 
Mazor,  E.,  160 
Mazur,  E.,  229, 231 
Mazur,  J.  M.,  219, 223 
Mazzolini,  M.,  229, 231 
McAlister,  S.,  414, 421 
McBride,  W„  10,79 
McCabe,  C.,  520, 524 
McCaffrey,  M„  409,  411,413, 423 
McCalla,  G.  I.,  290-291, 293, 294 
McCarthy,  B„  174, 175, 184 ,  574, 
575 

McCarthy,  J.  E.,  353 
McCarthy,  L.,  171 
McClean,  P.,320, 326 
McClelland,  J.L.,  40, 52, 53 
McConnell,  D„  264, 267,  41 1 , 421 
McCormack,  D.,  344 
McCracken,  D.  D.,  661, 669 
McCrorie,  P.,  565 
McDaniel,  B.,  222 
McDermott,  R.,  439 
McDonald,  C.,  409, 410,411, 412, 
421 

McDonald,  J.  K„  188, 796, 639, 641 , 
645 

McDonald,  S.,  767 
McElroy,  M.,  244,  245, 248 
McEwan,  P.  J.,  590,  593 
McFarland,  D„  321,327 
McGaghie,  W.  C.,  564 
McGill,  I.,  36, 37 
McGilly,  K.,  27 
McGinn,  M.K.,  657, 654 
McGinn,  R.  E.,  356, 364 
McGreal,R„  263, 267 
McGregor,  J.  H„  235, 240 
McGuinness,  D.,  641 , 645 
McGuire,  T.,  410,  421,422 
McHaffie,  D.,  565 
Mclnerney,  D.,  421 
Mclnerney,  V.,  409,  421 
McIsaac,M.S.,411, 423 
McKeague,  C.  A.,  1 14,  1 19, 124 
McKendree,  J.,  126,  128,  137, 142, 
478,450 

McKenney,  S.,  303,  739,  744,  745, 
754,  759,  760,  767 
McKeown,  M.  G.,  544,  545, 562 
McKinley,  D.,  603 


McLaren,  B.  M.,  140, 141,143, 480, 
483 

McLaughlin,  M„  624, 625,626, 627, 
629,630,637,656,657 
McLean,  L„  391, 397 
McLearn,  R.,  422 
McLeod,  P.,41, 52 
McLoughlin,  C.,  434,  438 
McMahon,  C.,  524 
McManus,  B.,  669 
McManus,  P.,  520, 524 
McMillan,  M.,  507 
McNally,  M.  M„  507 
McNamara,  D.  S.,  204,  205, 205, 
209,217,218,222,223,437 
McNichoLS.,  533, 535 
McParland,  M.,  490, 503 
McPherson,  J.,  439,  527 
McPherson,  K.,  535 
McQuaiLD.,  51, 52 
McTighe,  J.,  386,  399, 487, 505 
McWhorter,  J.,  635,639,641, 642, 645 
Means,  B„  146,760,676 
Meckling,  W.H.,  242,245 
Medsker,K.L„  583, 592 
Meeter,  M.,  96 
Megarity,  J.,  614,  677 
Mehrabian,  A.,  407, 427, 500,  505 
Mehrens,  W.A.,  770,  503 
Meikle,J.L„  86,  96 
Meira,L„  547, 564 
Meire,  M.,  353 
Meirhenry,  W.  C.,  190,796 
Meister.C.,  223, 432, 439 
Mejias,  U.  A.,  348, 353 
Melaragno,  R.  J.,  191, 795,  796 
Melendez,  R.,  342 
Melis,  E.,  139,742 
Mellar,H.,  331, 343 
Meloy,  J.  M.,  775,  503 
Meltzer,  A.,  507 
Menand,  L.,  58,  65 
Mendenhall,  A.,  177,  182,  754 
MengeLN.S.,  390, 397 
Merbitz,  C.,  63,  65 
Merbitz,  N.  H.,  65 
Mercer,  D.,  256 
Merchand,  J.  E.,  487, 504 
Mergendoller,  J.  R.,  489, 503, 504 
Merriam,  S.  B.,  436, 435 
Merrienboer,  J.  G.,  25,  86,  96,  790 
Merrill,  D.  C„  126,  128,  132, 137, 
742 


Merrill,  J„  132,  742 
Merrill,  M.  D„  16,  79,  24,  25, 27, 
146,760,  174, 177,  178,  179, 
180, 181, 182,754,212,223, 
347,348,353,386,387,388, 
390,392,393,394 ,396,397, 
398, 446, 447,  453, 455, 473, 
452,583,587,590,593,636, 
644,  645,  662,  665,  669,  673, 
676,654,707,708,709,773, 
784,503,810,575 
Merrill,  S.K.,  742 
Merrow,  J.,  300,  303 
Merryfield,  M.  M„  229, 237 
Merten,  C.,  285, 292, 293 
Messick,  S„  599,  604,  734,  759 
Mestre,  J.  P.,  563 
Metcalf,  S.J.,  734,  759 
Metcalfe,  J„  219, 223 
Mevarech,  Z„  409, 41 1 , 413, 427 
Meyer,  D.  K.,  219, 223, 433, 435 
Meyer,  L.  H„  50,  53 
Meyer,  S.R.,  191,796 
Mezirow,  J.,  35,  37, 37 
Michael,  J.  A.,  492, 507 
Michaelsen,  L.  K.,  777,  503 
Michalchik,  V.,  676 
Michel,  C.,  264, 267 
Michiuye,  J.  K.,  603 
Middle  School  Mathematics 

Through  Applications  Project 
Group,  57 

Middlebrooks,  S.  H.,  709 
Midgett,  C.,  505 

Miflin,  B.  M.,  493, 495, 504, 506 
Mikkila-Erdmann,  M.,  503 
Mikulecky,  L.,  709 
Miles,  C.L.,  386, 397 
Miles,  M.  B„  775,  781, 503,  505 
Milheim,  W.,  410, 427 
Milkovich,  J.  G.,  52 
Miller,  G.  A.,  41, 53,  307, 374,  373, 
357 

Miller,  H.R.,  191,796 
Miller,  K„  479 
Miller,  N„  37,  37 
Miller,  R.B.,  636,  645 
Millis.K.K.,  216,  223 
Millroy,  W.  L.,  65 
Mills,  C.,  502 
Mills,  G.,  754 

Mills,  R.,  209,  261, 265,  644 
Milrad ,  M .,  334 ,  34 1 , 343 ,  809 ,  575 


849 


Author  Index 


Milter,  R.G.,  497,  505 
Mink,  B.  P.,  631 
Mink,  O.G.,  621, 631 
Minton,  H.L.,  57,  65 
Mintzes,  J.  J.,  770 
Mioduser,  D.,  146, 160 
Miserandino,  M.,  50,  53 
Mishra,  P.,216, 225 
Mislevy,  R.  J.,  280,  286, 295,  598, 
600,  601, 604 
Mitcham,  C.,  356, 364 
Mitchell,  H„  222 
Mitchell,  J.,  229, 231 
Mitchell,  S.  M„  487, 488, 490, 497, 
500, 504,  561 , 562 
Mitrovic,  A.,  139, 142,  216, 225, 
287,288 ,295 
Miyagishima,  B.  K.,  259 
Moallem,  M.,  167, 171, 263, 267, 
772,  774, 803 
Mockus,  A.,  357, 364 
Modritscher,  F.,  292 
Moffitt,  M.  C„  503 
Mohammed,  S .,  78 1 , 802 
Mohanty,  J.,  11,20 
Moher,  T.  G.,  564 
Mohide.E.  A.,  497, 502 
Mol,  A.,  30,  32 

Molenda,  M.,  5,  17, 19, 20, 266 
Molinari  A.,  208 
Monahan,  T.,  252, 256 
Monk,}.,  342 
Montague,  E.  W.,  481 
Mookerjee,V.S.,  652,  657 
Moonen,  J.,  669 

Moore,  A.,  313, 475, 477, 482,  656 

Moore,  D.  J.,  326 

Moore,  D.  M.,  79,  64,  81, 196, 534 

Moore,  J.  C.,  270,  275 

Moore,  J.  D.,  224 

Moore,  J.L.,  229, 257 

Moore,  K.  A.,  438 

Moore,  M.  G.,  261, 267 

Moos,  D.C.,  201, 209 

Moran,  D.  J.,  63,  65 

Moran,  T.,  227 

Moreland,  R.  L„  776,  802 

Moreno,  K.,  222 

Moreno,  R„  43,  52,  137,  139, 142, 
759,214,217,218,225,307, 
314,324,327 
Morgan,  C.,  229, 257 
Morgan,  D.  L.,  774,  803 
Morgan,  Jr.,  B.  B„  648,  651, 658 


Morgan,  M.  T.,  140 
Morley,  M.,  602 
Morris,  E.,  347,  353 
Morris,  E.  K„  56,  57,  59,  60,  62,  64, 
65,66 

Morris,  J.,  70S,  534 
Morris,  L.,  65 1 , 658 
Morris,  L.  L.,  769,  803 
Morrison,  G.  R„  60,  66, 123, 195, 
473,455,707,  772,775,720, 
721.723,728,729,731,733, 
757,  760,  775,  804 
Morrison,  J.  B.,  54,  96, 224 
Mory,E.H„  126, 127, 130, 131, 133, 
134,  136,  137,  138,742,745, 
114,115,804 
Moseley,  J.L.,  302, 599 
Mosenthal,  P.,  96 
Moskal,  P.  D.,  275 
Mostow,  J.,  742 
Motschnig-Pitrik,  R.,  361 , 363 
Moust,  J.  H.  C.,  487, 494, 495, 504 
Movellan,  J.,  227 
Moxley,  R.  A.,  62, 66, 195 
Muehlenbrock,  M.,  483 
Muijtjens,  A.  M.  M.,  802 
Muilenburg,  L.  Y .,230 
Muirhead,B.,  41 1,427 
Muldner,  K.,  292 
Mulhern,  J.,  208 
Muller,  A.,412,  427 
Muller,  C.,  742 
Muller,  M.  J.,  661, 669 
Muller,  S.,  487, 504 
Mullin,T.M„  580,  593 
Mumford,  A.,  306,  309,  310,311, 
314 

Mumford,  E.,  661 , 669 
Munneke,  L.,  414, 427 
Munoz,  D.,  327 
Munro,  A.,  467, 155,760 
Muraida,  D.  J.,  28,  393, 397,  398, 
676,  684,  709,  713 
Murnaghan,  J.,  438, 504 
Murphy,  D„  685,  689,  690 
Murphy,  K„  352 
Murray,  J.,  105,  106,  709 
Murray,  R.  C.,  139,742 
Murray,  T„  220, 223, 292 
Murre,  J.  M.  J.,  96 
Muste,  M.,  224 
Mwanza,  D.,  364,  528, 535 
Myers,  A.  C.,  492, 507 
Myllyaho.M.,  780, 803 


N 

Nachmias,  R.,  760 
Nadolski,  R.  J.,  453, 455,  555, 564 
Nagasaka,  T.,  724 
Nagata,  N.,  130,  138,742 
Naidoo,G.,  265, 267 
Naidu,  S.,  261, 262,  263, 264,  265, 
266, 267 
Nairn,  R.,  507 
Najjar,  J.,  349,  353 
Najjar,  L.  J.,  48, 53 
Nakagawa,  K.,  108 
Nakano,  Y.,  292 
Nakayama,  H.,  438 
Nankivell,  C.,  535 
Nantel,R„  672,  684 
Narayan,  N.  H.,  759 
Narayanan,  S.,  293 
Narciss,  S.,  126,  128,  130, 131,  132, 
135,  136,  137,  138,  139,  140, 
742 

Nardi,B.  A.,  25, 27 
Nash.P.P.,  507, 505 
Nash,  R„  500,  554 
Nason,  R„  434, 438 
Nass,C.  I.  217,225,417,422 
Nassif,  S.,  645 

Nastasi,  B.,  410, 413, 479, 427 
Nath,  L.  R.,  169,777 
Nathan,  M.  J.,  99,  709,  602,  603 
Nathan,  R.,  535 

National  Council  for  Measurement  in 
Education  (NCME),  602 
National  Education  Association 
(NEA),  234, 239,  528, 535 
National  Institute  of  Child  Health 
and  Human  Development 
(NICHD),  544,545,564 
National  Research  Council  (NRC), 

459.467 

National  Science  Foundation  (NSF), 

247.248.457.458.467 
National  Telecommunications  and 

Information  Administration 
(NTIA),  528, 535 
Naylor,  J.  C.,  443, 454, 455 
Neal,  A.,  785,  806 
Neale,  D.  C„  668 
Neidermeyer,  F.,  191, 796 
Neisser  U.,  94, 96 
NeitzehC.,  433, 438 
Nejdl,  W„  286, 293, 415,411, 478, 
481,482 


850 


Author  Index 


Nelson,  B.,  466 

Nelson,  D.  W„  53,  119,  121 ,124 
Nelson,  H.  G.,  685, 690 ,  693,694, 

695.696 .701 
Nelson,  J.,  353,  643,  644 
Nelson,  L.  M„  323, 327,  384, 397, 

631.701 

Nelson,  T„  346,  353 
Nelson,  W.  A.,  360, 364,  394, 398, 
773,748,756,  760 
Nemeth,  E„  324, 327 
Nendaz,  M.  R„  495,496, 504,  505 
Nesbit,  J.  C.,  343 
Neufeld,  V.  R.,  495,504 
Neuman,  D„  234,  235,  238, 240 
Neumann,  U.,  294 
Neven,  F.,  353 
Neves,  D.M.,  584 ,593 
Newby,  T.  J.,  69,  71, 73,80,  81 
Newell,  A.,  41 , 53, 221, 242, 248, 
491 , 504, 785,  803 
Newhagen,  J.,  231 
Newlin,  M.  H.,  237 
Newman,  M.,  488,  504 
Newman,  S.  E„  81,109,437,644, 711 
Newmann,  F.  M.,  625,  626,  631 
Newnham,  R.,  532 ,535 
Newsome,  S.  L.,  639,  645 
Newton,  D„  270, 275 
Newton,  J.M.,  191,795 
Nguyen,  F„  380, 5 19, 520, 521 , 523, 
524,155,760 
Nicholas,  J.R.,  388, 397 
Nichols,  D.  G.,  794 
Nichols,  R.,  655,  658 
Nicholson,  S.  A.,  229, 237 
Nicolaidou,  I.,  658 
Niederhauser,  D.  S.,  200,  201. 203, 
204,  208, 209, 479, 483 
Niedermeyer,  F.,  761 
Nielsen,  J.,  521, 524 
Nielsen,  T.,  524 

Niemi,  D„  599,  600,  602,  603,  604 
Nieveen, N„  303, 746,752, 759,  760, 
761 

Nievelstein,  F.,  554, 564 
Nigam,  M„  176, 183, 372, 380, 465, 
"  466 

Nijhawan,  H.,  220, 227 
Nijhof,  W.  J.,  523 
Niles,  J.  A.,  723, 395 
Nill,  A.,  481 
Nilsson,  J.,  534 
Nirenburg,  S.,  641, 645 


Nisbett.R.E.,  661, 669 
Nitko,  A.,  770,  803 
Njoo,  M.,  460, 466 
Noble,  L„  503 
Noel.E.  S. ,13, 20 
Noel,  F.  W.,  13,20 
Noell,  J.,  409, 422 
Noldus,  L.  P.  J.  J.,  788,  803 
Nonaka,  I.,  242, 248 
Norcross,  J.  C.,  629, 637 
Norman,  D.  A.,  27, 46,  53,  87,  96, 
212,218,223,254,256,348, 
353,662,669 

Norman,  G.  R„  326, 490, 491 , 492, 
504,  506, 565 

Northrup,  P.  T„  528,  535,  664,  669 
Norvig,  P.,  242, 249 
Novak,  J.  D„  110,  220, 461 , 467, 
600,  604 

Noy,  N.,  641, 645 
Niickles,M.,  556, 563, 564 
Nugent,  G.  G.,  43, 53 
Nunes,  P.,  549, 564 
Nungester,  R.  J.,  603 
Nurmi,  S .,803 
Nussbaum,  E.  M.,  229, 237 

o 

O’Brien,  M.,  500, 534 
O’Connor,  D.  L.,  778,  781, 782, 803 
O’Connor,  S.,  227, 237 
O’Dell,  J.  K„  760 

O’Donnell,  A.  M .,420, 422,121, 759 
O’Donohue,  W.,  62,  63,  66 
O’Keeffe,  I.,  348,  353 
O’Mahony.S.,  41 1,422 
O’Malley,  C„  437 
O’Neal,  M.  R.,  771, 803 
O’Neil,  H.  F.,  212,  213, 223,  111, 
778,  779,  780,  781, 802,  803 
O’Neil,  Jr.,  H.F.,  321, 325, 598, 601, 
602,  603, 604 
O’Neill,  D.K.,  167,777 
O’Neill,  P.  A.,  491,504,805 
O’Neill,  S.,  298 ,303 
O’Shea,  T  .,342,423 
O’Sullivan,  P.B.,  229, 237 
O’Sullivan,  P.  S.,  503 
O’Toole,  J.M.,  271, 276 
Oates,  J.,  484,  564 
Oberlander,  J.,  487, 504 
Oblinger,  D.  G.,  212,275,  296,  299, 
303 


Oblinger,  J.  L„  272, 275,  296,  299, 
303 

Ocker,  R.  J.,  409, 422,  653,  654, 658 

Odom,  S.  L..  62,  66 

Oeseburg,  B.,  505 

Ofiesh,  G.  D.,  190,796 

Ogan,  A.,  479 

Ogborn,  J,  461 , 467 

Oh,  M.  S„  507 

Ohlsson,  S„  474, 482,  549,  550, 564 
Ohrazda,  C.,  28,  192,  194,  797,  709, 
773 

Okamoto,  T.,  476,  482 
Okey,  J.  R.,  388, 397, 409, 422,  669 
Olansen,  J.  B.,  799,  803 
Olde.B.  A.,217,222 
Oldridge,  R.,  248 
Olfman.L.,  237, 535 
Olian,  J.  D.,  657 
Oliver,  K„  535,  536 
Oliver,  M„  270, 275 
Oliver,  R„  759, 171,266, 434, 438, 
760 

Olivier,  B.,  361, 364 
Olkinuora,  E.,  773, 803 
Olkun.S.,  337,  338, 344 
Olney,  A„  222, 224 
Olsen,  B.,  622,  624,  626,  627,  637 
Olson,  D.R.,  612,  677 
Omanson,  R.  C.,  562 
Oppermann,  R.,  374, 475, 482, 483 
Opwis,  K.,  53 
Or-Bach,  R.,  479, 482 
Oren,  A.,  760 
Oriogun,  P.  K.,  229, 237 
Orion,  N„  153,760 
Orrill.C.H.,  498,  500, 504 
Orton,  J.  D.,  628,  637 
Orton,  P.Z.,  270, 275 
Ortony,  A.,  113,  724 
Osborne,  C.  E„  492, 497, 507 
Osborne,  R„  1 12,  723,  392, 397 
Oser,  R.  L.,  325 
Osgood,  C.E.,  445, 455 
Osgood,  M.  P.,  487, 504 
Osgood,  R.  E.,  455 
Osguthorpe,  R.  T.,  796,  270, 276, 
690 

Oshima,  J.,  434, 438 
Oshima,  R.,  434, 438 
Oshry,  B.,  637 
Osland,  J.,  35, 37 
Osmundson,  E.,  600,  603,  604 
Otero,  J.,  214, 223 


851 


Author  Index 


Otte,G„  271, 276 
Overman,  L.  T.,  757 
Overton,  W.  F.,  57, 66 
Owen,  K.Q.,  631 
Owens,  D.L.,  707,  709,  772 
Owensby,  J.  N.,  160,  535 
Owston,  R.  D„  613,  614,  615,  616, 
617 

Oxman,  R.  E.,  687,  690 
Ozuru,  Y.,214 ,225 

P 

Paas,  F.G.,  43, 53, 54,  88,  96, 143, 
204, 209,  210,  374,  375, 381, 
455, 456, 564,  565,  721, 760, 
785, 786, 789, 803, 805 
Paclwaskyj,  T.,  65 
Pacurar,  E.  G.,  362, 364 
PADI  Research  Group,  604 
Paechter,  C.,  253, 256 
Paechter,  M .,276 
Page,  E.  B.,  599,  604 
Page,  S.  H.,  603 
Pain,  H.,  219, 227, 478,  481 
Paine,  M.  E.,  501 

Paivio,  A.,  40, 42, 53, 205, 209, 214, 
223,301, 313,121, 761 
Pajares,  F.,  134, 143 
Palen,  L.,  245, 248 
Palincsar,  A.  S„  102, 108, 109,  228, 
230, 432, 437,  438 
Pallant,  A.,  156 ,160 
Palmer,  P.  J.,  28 
Panaoura,  A.,  309, 314 
Panyametheekul,  S.,  227, 231 
Paolini,  P.,635,  643 
Papachristou,  D.,  293 
Papaevripidou,  M.,  462, 467 
Papanikolaou,  K.  A.,  141, 312, 314 
Papert,  S„  28,  334,  336,  341 , 343, 
415, 422, 461 , 466 
Paquette,  G.,  358, 362, 364, 639, 645, 
684 

Paradise,  N.  E.,  396 
Pare-Blagoev,  J.,  788,  802 
Park,  H.,  208 
Park,  I.,  89,  90, 95,  96 
Park,  J.,  657 

Park,  O.,  279, 293, 423, 470, 471 , 
473, 474, 475, 476, 482, 483, 
722, 760 
Park,  S.H.,  52 
Parker,  B.  J.,  51 , 53 


Parker,  F.  L.,  565 
Parker,  L.  W„  8, 20 
Parkhurst,  P.  E.,  389, 396 
Parlett,M„  261, 268 
Parnas,  D.,  248 
Parra,  M.,  690 

Parrish,  P„  350,  353,  686,  690 
Parziale,  J.,  552, 563 
Pasca,  M.  A.,  222 
Pask,G„  640,  645 
Passmore,  C.  M.,  327 
Pasta,  D.  J.,  325 
Patel  Stevens,  L.,  432, 437 
Patel,  L.,  561, 564 
Patel,  R.,  669 
Patel,  V.K.,  491 ,504 
Paterson,  B.,  779,  803 
Patrick,  B.,  50,  53 
Patrick,  J.,  560, 564 
Patterson,  H.  L.,  603 
Patterson,  M.,  592 
Patterson,  N.,  206, 209 
Patton,  M.  Q.,  607,  611,612,  613, 
617, 119,  780,  781, 803 
Paul,  P„  787,  802 
Pauli,  C.,  415 ,422 
Paulsen,  M.  F„  770,  803 
Paulus,T„  264, 267 
Pavio,  A.,  88,  96 
Pavitt,  C.,  779,  803 
Pavlov,  I.  P.,  192 ,196 
Payne,  S.  C.,  327 
Pazzani,  M.  J.,  294 
PDP  Research  Group,  52 
Pea,  P.,  160 

Pea,  R.  D„  159,  164, 171, 264, 267, 
330,  343, 409, 415, 422, 432, 
438, 133,760 
Pear,  J.J.,  63,  65, 431,438 
Pearce,  C.  L.,  657 
Pearl,  J.,  284, 293 
Peck,  A.  C„  451, 455 
Peck,  K.L.,  630, 631 
Pedersen,  S.,  759 
Pederson,  P.  B.,  321, 327 
Peek,  J.,  137,742 
Pelgrum,  W.  J.,  16, 20 
Pellegrino,  J.,  108, 534,  597,  599, 
600,  604 

Pellegrino,  J.  W.  E„  27, 214, 222, 
657,  654,  657,  658 
Pelletier,  R„  140, 220, 292,  643 
Pellom,  B.,  227 
Pelto,G.H„  771, 803 


Pelto.P.  J.,  771, 803 
Pena,  C.  I.,  312, 373, 374 
Penner,  D.  E„  46, 53, 458, 465, 467 
Penney,  C.  G.,  43, 53 
Pennington,  T.,  227, 237 
Pennypacker,  H.,  65 
Peper,  R.  J.,  1 14,  1 18,  724,  389, 397 
Pepper,  S.  C.,  55,56,  57,  58,  59,  64, 
66 

Perani,  D.,  381 
Perencevich,  K.  C.,  54, 184 
Perez, R.S., 212, 213 ,223,482,483, 
742,743,750,756,760 
Perez-Prado,  A.,  774,  803 
Perie.M.,  542, 564 
Perkins,  D.  N„  255, 256, 443, 455, 
761,111,804 
Perlmutter,  M.,  412, 427 
Perraton,  H.,  261, 267 
Perrenet,  J.  C.,  504 
Perret-Clermont,  A.-N.,  549, 564 
Perrewe,  P.  L.,  431, 439 
Perrin,  D.  G.,  79 
Perry,  J.D.,  78,  87, 534,  565 
Perry,  T.,  479 

Pershing,  J.  A.,  509,  5 13,  524 
Persky,  H.,  602 

Person,  N.  K.,  214, 217, 222, 292 

Person,  P.,  222 

Pesin,  L.,  476, 480 

Petajan,  E.,  222 

Peters,  O.,  261, 267 

Peters.  L.,  52 

Petersen,  N.  S.,  599,  604 

Peterson,  L.  P.,  472, 482 

Peterson,  M.,  343 

Peterson,  R.  W.,  27 

Peterson,  S.  J.,  510,  518,  522, 524 

Petkovich,  M.  D.,  728,  760 

Petraglia,  J„  100, 109,  322, 327 

Petrosino,  A.,  677, 656 

Petroski,  H.,  6, 20 

Petrucelli,  R.  J.,  556, 564 

Petrusa,  E.  R.,  564 

Pettigrew,  T.  F.,  46,  53 

Peverley,  S.,  123 

Pfeffer,  J.,  592 

Pfister,  M.,  110 

Phelps,  R.,  32 

Philippou,  G.,  309, 374 

Phillips,  B.,  274 

Phillips,  D.  C.,  60,  66,  69,  74,  82, 
767 

Phillips,  T.L.,  396 


852 


Author  Index 


Phye,  G.  D.,  138,  139,742 
Piaget,  1.,  23,  24, 28, 49, 53,  206, 
209,405,422 

Picard,  R.  W„  219, 221,223, 280, 
285, 293 
Pickard,  P.,274 
Pickering,  D.  J.,  183 
Pieters,  J.  M„  386, 397,  742,  760 
Pilkington,  R.,  422 
Pilot,  A  .,562 
Pinkwart,  N.,  335,  343 
Pinto  Pereira,  L.,  500 
Pintrich,  P.  R„  121 , 124,  134, 140, 
142, 141,160, 206,207,209 
Pirnay-Dummer,  P.  N.,  53, 181,803 
Pirolli,  P„  756,  758 
Pitts,  J.M.,  235, 240 
Pitts,  K„  617 
Pivec,  M.,  292 
Pizzini,  Q .,467 
Plaisant,C.,  218, 224 
Plake,  B.S.  ,802 
Plank,  R.  E.,  51, 55 
Plass,  J.  L.,  800 
Plato,  71, 82 
Platt,  J.  R.,  44, 53 
Plaza,  E.,  553, 562 
Pleban,  R.,  321 , 327 
Pletinckx,  J.,  487,  504 
Ploetzner,  R.,  96, 342 
Ploger.D.,  462,  467 
Plomp,  T.,  16, 20, 28,  761 
Plucker,  J.  A.,  75,  80, 99, 108,  254, 
255 

Plummer,  K.  H„  751, 760 
Plunkett,  K.,  52 
Podolskij,  A.  I.,  386, 397 
Poindexter,  S.,  445, 455 
Pokorny,  R.  A.,  592 
Polanco,  R.,  489, 504 
Polanyi,  M.,  638,  645 
Pollatsek,  A.,  542, 565 
Pollio.H.R.,  445, 455 
Pollitis,  P.,  466 

Pollock,  E.,  374,  379, 381, 452, 455 

Pollock,  J.C.,  732,  760 

Pollock,  J.  E.,  183 

Poison,  M.  C.,  28,  713 

Poison,  P.,  380 

Pombortsis,  A.,  801 

Pomplun,  M.,  801 

Ponte,  I.,  342 

Pontificia  Universidad  Catolica  del 
Peru  (PUCP),  488, 504 


Poole,  G„  609,  676 
Poole,  M„  620,657 
Popescu,  O.,  479, 479 
Popham,  J.  W.,  770,  803 
Popham,  W.  J.,  598,  604 
Popper,  K.,  488,  504 
Porter,  D„  190,  191,795,796 
Posavec,  E.,  28 
Posner,  G.  J.,  331, 344 
Possey,  L.  O.,  27 
Postman,  N.,  649,  658 
Postthast,  M.,  409, 422 
Potamianos,  A.,  280, 293 
Potelle,H„  204, 209 
Potter,  C.S.,  265, 267 
Potter,  J.,  640,  645 
Potter,  M.  C.,  44, 53 
Potts,  L.,  796 
Powers,  D.  E.,  601, 604 
Poysa,  J.,  14.79,252,256 
Pratt,  K„  625,  626,  631 
Prawat,  R.  S.,  57,  58,  60,  61, 66,  69, 
71 , 72,  74,  75,  76,  82 
Preacher,  K.  J.,  438 
Preece,  J.,  245, 248,  649,  658 
Preese,  F„  673,  682,  684,  749,  760 
Preis,  A.,  292 
Preissle,  J.,  65 

Prensky ,  M .,  263 , 267, 272 , 276, 320 , 
327,  570,571,574,575 
Pressey,  S.  L.,  188,  189,796 
Pressley,  M„  133, 143,  389, 397, 
121,761 
Price,  D.  A.,  504 
Prichard,  J.S.,779,  780,  803 
Pridemore,  D.  R.,  143,  773,  802 
Prince,  K.  J.  A.  H„  490, 504,  560, 
564 

Prins,  F.  J.,  564 
Privman,  M.,  96 
Prochaska,  J.  O.,  629,  631 
Proctor,  R.  W.,  60,  64 
Programme  for  International  Student 
Assessment  (PISA),  541, 564 
Project  LISTEN,  286, 293 
Prosser,  M.,  255, 256 
Pruitt,  J.,  662,  669 
Prusak.  L„  242, 248,  407, 422 
Psillos,  D.,  807 
Psotka,  J.,  285 ,294, 474, 483 
Pugh,  R.  C.,  668 

Puntambekar,  S.,  777 , 203, 209, 432, 
434, 438,  503 
Purdy,  R.  A.,  502 


Q 

Qin,  J.,  349, 353 
Qu,  L.,  280, 293 
Quann,  V .,677 
Quardt,  D.,  602 
Quigley,  E.  J.,  242, 248 
Quilici,  J.,  759 
Quina,  J.,  57,  66 
Quinby,  P.  M.,  500 
Quinlan,  K.  B.,  230 
Quinn,  J.,  27, 620, 621 , 628, 649, 657 
Quinn,  R.  E.,  631 
Quintana,  C.,  146,  760, 459, 465, 
467, 478,  482,  654,  658 
Qureshi,  S.,  780,  781, 803 

R 

Rabaey,  J.,  639,  644 
Radinsky,  J.,  100,  709 
Rafaeli,  S.,  231 

Ragan,  T.  J.,  28,  126, 128, 136, 143, 
^  194,797,386,387,389, 
391-392,394 ,397,398 
Raghavan,  K„  294, 322, 327 
Ragone,  A.,  352 
Railsback.S.F.,  334, 343 
Raines,  C.,  300,  303 
Rainie,  L.,  527, 535 
Rajasingham,  L.,  770,  805 
Rakow,  E.  A.,  476, 481,483, 132, 
760 

Ram,  P.  M.,  802 
Ram,  S.,  657 
Ramamurthy,  U.,  52 
Raman,  M.,  228, 231, 527, 535 
Rambow,  R.,  563 
Ramey,  C.T.,  543, 564,  565 
Ramey,  J.,  787,  800 
Ramharter,  H.,  795 
Rankin,  R.  J.,  137,743 
Ranney,  M.,  742 
Rapaport,  M.,  264, 267 
Rasch.  M.,  95,  96,  387,  393,  398 
Rashev,  R.,  314 
Rashotte,  C.  A.,  565 
Rashotte,  J.  K.,  566 
Raskin,  J.,  667,  669 
Raskin,  V.,  641, 645 
Rasku-Puttonen,  H.,  433, 438 
Rasmussen,  C.  M.,  503 
Rathbun.G.  A.,  665,  669 
Raths,  J.,  740 


853 


Author  Index 


Rauh,  V.  A.,  543, 565 
Raven,  B.H.,  621, 628,  637 
Ravenscroft,  A.  ,237 ,421 
Ravitz,  J,  609,  677 
Ray,  W.E.,  392, 397 
Raybould,B„  519, 524 
Raymond,  E.  S.,  357,  361, 364 
Rayner,  K„  285, 293,  542, 565,  786, 
804 

Read,  J„  661, 663,  669 
Reasons,  S.  G.,  271 , 276 
Reber,  A.  S.,  70,  71, 72, 82 
Rebolledo  Mendez,  G.,  478, 482 
Recesso,  P.,  535 
Rechenberger,  K.,  483 
Redding,  R.E.,  583,  587,  593 
Reder,  L.  M.,  170,  800 
Redfern,  S.,  629,  631 
Redish,  J.C.,  662,  665,  669 
Reed,  E.S.,  98, 109 
Reed,  M.  W„  123 
Reed,  R.  S.,  70,  82 
Reed,  S.  K.,  42,  48,  53 
Rees,  E.,  380 

Reese,  H.  W„  57,  59,  60,  65,  66 
Reeve,  J.,  324, 327 
Reeves,  B„  217, 223 , 417, 422, 481 
Reeves,  T.,  266,  746,  760 
Reeves,  T.  C„  159,  167,  168, 171, 
331,343,654.658,665,669, 
707,710,775 
Reezigt,  G.  J.,  437 
Regehr,  G.,  495,  504 
Regian,J.  W.,291 ,294 
Regin,  G.,  409,  422 
Rehak,D„  220, 227, 223 
Reiber,L.  P.,95, 389, 397 
Reid,  D„  50, 53 
Reigeluth,  C.  E.,  362, 364 
Reigeluth,  C.  H„  390,  397 
Reigeluth,  C.  M„  24,  25, 28,  76,  80, 
82, 146, 760, 174, 184, 387,389, 
391,392,394 ,396,397, 398, 
410,  422, 448, 452,  454, 455, 
456,570,571,574,575,580, 
593,621,624,631,640,645, 
649,  658,  662,  664,  665,  670, 
692,693,695,696,697,707, 
706,707,708,709,  773,732, 
758,  768,  804,  810,  875 
Reijnen,  W.,  43, 53 
Reilly,  B„  772,  804 
Reilly,  R  .,223 
Reimann,  G.,  142 


Reimann,  P.,  201, 202,  210, 480 
Reingold,  E.  M.,  801 
Reiser,  A.  R„  470, 471 ,483 
Reiser,  B.J.,  742, 146, 160,433,438, 
467,  658 

Reiser,  R.  A.,  63,  66,  194, 196, 453, 
455,740,755,760, 774,775, 
804 

Reisman,  K.,  462, 468 
Reiter,  S.,  630 
Remidez,  J.,  434, 438 
Renkl,  A.,  96,  311,381,564 
Renner,  K.E.,  137,742 
Renshaw,  P.,437 
Rentsch,  J.  R.,  782, 804 
Repenning,  A.,  336, 344 
Repman,  J.,  409, 422 
Repo,  S„  415, 427 
Resing,  W.C.M.,  552, 565 
Resnick,  L.  B.,28,81, 230, 264, 267, 
388,389 ,395, 398, 399, 412,481 
Resnick,M. ,331, 334,344,461, 462, 
466,467,641,645 
Resnick,  P.,  246, 248 
Retalis,  S.,  480 
Reusser,  K.,  415, 422 
Reynolds,  C.W.,  334, 344 
Reynolds,  F.,  487, 504 
Reynolds,  K.M,  230, 237 
Reynolds,  R.  E.,  209 
Reynolds,  S.  B.,  603 
Rezabek,  L.,  20 
Reznich.C.B.,  498,  504 
Reznick,  R.,  504 
Rheinberg,  F.,  134,  743 
Rheinfrank,  J.,  635,  645 
Rhodes,  R.,  650,  658 
Ribiere,  V.  M.,  244, 248 
Riccardi,  G.,  293 
Ricci,  K„  321, 327 
Rice,  E.,  772,  773,  774,  802 
Rice,  J.  M.,  134,743 
Rice,  M.,  534 
Rich,  C„  284, 292 
Rich,  E„  294 
Richards,  D.,  246,  249 
Richards,  G„  263, 267, 364 
Richards,  L.,  788,  800 
Richards,  P.,421 
Richards,  R.,  353,  644 
Richards,  S.,  427 
Richardson,  J.  C.,  264, 267 
Richardson,  W„  227, 237 
Richards-Ward,  L.  A.,  307,  374 


Richey,  R.C.,  28,360, 364, 389, 394, 
398,510,511,512,514,575, 
640,  645,  665, 669,  707,  708, 
773,739,740,741,743,745, 
748,749,750,753,755,756, 
759,  760,  767, 813,  814,  875 
Richmond,  B„  334,  344,  733,  760 
Rickards,  J.  P„  114,724 
Rickel,  J.  W„  222, 287, 293,294, 644 
Riconscente,  M.  M.,  598,  600,  604 
Riddell,  J.C.,  795 
Rieber,L.  P.,215, 223, 410 
Riel,  M„  608,  677 
Rieman,  J.,  481 
Riera,  A.,  363 
Riffell,S.K.,  271, 276 
Rigney,  J.  W.,  191, 196,262, 267 
Riley,  D„  330,  344 
Riley,  G.D.,  242, 248 
Riley,  S.L.,  507 
Rinke,W.J„  444, 455 
Ripkey,  D.  R.,  603 
Ripley,  D.  E.,  27 
Risko,  V.  J.,  53 

RIT  Online  Learning  Department, 
212,276 

Ritchie,  D„  116,498,503 
Rittle-Johnson,  B.,  136,  743 
Rizk,D.E.E„  558, 563 
Rizzo,  R.,  476, 477,  480 
Rizzolatti,  G.,  381 
Robb,  L„  487, 503 
Robbs,  R.  S„  501 
Roberts,  J.,  253, 256 
Roberts,  K.,  805 
Roberts,  N„  341, 342 
Roberts,  R.  C.,  28,  815 
Robin,  A.  R„  192,  796 
Robinson,  D.H.,  719,  759 
Robinson,  E.,  505 
Robinson,  E.  R.  N.,  390, 398 
Robinson,  R.,  15, 20 
Robinson,  R.  A.,  271 , 272, 275 
Robinson,  R.  S.,  61, 66,  767,  765, 
767,768,772,773,774,776, 
804 

Robison,  R.,  270, 271, 273, 275, 276 
Robson,  R.,  227 
Roche,  B.,  62,  65,  66 
Rock,  B.  N.,  105,709,770 
Rocklin,  T„  409, 41 1 ,420,422 
Roe,  A.  A.,  190,796 
Roehler,  L.  R.,  432, 438 
Roenker,  D.,  321, 327 


854 


Author  Index 


Roesler,  A.,  657 

Roessingh,  J.  J.  M„  321 , 327 , 452, 
455 

Rogers,  C.  A.,  390,  391 ,398 
Rogers,  E.  M.,  356, 364,  631,656, 

" 658 

Rogers,  P.  C„  622,  628,  629,  635, 
636,638 ,644 
Rogers,  P.  J.,  607,  61 7 
Rogers,  S.  E.,  93, 96 
Rogers,  Y.,  658 
Rogerson,  L.  G.,  701 
Rogoff,  B.,  101,770,254,256,427, 
" 428, 433, 439,  734 ,760 
Rogonzenski,  L.  T.,  123 
Rohrer-Murphy  ,  L .  C . ,  747 , 707 , 709 , 
772 

Rojahn,  K.,  46,  53 

Roll,  I.,  134, 140, 143, 478, 480,  483 
Rolls,  E.T.,  52 
Romero,  C„  139,743 
Romiszowski,  A.  J.,  27,  226, 237, 
261 ,267, 706,712 
Ronen,  M.,  154, 160 
Ronen-Fuhrmann,  T.,  160 
Ronnkvist,  A.,  16, 18 
Rorer,  L.,  592 
Rorty,  R.,  72, 82,  628 
Roschelle,  J.,  146, 160,  165,  166, 
171,286,294 
Roscoe,  R.  D.,  554,  563 
Rose,  C.  P.,  224 
Rose,  E.,  565 
Roselli,  T.,  353 
Rosenberg,  S.,  635,  645 
Rosenblatt,  L.  M.,  105,  770 
Rosenblatt,  R.  D.,  483 
Rosenblatt,  T.,  274 
Rosenbloom,  P.,  785, 803 
Rosenshine,  B.,  176, 784, 217, 223, 
432 ,439 

Rosenthal,  H.,  502 
Roskos,  K.,  761 
Rosnow,  R.  L.,  59,  66 
Rosow,  E.,  799,  803 
Ross,  A.,  227, 237 
Ross,  G.,  777, 439 
Ross,  L.  P.,  603 
Ross,  M.  S.,  481 
Ross,  M.  W„  227, 237 
Ross,  S.  M„  66, 123,  169,  777, 195, 
473, 416,483,101 ,712,713, 
719,722,727 ,728,729,731, 
732,733,757,  760,115,804 


Rossano,  V.,  347, 353 
Rossen,  S.,  770,  802 
Rossett,  A.,  28,  519,  520, 524,  660, 
670,  685 ,690 
Rosson,  M.  B.,  668 
Roth,  K.  J.,  565 

Roth,  W.-M.,  75,  87, 98,  99, 106, 
108,110,556,654,657 
Rothen,  W„  413,483, 73 1,732, 735, 
761 

Rothmayer,  A,  224 
Rothwell,  W.  J.,  509,  514, 524,  707, 
773 

Rouet,  J.-F.,  204, 209,  215 ,224 
Rourke,  L„  411, 422,  780,  804 
Rousseau,  J.  J.,  398 
Rouwette,  E.  A.  J.  A.,  335 ,344 
Rovai,  A.  P.,272, 276 
Rowe,  A.  L„  781, 804 
Rowe,  D.,  688,  690 
Rowe,  M.,  223 

Rowland,  G.,  708,  773,  741, 742, 
743,756,760 
Rowland,  R.,  685, 690 
Royer,  J.  M„  445, 455 
Rozelle,  R.,  420 
Rozin,  P.,  49, 53 
Rubin,  A.,  412,  422 
Rubin,  I.,  37 
Rudd,  J.,  666,  670 
Rugarcia,  A.,  805 
Ruiz-Primo,  M.  A.,  600,  604 
Rumbaugh,  J.,  363 
Rumble,  G.,  261, 266, 267 
Rumelhart,  D.  E.,  41, 44, 52, 53,  76, 
82 

Rummel,  N.,  721,  767 

Rummelhart,  D.  E.,  1 13,  724 

Rummler,  G.  A.,  26, 28,  512, 524 

Ruopp,  R.,  98,  770 

Ruskin,  R.  S.,  192,795 

Russel,  T.,  226, 237 

Russell,  J.  D.,  266 

Russell,  S.J.,  242, 249 

Russo,  J.  E„  579, 593,  787,  804 

Rust,  F.,  622,  637 

Ruth,  M„  334, 343 

Rutherford,  M.,  108 

Ryan,  B.  A.,  192,796 

Ryan,  G„  492, 504 

Ryan,  J.  M„  803 

Ryan,  L.,  54 

Ryan,  M„  777 

Ryan,  R.  M„  324, 325,  327 


Ryan,  T.,  231,535 
Ryder,  J.M.,  583,  587,  593 
Ryle,  G„  387, 398 
Rynearson,  K.,  224 
Rysavy,D.,411, 422 
Rysavy,  S.,  723 
Ryu,  E„  743, 483 
Ryu, J„ 243, 247 

s 

Saabagh,K„  639,  645 
Saadat,  F.,  603 
Saari,  L.  M.,  326 
Sabherwal,  R.,247 
Saccuman,  M.,  387 
Sadagic,  A.,  327 
Sadoski,  M.,  96 

Saettler,  P.  J.,  5, 6, 7, 8, 9, 14, 20, 77, 
82 

Saez,  A.,  342 
Sage,  S„  486, 487,  505 
Saini-Eidukat,  B.,  326 
Saito,  R.  S.,  669 
Sakallah.K.,  639,  645 
Salaff,  J„  423 
Salanova,  M.,  50, 53 
Salas,  E„  323, 325, 327,  589,  593, 
651 ,656,657,658,  776,781, 
800,  801,803,  804 
Salen,  K.,  105, 106,  770,  218, 224 
Sales,  G.C.,  28,  723,  133,747,743, 
409,410,411,413,479,422 
Salgado,  A.  C.,  248 
Saljo,  R.,  256 
Salmen,  D.  J.,  209 
Salmeron,L„  203,  204, 209 
Salmon,  G.,  264, 267 
Salo,  O.,  803 

Salomon,  G.,  28, 70,  82, 273, 276, 
338,344,410,422,728,731, 
733,734, 761,111,804 
Salter,  M„  327 
Saltzman,  P.,  523 
Salvendy,  G„  579,  580,  584,  585, 
589,590,597,593 
Salvucci.D.D.,  788, 804 
Salzman,  M.,  325 
Samarapungavan,  A.,  209 
Samaras,  H.,  48,  53 
Samouilova,  M.,  507 
Sampson,  D.,  220, 224, 482 
Sampson,  V„  150,  151,759 
Sanchez,  E.,  363 


855 


Author  Index 


Sanchez,  F.,  327 
Sancho,  P.,347,  352 
Sandberg,  J.  A.  C„  477, 480 ,  805 
Sanders,  C.  E„  139 ,142 
Sanders,  J.  R.,  617 
Sanders,  M.  G.,  760 
Sandoval,  W.  A.,  160 
Sansone,  C.,  134, 143 
Santic,  T.,  481 
Sanz,  C.,  138, 143 
Sapon,  S.  M.,  190, 195 
Sapsford,  R.,  766,  804 
Sarason,  S.  B.,  622,  631 
Sarbin.T.R.,  59, 65,  66 
Sarewitz,  D.,  649,  658 
Sarfo,  F.  K.,  453, 455, 456 
Sargent,  C.,  468 
Sartoris,  M.  L.,  327 
Sasayama,  G.  M.  D.,  390,  398 
Sasson,  J„  245, 249 
Sato,  E.,  602 
Satoris,  M.  L.,  327 
Satwicz,  T.,  169,777 
Saunders,  N„  91, 502,  610 
Saussure,  F.,  24,  28 
Savage,  R.  S.,  656,  657 
Savelsbergh,  E.  R.,  343, 344, 467 
Savenye,  W.C.,61,  66,209,1X9, 
767,765,767,768,770,771. 
112,113,114,116,804,805 
Savery,  J.  R.,  74, 82,  101,770,432, 
439 

Savin-Baden,  M„  487, 495,  504 
Savoie,  J.  M„  487,  504 
Savoy,  M.,  662,  663,  668 
Sawyer,  M.  O.,  190,795 
Sax,  G.,  770,  804 
Saye,J.  W.,335 ,342 
Sayeki,  Y„  124 
Scalese,  R.  J.,  564 
Scalise,  K„  599,  600,  604 
Scandura,  J.  M.,  446, 456, 473,  483 
Scanlon,  E.,421, 423, 459, 467, 609, 
611,677 

Scardamalia,  M.,  28,  104, 108, 110, 
146,760, 166,777,214,219, 
221,224,  229, 257,409, 415, 
422, 434, 437,  438,439 
Scarr,  S.,  733,  767 
Scarupa,  H.  J.,  438 
Schaafstal,  A.,  583,  586,  590, 593 
Schacter,  D.  L.,  40, 48, 55 
Schacter,  J„  599, 600, 601 , 602, 602, 
603,  604,  802 


Schaefer,  H.  H„  197 
Schafer,  L.,  519, 524 
Schallert,  D.,  209,  805 
Schank,  R.  C„  28,  102,  770,  323, 
327, 449, 456,  554,  565,  649, 
658 

Schatschneider,  C.,  543, 565 
Schatz,  S.  C„  526,  536,  756,  767 
Schauble,  L„  322, 326,  33 1 , 343, 

466.458.552.565,  758 
Schecker,  H.  P.,  467 
Scheckler,  R.,  70S 
Scheer,  A.  W„  334, 544 
Scheibe,  K.  E.,  59,  66 
Scheiter,K„  201, 208,  802 
Schenk,  K.,  429, 439 
Schenker,  J.,  542 

Scherpbier,  A.  J.J.  A.,  455, 505, 504, 
505, 565, 564, 565 
Scherpereel,  C.,  321, 527 
Schiaffino,  S.,  314 
Schimmel,  B.  J„  126,  128, 132, 143, 
389 ,398 

Schimming,  J.  L.,  420 
Schittekatte,  M.,  651, 658 
Schkade,  L.  L.,  54 
Schlechty,  P.  C„  625, 657, 699,  707 
Schlundt,  D.  G„  489, 491 , 504 
Schmid,  R.F.,  655, 658 
Schmidt,  H.  G„  486, 490,  491 , 492, 
493,494,495,496,498 ,502, 

504.505.554.560.561.565, 

565.101.713 

Schmidt,  R.  A.,  319, 327, 443, 456 
Schmidt,  W.  H„  389, 399,  620,  630 
Schmitz,  J.,  209 
Schnackenberg,  H.  L.,  804 
Schneider,  R.  M„  654,  658 
Schneider,  W„  44, 55, 444, 445, 456, 
583,  584,  593,  785,  804,805 
Schneiderman,  B.,  320,  326 
Schnotz,W„  95, 96 
Schnur,  E.,  564 
Schoelles,  M.  J.,  222 
Schoenfeld,  A.  H„  102,  104,  770, 
436,459,746,757 
Scholastic  Library  Publishing,  255, 
256, 257, 238, 240 
Scholes,  J.,694,  707 
Schon,  D.  A.,  37, 37,  559,  565,  635, 
642,645,662,665,670 
Schooley.C.,  298, 303 
Schott,  F  .,20,28, 183, 184, 396, 644, 

645.101.712.713 


Schraagen,  J.  M.,  579, 583, 589, 590, 
591,593 

Schrader,  A.  W .,26 
Schramm,  W.,  264, 267 
Schroeder,  R.,  327 
Schroth,  M.  L.,  137,745 
Schultz,  S.E.,  604 
Schultz-Ross,  R.  A„  489, 493, 505 
Schulze,  K.  G.,  143, 224, 294 
Schunk,  D.  H.,  121,724,  134,745, 
323,527 

Schutzwohl,  A.,  49, 55 
Schuurman,  J.,  381 
Schuwirth,  L.  W.  T„  784,  805 
Schwab,  S .,397 
Schwabe,  D.,  481 

Schwartz,  B.  J.,  319.  325,  311,380 
Schwartz,  D.  L„  101 , 70S,  770,  146, 
760, 115,184,221,231,292, 
505,534,583,593,656 
Schwartz,  R.  D.,  805 
Schwartz,  R.  W„  490, 501,503, 505 
Schwartz,  S.,  227 

Schwarz,  C.  V.  33 1 , 341 , 344, 458, 
462,463,465 ,467 
Schwarz,  E„  475, 480,483 
Schweiger,  D.  M.,  779, 181,804 
Schweizer,  K.,  274, 276 
Schwen,  T.  M„  524,  668,  669,  712 
Schwert,  D.,  326 
Schwier,  R.  A.,  690 
Schworm,  S.,  140, 480,  757 
Scifo,  P.,  5S7 
Scots  Community,  6, 20 
Scotti,  J.  R.,  63,  66 
Scradamalia,  M.,  154,  760 
Scribner,  S.,  254, 256 
Scriven.M.,  606,  61 1,677 
Scrivener,  S.,  686,  687,  690 
Scroggs,  G.  F.,  795 
Seale,  C„  559, 564 
Seaman,  J.,  274, 274 
Seay,  A.  F„  95,  96 
Sechrest,  L.,  805 
Seeberg,  C.,  483 

Seel,  N.M., 20, 25,40,43,44,45, 46, 
47, 53, 183,184,  341. 344, 396, 
429, 439, 644, 645,  706,  772, 
713, 115,183,804, 809, 8 10,  S75 
Seels,  B.,  640,  645,  740,  748,  767 
Seely,  B„  603 
Seethaler,  S.,  149,  760 
Segers,  M.,  487, 502, 504, 592,  783, 
805 


856 


Author  Index 


Seibert,  D.,  208 

Seidel,  R.  J.,  179, 180, 184, 474, 476, 

482.483 

Seifert,  K.  L„  57,  66 

Self,  A.  J„  474, 477, 479 

Self,  J.,  243, 249,  286 ,292 

Seltzer,  S.,  487,  505 

Senge,  P.  M„  100, 110,  245, 249, 

693,694,695,696,707 
Senko,  C.,  135,  138,745 
Senn,  P.  R.,  505 
Seo,K„  280, 294 
Setzer,  J.,  18, 20 
Severin,  D.  G.,  40, 43,  796 
Severin,  W.  J.,  53 
Sewart,  D.,  261, 267 
Sfard,  A.,  80,  82, 98,  770 
Shabo,  A.,  433, 439 
Shadbolt,  N.  R.,  592 
Shalin,  V.  L.,  597 
Shanner,  W.M.,487 
Shannon,  B.,  98,  770 
Shannon,  C.  E„  11,20, 24 ,28 
Shapira,  B.,  481 

Shapiro,  A.  M„  200,  201, 202,  204, 

205. 206. 209. 479. 483 
Shapiro,  J.  A.,  143, 294 
Shapiro,  W.L.,  733,  761 
Sharan,  S.,  326 

Share,  D.L.,  543, 564 
Sharma,  P.,529,  532, 536 
Sharma,  R.,  293 
Sharp,  D.  L.  M.,  46,  53 
Sharp,  H.,  658 
Sharpies,  M„  255, 256 
Shashaani,  L.,  408, 421 
Shavelson,  R.  J.,  600, 604,  616, 720, 

726,  737,  758,  761 
Shaw,  E„  285 ,294 
Shaw,  K.N.,  326 
Shaw,  M.  L.,  46, 53 
Shaw,  R.  E„  70,  71, 72,  73,  74,  75, 

81,82,98,108,110 
Shcerlis,  W.L.,807 
Shcherbinina,  O.,  292 
Shea,  G.  P.,  776,  802 
Shea,  P.,271, 274 ,275,276 
Shear,  L  .,503 
Sheehan,  K.  M„  603 
Sheffield,  F.D.,  79, 196 
Sheingold,  K.,  420 
Shelby,  R„  143,224,294 
Shelly,  P.  B.,  28 
Shelton,  J.B.,  490, 505 


Shepard,  L„  784,  805 
Shepherd,  N.G.,  492, 505 
Sher,B.T„  502 
Sherin,  B.,  342 

Sherman,  J.  G.,  191 ,  192,  795, 196, 
411,482 

Sherwood,  R.  D.,  325,  656 
Shiffrin,R.M., 41, 44, 57, 53, 87, 95, 
445, 456,  584, 593, 185,804, 
805 

Shih,L„  264, 268 
Shih,  P.,321, 327 
Shimoda,  T.  A.,  484 
Shimura,M„  284, 294 
Shin,  E.  J.,  204, 209,  774,  805 
Shipman,  H.L.,  495, 505 
Shirley,  A.,  500 
Shlechter,  T.,  409, 422 
Shneiderman ,  B . ,  2 1 8 , 224 
Short,  J.,  411 ,422 
Shortliffe,  E.,  641 , 644 
Shortliffe,T„  243, 249 
Shoval,  P.,  481 

Shrager,L„  114, 115, 118, 121,724, 
389, 398 
Shrock,  S.  A.,  28 
Shternberg,  B.,  155,  156, 160 
Shuell,T.  J.,  530,  536 
Shute,  V.  J.,  214, 224, 245,249, 279, 
280,284,285,287,291,294, 
451,455,459,460,467,474, 
483,593,731,761 
Siantz,  J.  E.,  668 
Sibley,  D.F.,  271, 276 
Sideridis.G.,  321, 327 
Sidner,C„  292 
Siegel,  A.  W.,398 
Siegel,  J.,  410, 427, 422 
Siemens,  G.,  571, 574, 575 
Silber,  K.  H„  183 
Silber,  O.,  427 

Silberman,  H.  F„  190,  795, 196 
Siler,  S.  A.,  747 
Sillin,  L.  F.,  593 
Silver,  M.,  505 
Silverman,  B.,  641, 645 
Silverman,  L.  K„  307,  309,  31 1 , 
373 

Silvern,  L.  C.,  13,20 
Silverstein,  M.,  159,  255 
Sime,  J.,  252, 256 
Simm,  H.,  314 
Simmons,  J.  R.,  627, 631 
Simmons,  S.  J.,  627 ,  631 


Simon,  H.  A. ,28, 41, 54,  146,767, 
770,  242,  245, 248,  319,  326, 
356,364,371,380,387,410, 
491,504,579,581,592,648, 
649,  658,  665, 670,  773,  779, 
180,186,181,188,800,801 
Simone,  G.,  630 
Simpson,  J.  A.,  806 
Simpson,  O.,  261 , 267 
Sims,  H.  P.,649,  652,  657 
Sims,  R„  570,  571, 572,  573,  574, 
575, 657 

Simsek,  A.,  412, 422 
Sinai,  J.,  565 
Sinatra,  G.  M.,  237 
Sinclair,  K.,  476, 481,553,  565 
Sinclair,  S.,  557,  558, 565 
Singer,  D.  G.,  49, 54 
Singer,  J.,  654,  658 
Singer,  J.  L„  49,  54 
Singer,  N.R.,  230, 237 
Singleton,  J.,  252, 257 
Singley,M.K„  445, 456 
Sinha,  R.,  603 
Sins,  P.H.M.,  464, 467 
Sioutine,  A.  V.,  344 
Sired,  S.G.,  599,  600,  603 
Sison,R.,  284, 294 
Sistrunk,  F.,  451, 456 
Sitar,  A.,244 ,248 
Skamp,  K.,  438 
Skibba,K„  275 

Skinner,  B.  F„  12, 20, 24, 28, 63, 66, 
188, 189, 190, 191, 193, 194, 
796,406,422,707,708,710, 
773 

Skinner,  E.,  53 
Skolmoski,  P.,  209 
Skopeliti,  I.,  566 
Slack,  R„  252, 257 
Slater,  J.  S.,  629,  637 
Slater,  M„  324, 327 
Slator,B.M„  218, 224,  326 
Slaughter,  J.E.,  435, 439 
Slavin,  R.  E„  264, 267,  655,  658, 
726,  729,  767 
Slavin,  S.,  326 
Slavkin,  M.,  276 
Sleight,  D.  A. ,522, 524 
Sloan,  D.  A.,  505 
Sloane,  K.  D.,  396 
Sloep,  P.,365,  759,  S02 
Slonim,N„  544,  563 
Slotman,  K.,  747 


857 


Author  Index 


Slotta,  J.  D.,  148, 157, 160, 161,503, 
535 

Slovic,P  .,592 
Slusher,  J.,  248 
Slymen,  D.,  631 
Small,  E.  E.,  804 
Small,  R.  V„  239, 240 
Smith,  A.,  257, 

Smith,  A.  J„  502, 503 
Smith,  B„  651,  701 
Smith,  B.B.,  795 
Smith,  B.K.,  760 
Smith,  C„  340, 344 
Smith,  D.L.,  487, 505 
Smith,  D.  E.  P.,  26 
Smith,  D.K.,  657 
Smith,  E.  R.,  436,  439 
Smith,  G.  G.,  337,  338, 344 
Smith,  J„  57 , 242, 248,  580, 592, 
784, 801 

Smith,  K„  114,  115,  118,724,427, 
524 

Smith,  K.  A.,  657 
Smith,  K.G.,  657 
Smith,  K.M.,  690 
Smith,  K.U.,  796 
Smith,  M.F.,  796 
Smith,  M.  K.,  35,57 
Smith,  N.H.,  797 

Smith,  P.  L„  28,  126,  128,  136, 143, 
190, 194,797,386,387,389, 
390,391,392,394 ,397,398, 
399,113,805 
Smith,  R.B.,  459, 467 
Smith,  R.F.,  490, 505 
Smith,  R.M.,  495, 505 
Smith,  T.,  274 
Smith,  V.D.,  506 
Smither,  J.  A.,  50, 52 
Smith- Jentsch.  K.  A.,  322, 326,  327 
Smits,  D.,  481 
Smits,  J.G.M.M.,  504 
Smits,  P.B.  A.,  488,  505 
Smizak,  B.,  427 
Smolensky,  P.,  53 
SmordelhO.,  263, 267 
Smuts,  J.C.,  444, 456 
Snelbecker,  G.  E„  640,  645 
Snell,  L„  565 
Snider,  B.,  423 
Snir,  J.,  344 
SNOW,  286, 294 
Snow,  C.E.,  294 
Snow,  E.  R.,  47 1 , 476, 481 , 483 


Snow,  R.  E„  46,  54,  279,  280, 294, 
410,422,470,472,480,731, 
758 

Snyder,  J.L.,  458, 467 
Snyder,  W.,  245, 249, 439 
Soejima,K„  799,  805 
Soetaert,  E.,  256 

Soller,  A.L., 219, 224, 286, 294, 479, 
482,483 

Soloman,  B.  A.,  307,  312, 313 
Solomon,  D.,  800 
Solomon,  G.,  265,  266 
Solomon,  M.,  698,  707 
Solomon,  P.,235, 240 
Soloway ,  E„  708, 759, 760, 466, 631 , 
656,  657,  759 
Sommer,  B.,  276 
Song,  I„  51, 54 
Song,  L.,  535 

Songer,  N.  B„  146,  149,767 
Sonnenwald,  D.  H.,  437 
Sorg,S„  275 

Sosik,  J.  J.,  648,  651, 652,  657,  658 
Sosnovsky,  S.,  292 
Sowa,  J.F.,  246, 249 
Spada,H„  342 
Spagnolli,  A.,  321 . 327 
Spandel,  V„  784, 800 
Spannaus,  T.,  28,  875 
Sparrow,  J.,  445, 454 
Specht,M.,  475, 483 
Spector,  J.  M.,  24, 27, 28,  192,  194, 
797,213,224,337,341,345, 
344,  348, 353, 395,  393, 397, 
398, 443, 456, 462, 467,  676, 
678,684,708,709,773,781, 
805,809,811,812,875 
Spencer,  J.,  560, 564, 565 
Spero,  L.,  326 

Spillane,J.  P.,622,  624, 637 
Spiller.R.,  398, 455 
Spillers,  W.  R.,  645 
Spiro,  R.  A.,  710,  773 
Spiro,  R.  J.,  200, 205, 209, 270, 214, 
224,487,503,655,657,658 
Spitzer,  K.  L.,  534 
Spitzer,  M.  E.,  794 
Spool,  J.M.,  521, 524 
Spradley,  J.  P.,773, 775,  805 
Sprague,  D.,  325 

Sproull,  L„  41 1 , 422,  651 , 652,  657 
Spurlin,  J.,  309, 373 
Squire,  K„  708,  320,  323, 327, 416, 
421,422,701,146,757 


Squire,  K.D.,  72,  87, 708,  158,759, 
621,692,637 
Stafford,  M.  R„  54 
Stafford,  T.F.,  51, 54 
Stahl,  E„  740, 480,  757 
Stahl,  G.,  164,  166,  167,777,222, 
264,267,  77 6,  111,  778,  779, 
805 

Stahl,  R.  J.,  390,  398 
Stahl,  S.  A.,  544,  545, 565 
Stajkovic,  A.  D„  517,  518,  523, 524 
Stake, R.E., 608, 611, 612, 677 
Stalcup,  K.  A.,  224 
Staller,L„  799,  805 
Stallman,  R.,  357 
Stampe,  D.  M.,  807 
Stamps,  J.,  651, 657 
Stanley,  J.C.,  676,  758 
Stanne,  M.,  427 
Stanne,  M.  B.,  802 
Stansfield,  J„  284, 294 
Starenko,M„  271, 273,276 
Starr,  J.  M.,  373 
Stash,  N„  487 
Stasheff,  E.,  8,  79 
Stasko,  J.,  439 
Stasser,  G . ,  65 1 , 658 
Staszewski,  J.  J.,  785, 807 
Steed,  M„  462, 467 
Steenbeek,H„  552, 565 
Stefanou,C.R.,  48, 54 
Stein,  B.S.,  103,  708,  1 14,  724 
Stein,  F.  S„  390,  391 , 398, 410, 422, 
448,455 
Stein,  N„  49, 54 
Steinacker,  A.,  475, 483 
Steinberg,  E.  R„  139,  743,  392, 398, 
410,422,475,483 
Steinberg,  L.S.,  293,  604 
Steinhart,  D.,  222 
Steinmetz,  R.,  483 
Steinmuller,  F.,  760 
Stelmach,  L.  B.,  786,  805 
Stelnicki,  M.  B„  783 
Stelzer,  J.,  483 
Stent,  A.,  224 
Stephan,  M.,  548,  565 
Stephens,  D.  L.,  593,  804 
Stephenson,  S.,  409, 422 
Stepien,  W.C.,  505 
Stepien,  W.  J.,  487, 489, 490, 497, 
502,505 

Sterling,  C.,  418, 479 

Sterman,  J.  D„  28, 336, 344,463,466 


858 


Author  Index 


Stem,  D.,  427 
Stern,  F„  215 ,224 
Stern,  J.  L„  759 
Stern,  K„  670 
Stern,  M.,  248 

Sternberg,  R.  J„  544, 565,  580, 593 
Sternberger,  C.  S.,  50, 53 
Stevens,  A.L., 27, 458, 466, 641, 645 
Stevens,  C.  K.,  326 
Stevens,  R.,  164, 165, 166, 167, 169, 
170,777,604 

Stevens,  R.  H„  601, 602,  604 
Stevenson,  M.,  563 
Stevenson,  W.  G„  799,  805 
Stewart,  C.,  480 
Stewart,  C.  D.,  482 
Stewart,  G.  L.,  652,  658 
Stewart,  J„  322 ,325,327 
Stewart,  K.  K„  430, 437, 439 
Stice,  J.  E„  805 
Stikeleather,  G.,  65 
Stillman,  D„  296,  297,  300,  302, 303 
Stinson,  J.  E.,  497, 505 
Stock,  W.  A.,  127,  132,  133,  135, 
137, 138,747,742,743,214. 
223 

Stocke,  J.T.,709 
Stocklin,  M.,  53 
Stockly,  S.,  630 
Stoddart,  B.,  437 
Stofflet.R.,  583, 593 
Stoll,  C„  649,  658 
Stolovitch,  H.  D.,  28, 523 
Stolterman,  E.,  685,  690,  693,  694, 
695,696,707 

Stolurow,  L.  M.,  190,  795,  797 
Stone,  C.  A.,  395 
Stone,  D.  L.,  524 
Stone,  S„  297, 303 
Stopford,  A.,  563 
Stout  R.  J.,  807 
Strand,  E„  772,  804 
Stratford,  S.J .,466,759 
Stratton,  T.  D.,  507 
Strauch,  R.,  442, 444, 456 
Strauss,  A.  L„  74, 75, 80, 87, 82, 772, 
805 

Strauss,  W.,  296, 297,298, 299, 300, 
302,303 

Streatfield,D„  236, 240 
Street,  E.  M.,  63, 65 
Streumer,  J.  N.,  523 
Stright,  A.  D.,  433, 438 
Strijbos,  J.-W.,  777 


Strike,  K.  A.,  331, 344 
Strobel,J.,331, 341. 343, 344 
Stroble,  B.,  535 
Strosahl,  K.  D.,  65 
Stubbs,  S.  T„  638,  640,  642,  645 
Stufflebeam,  D.  L„  607,  61 1, 677 
Sturges,  P.  T.,  137,743 
Stylianou,  A.,  203, 209 
Su,B„  669 
Su,  Y.,  203,270 
Subhiyah,  R.  G.,  603 
Subrahmanyam,  K.,  227, 237 
Suchan,  J.,  620,  628,  637 
Suchman,  E.,  607,  67  7 
Suchman,  J.  R.,  10,20 
Suchman,  L„  98, 770, 169, 170,777, 
254,257 
Sue-Jen,  C.,  267 

Sugar,  W.  A.,  433, 439,  662,  665, 

"  666,667,670 
Suhaka,  M.,  784 
Sullins,  J„  227, 292,  875 
Sullivan,  H.  J„  237 , 524,  722,  732, 
750,755,759,  760,  767 
Sulzen,  R.,  196 
Sumner,  M.,  628,  637 
Sun,  C„  756,  758 
Suppes,  P.,473, 483 
Supplee,  L.,  437 
Suraweera,  P.,  223 
Surber,  J.  R.,  137,743 
Surmon,  D.,  467 
Surowiecki,  J.,  246, 249 
Susman,  E.,  41 1 , 422 
Suthers,  D.  D.,  777, 409, 414, 427, 
422,423 
Sutinen,  E.,  353 
Sutliff,  R.,  119,  121,724 
Sutton,  A.,  564 
Sutton,  S .,  4 1 1 , 423 
Suwa,M.,  686,  690 
Suzuki,  H„  777 
Swaak.J.,  342, 465, 467 
Swallow,  J„  422 
Swan,  K.,  264, 267, 268 
Swanson,  D.  B„  471, 495, 505,  603 
Swanwick,  T.,  560, 565 
Sweet,  R„  261, 268 
Sweller,  J„ 28, 40, 42, 54,  86, 88, 96, 
138,747,783,204,270,214, 
222,224,374,370,371,372, 
374.375,376,377,378,380, 
381,455, 523, 593,  712,  723, 
758, 783,807,805,875 


Sweller,  S„  370,  371, 387 
Swenson,  R.,81 , 108 
Swindell,  L.  K„  747 , 742,  743 
Swing,  S.,  482 

Swisher,  M.  V.,  130,  138,  742 
Sy,T„  779,  780, 805 
Sy-Miaw,  L.,  757 
Symonds,  P.  M.,  134, 143 
Szeberenyi,  J.,  487,  505 
Szechter,  L.  E.,  552, 565 
Szucs,  A.,  261 , 266 

T 

Tabak,  I„  760, 432, 433, 434  439 
Tabbers,  H.,  803 
Taber,  J.  I. ,  190,  191,797 
Tait,A„  261, 268 
Takenaka,  M.,  438 
Talbert-Johnson,  C.,  487, 504 
Talbi,H.,  523 
Talmy,L„  641, 642,  645 
Talsma,  V.  L„  654,  658 
Tamblyn,  R.  M.,  101, 708, 486, 487, 
488,507,641,643 
Tamin,  R.,  118,  723 
Tan,  O.S.,  500, 505 
Tan,  P.,780,  802 
Tang,  J.,  52 

Tannenbaum,  K.  R.,  545, 565 
Tannenbaum,  S.  I.,  322, 327, 656, 
658 

Tanner,  D.,  10, 20 
Tanner,  E.  J.,  230 
Tanskanen,  T.,  651, 658 
Tao,  P.-K.,  339,  344 
Taplin,  L.  J.,  628,  637 
Tapscott,  D.,  296, 303 
Taraban  R.,  213, 224 
Tarabanis,  K.,  53 
Taricani.E.,  115,  118,724 
Tate,  E.,  150,767 

Tatsuoka,  K.  K.,  130,  740, 598, 601, 
602, 603, 604 

Tatsuoka,  M.  M„  598,  601, 604 
Tatters  all,  C.,  361 , 362, 364, 365, 
639, 644 

Taveggia,  T.  C„  192,  797 
Taylor,  C.  A.,  389, 396 
Taylor,  F.  W„  510, 524 
Taylor,  I.,  438 
Taylor,  J.,  263, 268, 423 
Taylor,  K.L.,  787,  788,  805 
Taylor,  L.,  126,  743, 224, 294 


859 


Author  Index 


Taylor,  P.,  256 
Taylor,  R  .,143,325 
Taylor,  S.  A.,  553,  556,  565 
te  Molder,  H„  640,  645 
Teachable  Agent  Group  at 
Vanderbilt  (TAG-V),  292 
Tearle,  P„  623,  626,  627,  631 
Teasley,  S.  D„  28, 267,  286, 294 
Tedder,  M.  C.,  203, 209 
Teich,  P.,553, 565 
Tekian,  A.,  495, 496, 504 
Temiyakarn,  C.,  420 
Tempelaar,  D.  T.,  502 
Ten  Cate,  O.,  560,  565 
Tenenbaum,  G.,  780, 801 
Tenenbaum,  J.  B.,  341, 344 
Tennant,  M.,  35,  36, 37 
Tenney,  Y.,  459, 466 
Tenny,  J.,  206, 208 
Tennyson,  D.  R„  47 1 , 473, 474, 475 , 
482,483 

Tennyson,  R.  D.,  14, 20, 27, 28, 174, 
183, 184,  387,  392,  393, 396, 
398, 410, 423,  640,  644,  645, 
706,709,710,772,773,722, 
728,731,732,735,756 ,760, 
761 

Teong,  S.  K.,  430, 439 
Terada,  R.,  669 

Tergan,  S.  O.,  204, 210,  330, 344 
Terlouw,  C.,  386, 398 
Ternes,  D.,  292 
Ternier,  S.,  353 

Tessmer,  M„  27, 141,592, 644, 662, 
664,665,666,670, 772,742, 
761 

Tettamanti,  M.,  372, 381 
Thagard,  P.,555, 565 
Thangiah,  S.,  334,  344 
Tharp,  R„  427, 428, 432, 433, 437, 
439 

Theodorou,  D.,  593 
Therrien,  M.,  658 
Thibaut,  J.,  406, 427 
Thibodeau-Hardiman,  P.,  563 
Thielo,  M.  R.,  466 
Thirunarayanan,  M.,  774,  803 
Thomas,  D.,  465, 466 
Thomas,  G.,  759 
Thomas,  M.,  108 
Thomas,  R.  M.,  264, 268 
Thompson,  J.,  617 
Thompson,  S.,  784,  805 
Thomson,  D.  M.,  48, 54,  177,  184 


Thorley,N„  583, 593 
Thorndike,  E.L..  126,143,445,456, 
470 ,483 

Thorndike,  R.  M„  770,  771, 805 
Thornton,  N.  E.,  742 
Thurman,  R.  A.,  141 
Thyer,B.  A.,63 ,65 
Tiberius,  R.  G„  558,  560, 565 
Tidwell,  L.C.,  227, 237 
Tierney,  P.,381 
Tiffin,  J.,  770,  805 
Tiffin,  J.W.,  11,20 
Tikkanen,  R.,  227, 237 
Tilburg  University,  527, 536 
Tilden,  D.  V.,  391,398 
Tillema,  H.  H„  137, 142,  307,  375 
Tiller,  T„  479 
Tillman,  M.  H„  772 
Tilton,  J.  W.,  7,  79 
Tindall-Ford,  S.,  376,  380, 381 
Tinker,  R„  146,  156,  760, 161,467 
Titchener,  E.  B.,  790,  805 
Tobias,  R.,  334, 344 
Tobias,  S„  50,  54,  280, 294, 410, 
423, 470, 472, 474,  483 
Todaro,  S.,  223 

Todd,  R.  J.,  235,  237,  238, 240 
Toffler,  A.,  693,  707 
Tomczak,  R.  L.,  490, 505 
Tompsett,  C.,  351, 353 
Toney,  C.D.,  239 
Topp,  B.  W.,  142 

Torgesen,  J.  K„  543,  544, 565, 566 

Torkelson,  G.  M.,  12,20 

Torp,  L.,  101,709,486,487,505 

Torreano  L.  A.,  249 

Toth,  J.,  222 

Toulmin,  S.  E.,  555, 565 

Toutouzas,  K.  G.,  593 

Towle,  B.,  483 

Towne,  D.  M.,  459, 467,  755,  760 
Towne,  L.,  676,  720,  758,  761 
Towns,  S.  G.,  482 
Towstopiat,  O.,  395 
Trabasso,  T.,  803 
Tracey,  M.  W„  742,  753,  756,  767 
Trafton,  P.  R.,  142,505 
Traiger,  S.,  71, 82 
Training  Magazine,  509, 524 
Traum,  D.,  219, 227 
Traxler,  J.,  262,  263, 267 
Treacy,  D.,  143, 294 
Trella,M.,  281, 294 
Tremayne,  M.,  527, 535 


Trepper,  T.,  137, 143 
Trevino,  L.,  54,  407, 423 
Trevitt,  A.,  256 
Trienes,  R.  J.  H.,  803 
Trigano,  P.,  364 
Trig  well,  K„  255, 256,  270, 275 
Triole,  J.,  357,  364 
Tripp,  S.,  396,  664,  665, 666, 670 
Trochim,  W.  M.  K„  530, 536 
Troitzsch,  K.  G„  335,  344,  361, 
364 

Trollip,  S.  R.,  26, 95,  707,  777 
Tron,  M.,  208 
Trotschel,  R.,  597 
Trowbridge,  D.,  409, 410, 423 
Trummer,  C.,  292 
Tsai,  B.,  412,  422 
Tsai,  C.C.,  430, 439 
Tsikalas,  K.  E.,  478, 484 
Tu,  C.H.,  411, 423 
Tucker,  B.,  658 
Tucker,  T.  L.,  237 
Tuckman,  B .,  65 1 , 658 
Tudge.J.,  552, 565 
Tudhope,  D.,  257 
Tufte,  E.  R.,  24, 28,  147,767 
Tulgan,B.,  296,  300, 303 
Tulving,E.,  48, 54 
Tunteler,  E.,  552, 565 
Tuovinen,  J.  E.,  380,  760,  789,  803, 
805 

Turban,  E„  334 ,344 
Turgeon,  A.  J.,  437 
Turner,  A.,  380 

Turner,  J.  C„  54,  219, 223, 433 ,438 
Turner,  M.  E.,  37 
Turns,  J.,  434,  437 
Turvey,  M.  T„  70,71, 72, 73,74, 75, 
82 

Tusting,  K.,  640,  643 
Tutoring  Research  Group  (TRG), 
292 

Tuzun,  H.,  108 

Tversky.B.,  43, 54, 91, 96, 215, 224, 
686,  690 

Twenge,  J.  M„  298,  300,  302, 303 
Twigg,C.,  272, 273, 276 
Twining,  P.,  256 
Twitched,  D.G.,  24,  25, 27 
Twyman,  J.  S.,  65 
Tyler,R.W„ 606,611, 677 
Tynan,  R.,  364 
Tynes,  B.,  237 
Tzanavari,  A.,  480 


860 


Author  Index 


u 

U.S.  Department  of  Education,  16, 
17,20 

Uchiyama,  K.  P.,  630 
Ueno,  N.,  124 
Ujcich,  K.  J„  66 
Ulich,R.,  385, 398 
Ulicsak,  M.  H„  134,747 
Ullmer,  E.  J.,  727,  761 
Underwood,  B.  J„  727,  761 
Underwood,  G„  409, 411,413, 423, 
786, 805 

Underwood,  J.,  805 
UNESCO,  351, 353 
University  of  Central  Florida,  27 1 , 
276 

Unklesbay,  R.,  553, 563 
Upitis,  R.,  420 

Urch  Druskat,  V„  780,  782,  805 
Ure,D.,  275 

Ureel,  L.  C.,  340, 342, 344 
Uribe,  D.,  227, 257 
Usher,  E.L.,  134,745 
Usoh.M.,  527 
Utterback,  J.,M„  620,  630 
Utts,  J.,  272, 276 

V 

Valadares,  K  .,276 
Valdez,  A.,  137,  742,  307,  314 
van  Aalst,  J.,  223 
van  Aken,  J.  E„  648,  649,  658 
van  Amelsvoort,  M.,  427 
van  Berkel,  H.  J.  M.,  504 
Van  Borkulo,S.  P.,463, 467 
Van  Boxtel,  C.,  556,  565 
van  Bruggen,  J.,  364, 365 
van  Bruggen,  J.  M.,  414, 423, 564 
Van  Dalen,  J.,  561, 565 
Vande  Ven.  A„  620, 657 
van  den  Akker,  J.,  146, 161,  302, 
303,394,398, 739,740,744, 
746,  752,  754,  759,  760,  761 
Van  den  Besselaar,  P.,  663,  667,  668 
van  den  Boom,  134, 143 
Van  den  Bossche,  P.,  488,  502,  505 
Van  den  Brande,  L„  252,  253, 257 
van  den  Hurk,  M.  M„  492, 505 
Van  der  Aalsvoort,  G.  M.,  550,  565 
van  der  Hulst,  A.,  203, 208 
van  der  Klink,  M.  R.,  456 
Van  der  Linden,  J.,  565 


van  der  Linden,  W.  J.,  599,  604 
van  der  Meer,  S.  B.,  502 
van  der  Meij,  H„  178,  179, 184 
van  der  Meij,  J.,  215, 224 
van  der  Molen,  H.  T.,  492, 504, 505 
van  der  Vleuten,  C.  P.  M.,  455, 495, 
501,504, 505,  562,  564, 565, 
784 ,802,805 

van  der  Wende,  M.,  273, 274 
van  Dijk,  T.,  708,  772 
van  Eck.  R„  341, 344 
Van  Es,R„  362, 365 
Van  Geert,  P.,  552, 565 
Van  Gerven,  P.  W.  M„  786, 803, 805 
van  Gessel,  E.,  497,  505 
van  Gog,  T„  43,  53, 143,  555 ,565, 
765, 161 ,  786, 787, 789, 805 
Van  Hiele,P.M„  546,  565 
Van  Hout,  J.  C.  H.  M.,  565 
van  Hout- W olters ,  B .  H .  A .  M . ,  343 , 
467 

van  Joolingen,  W.  R.,  308, 315, 342, 
331,335,340,341,343,544, 
458,459,462,463 ,466,467, 
419,482 

Van  Lehn,  K.,  208 
Van  Maanen,  J.,  775,  805 
Van  Mameren,  H.,  504, 564 
van  Merrienboer,  J.  G.,  24, 28, 54, 
143,  174,  176,  179,  180,  181, 
182,  183, 184, 197,  204,209, 
270, 222,  374,  375,  378, 387, 
386, 399,  442, 444, 446,  447, 
450, 455,  456, 470, 483,  564, 
565,579,583,587,589,593, 
643,101,712,713,759,760, 
774,784,785,786,789,502, 
803,805,  809,810,575 
van  Mullekom,  T.,  344 
Van  Note  Chism,  N„  252, 257 
van  Oostendorp,  H.,  204, 209 
Van  Ormer,  E.  B.,  9,  79 
Van  Patten,  J„  391 , 399, 448, 456, 
707  ,  773 

Van  Rosmalen,  P.,  362, 365 
Van  Schaack,  A.,  25 
Van  Someren,  M.  W„  787, 505 
Van  Tiem,D.M„  386, 399 
van  Vuuren,  S.,  227 
Vandevelde,K.J„  386,  565 
VanLehn,  K.,  126,  127,  128,  133, 
136, 138,740,742,743,213, 
216,222,224,284,285,292, 
294,437,  785,505 


Varelas,  M„  436, 439 
Vargas,  E.  A„  189,797 
Vargas,  J.,  189, 197,274 
Varian,  H.  R„  246, 245, 527, 535 
Varotto,  D.,  327 
Vartiainen,  M.,  520, 524 
Vasama,  S.,  564 
Vasconcelos,  A.,  245 
Vasireddy,  G.,  224 
Vasquez-Abad,  J.,  742,  767 
Vassileva,  J.,  281, 284, 287,  288, 
292,293,294,416,480 
Vaughan,  N„  274, 275 
Veach,  T.  L.,  245 
Veerman,  A.  L.,  414, 423 
Veermans,  K„  308, 375, 465, 467 
Veletsianos,  G„  417, 479, 423 
Velmahos,G.C.,  58 1,583, 586, 587, 
590,593 
Vemeer,  D.,  709 
Vendlinski,  T.  P„  600,  601, 604 
Vennix,  J.  A.  M.,  344 
Ventura,  M„  52, 227, 222 
Ventura,  S.,  743 
Verbeek,J.H.  A.  M.,  505 
Verbert,K„  350,  353 
Verhoeven,  B.  H.,  490, 505 
Vermeulen,  B.,  505 
Vermeulen,  L.,  505 
Vermunt,  J.,  565 

Vernon,  D.T.A.,488, 490, 505, 561, 
565 

VerschaffeLL.,  550, 566 
Verwijnen,  G.  M.,  505, 565 
Vicari,  R.  M.,292 
Vidal-Abarca,  E.,  96 
Vieitez,  D.,  65 
Viera,  A.,  342 

Vignare,  K„  271 , 272, 275, 276 
Vila,  X„  363 
Vilhjalmsson,  H.,  293 
Villachica,S.W„  521, 524 
Vincenti,  W.  G„  356, 365, 687, 688, 
690,  814,  <375 
Vipond,S„  672,  684 
Virvou,  M.,  218, 224,  281, 293 
Vishwanath,  A.,  628,  637 
Vispoel,  W„  209 

Visscher-Voerman,  I.,  667, 668, 670, 
742,743,754,767 
Visser,  L.,  752,  756,  767 
Visser.P.R.  S.,  554, 562, 565 
Vlissides,  J„  759,  807 
Vogel,  D.S.,  327 


861 


Author  Index 


Vogel,  J.  J.,  318,  320, 327 
Vogiatzis,  D.,  480 
Vogten,  H.,  364,  365 
Voit,  G.  M„  109 
Volkl.C.,  116,724 
Vollmeyer,  R.,  134, 143 
Volpe,  C.  E„  656 

von  Glasersfeld,  E„  15, 20,  69,  72, 
74,  78,  82,  708,  713 
Von  Hurst,  E.  M„  390, 399 
Voorhees,  E.,  217, 224 
Vosniadou,  S„  550,  552, 565,  566, 
593 

Voss,  J.F.,  658 
Vossen,  G„  358 ,365 
Votel,  J.  L.,  244, 248 
Vreman-de  Olde,  C.,  461 , 467 
Vu,N.V„  507, 505 
Vye,  N.  J.,  53, 108, 293,  534,  650, 
656,  658 

Vygotsky,  L.  S„  24, 28,  78,  82,  91 , 
96, 1 65 , 7  77 , 264, 268, 404,405 , 
423, 426, 428,  439, 478, 484 

w 

Waddington.  T.,  455 
Wade,  A.,  656 
Wade,  D.,  353 
Wade,  V.,353 
Wade,  W„  253, 257 
Wade-Stein,  D.,  227 
Wadley,V„  327 

Wager,  S.  U.,  126, 133, 137, 747, 745 
Wager,  W., 27, 32, 87, 126,745, 191, 
197, 396, 592,  665, 670,  690 
Wagner,  D.,  747 
Wagner,  H.,  483 
Wagner,  J.  E.,  747 
Wagner,  R.  K„  543, 565, 566 
Wainer,  H.,  805 
Walder,  F.,  53 
Walenstein,  A.,  661 , 670 
Walker,  A.,  422 
Walker,  D„  645 
Walker,  K„  637 
Walker,  M„  216, 224 
Wall,  D„  252, 256 
Wallace,  R„  740,  757 
Wallace,  R.  M.,  480 
Wallach,  D.,  53 
Waller,  D„  321, 327 
Walles,R„  292 
Walls,  W.  F„  743 


Walrond,  E.,  500 
Walther,  J.  B.,  89,  227, 237 
Walti,C.,  266 
Wanberg,C.R„  435, 438 
Wandersee,  J.  H.,  98,  770 
Wang,  A.  Y„  228, 237 
Wang,  C.  X.,  115,  118,724 
Wang,  C.-H.,  219, 224 
Wang,  F.,  394, 399,  535,  719,  727, 
730,733,755,767 
Wang,  F.K.,  429, 439 
Wang,  H„  208,  350, 353 
Wang,  J.,  723,  604 
Wang,  M.,  470, 471, 484 
Wang,  M.C.,  388,  398, 399 
Wang,  N„  293 
Wang,  Q„  227, 237 
Wang,  S.,  87, 201, 209,  803 
Wang,  Y.,  783, 293 
Ward,  T„  420 
Ward,  W„  221 
Warg,F„  534 
Waring,  B.,  275 
Warren,  P.,246, 249 
Wasson,  B„  284, 294, 478, 484 
Waters,  S„  353, 635,  642,  645 
Watkins,  C.L.,  193,794 
Watkins,  D„  392,  399 
Watson,  C.,  783 
Watson,  G.,  498, 505 
Watson,  J.  B.,  7,  79 
Watt,  Jr„  J.H.,  43, 46,  54 
Watts,  D.M.,  341, 343 
Waugh,  M.L.,  308, 375 
Waugh,  N.C.,  87,  96 
Wavell ,  C„  235, 236,  238, 240 
Weatherly,  R.,  621, 637 
Webb,  E.  J.,  772, 805 
Webb,  G„  284, 294 
Webb,  N„  409, 410, 412, 413, 423, 
777,  779, 780, 805 
Webb,  R„  574, 575 
Weber,  G„  480, 483 
Weber,  R„  245, 249 
Webster,  J.,  50, 54 
Wedman,  J.  F„  391, 399,  662,  665, 
666,670,742,773,767,805 
Weerasinghe,  A.,  223 
Wegner,  D.  M„  584,  585,  590, 593 
Wei,  J„  579,  589,  590, 593 
Weick,  K.  E„  620,  621, 628,  637 
Weidenmann,  B.,  276 
Weigle,  K.  L.,  66 
Weil,  S.  W„  36, 37 


Weinberger,  A„  228, 237, 409, 423, 
438 

Weiner,  A.,  427 
Weinstein,  A.,  743, 224, 294 
Weinstein,  L.,  248 
Weisfeld,M„  246, 249 
Weiss,  A.,  602 

Weiss,  C.  H„  607,  611,612,  677 
Weiss,  D„  363 
Weiss,  R.E.,  497, 505 
Weitzman,  E.  A.,  775, 803,  805 
Weitzman,  L.,  466,  644 
Welch.  A.  J„  43, 46,  54 
Welhage,  G.  G.,  625,  626,  637 
Wellington,!.,  534, 536 
Wellman,  B .,  245 , 246, 249, 41 1 , 423 
Wells,  K„  32 

Wenger,  E„  28,  82,  75, 79,  98,  104, 
106,  107,709,770,  168,  169, 
777,243,245,249, 254,256, 
264,268, 285,294,427,428, 
435, 436, 438, 439, 474, 484, 
560, 566,  637,  640,  644,  645, 
708  ,  772 

Wenger,  M.  S.,  270, 276 
Wenzel,  S„  439 
Werner,  E.,  498,  504 
Wertheimer,  ML,  9, 20, 96, 407, 423, 
442,456 

Wertsch .  J .  V .,  78 , 82 , 1 65 , 7  77 , 264 , 
268,427,439 
West,  C.K.,  389,  393, 399 
West,  R.E.,  527, 535, 536 
West,  R.  P.,62,  66,  193,797 
West,  S„  500 
Westbrook,  R.  D.,  395 
Westerkamp,  P.,  358, 365 
Wey,  P.,308, 375 
Whalley,  L.  J.,  373 
Wheatley,  M„  696,  707 
Wheatley,  T„  584,  590,  593 
Whitaker,  S.D.,  222 
White,  A.,  326 

White,  B„  217, 224,  282, 292,  331, 
341,344 

White,  B.  Y„  146,  158,767,458, 
459, 460, 462, 463, 465, 467, 
468, 474, 478, 484 
White,  M.,  412, 479 
White,  M.  T„  742 
White,  R.T.,  388, 399 
White,  S.,  255, 257 
Whitehead,  A.  N„  99,  100,  770 
Whitehead,  L.K.,  678,  684 


862 


Author  Index 


Whitehouse,  C.  R„  805 
Whitelock,  D„  358,  363 ,  409, 423 
Whiteman,  M.  C.,  313 
Whitener,  E.  M.,  474, 484 
Whiteshew,  N.,  484 
Whitney,  G.,  227,  231 
Whitson,  J.  A.,  75,  Si,  98, 109 
Whittaker,  S„  224 
Wickens,  C.  D„  88,  96,  290, 294 
Wickstrom,  C.  D.,  229, 231 
Wideman,  H.  H„  613,  617 
Wiemer-Hastings,  K.,  223 
Wiener,  M„  500, 505 
Wiener,  N„  11,20,  126 ,143 
Wieseman,K.C„  487,  505 
Wiesner,  C.,  48,  51 
Wiggins,  G„  386, 399, 487, 505 
Wightman,  D.  C.,  443, 451 , 456 
Wijnen,W.F„  505, 565 
Wilde,  R„  351 ,353 
Wildemuth,  B.  M„  437 
Wilensky,  U.,  462,  468 
Wiley,  D.  A.,  28, 347, 348,  349, 35 1 , 

352 ,353,535, 526,527,528, 

536.684.101.713 
Wilhelm,  P.,351, 353 
Wilkerson,  L.,  487 , 494, 495 , 505, 506 
Wilkie,  K„  487,  504 

Willems,  1.,  253, 257 
William,  D.C.,  498, 506 
Williams,  C.M  .,196 
Williams,  D.,  235, 236,  238, 240, 

416 ,423,759 

Williams,  D.  M„  475, 476, 484 
Williams,  E  .,422 
Williams,  J.M.,  584, 592 
Williams,  M„  141,161,420 
Williams,  M.  D„  758 
Williams,  R.H.,  650.65S 
Williams,  S„  658 

Williams,  S.  M„  325, 486, 506,  656 
Williamson,  D.  M„  600,  601, 604 
Willis,  J„  660,  662,  663, 665,  670 
Willis,  R.E.,  249 
Willis,  S.  C„  779,  780,  805 
Willis-Flurry,  L.  A.,  228, 230 
Wilson,  B.  G„  28, 81,384, 390, 391 , 

398.399.455.101.108.713 
Wilson,  C.D.,  386, 397 
Wilson,  C.  M„  325 

Wilson,  G.M.,  563 
Wilson,  J.  R.,  802 
Wilson,  K.  G„  65 
Wilson,  T.D.,  661, 669 


Wilson  Downie,  W„  563 
Wilson-Pauwels,  L.,  438 
Windschitl,  M.,  340,  344 
Wineburg,  S.,  171 
Winer,  L.R.,  742,  761 
Winkles,  J.,  388,  399 
Winn,  W„  530,  536,  756,  761 
Winn,  W.D.,  40, 53, 61, 66, 218, 224 
Winne,  P.  H.,  127,  128,  131,  133, 
134, 141,  206,  207, 270, 215, 
224,  343 

Winograd,  T.,2S,  635,641, 645, 661 , 
662,663,665 ,669 
Winston,  K.S.,  142 
Winter,  M„  281, 294 
Wintersgill,  M.,  143, 294 
Wirsing,  M.,  361 , 364 
Wisher,  R.  A.,  213,214,  216, 224, 
210,276,616 
Witherspoon,  A.,  222 
Witkin,  H.,  37 
Witt,  C„  524 
Witt,  C.L.,  270,  670 
Witte,  P .,805 
Wittgenstein,  L.,  24, 28 
Wittrock.M.C.,  112, 113,  114,  115, 
116,  118,  119, 123, 124, 140, 
323,325,392,397, 399 
Wittwer,  J.,  564 
Wizer,  D.  R.,  417, 423 
Woerdehoff,  F.  J„  10,20 
Wogulis,  J.,  467 
Wolcott,  H.F.,  775, 805 
Wolf,  G„  650,  658 
Wolf,  P.M.,  399 
Wolf,  S.  A.,  630 
Wolfe,  R.  J.,  661, 669 
Wolfhagen,  I.  H.  A.  P.,  503, 505 
Woloshyn,  V.  E„  121,761 
Wong,  D„  461, 467 
Wood,  D.,  8,  10, 20,  69,  74,  82,  165, 
171,432,437, 439,418,484 
Wood,  H.,  478, 484 
Wood,L.E  .,397 
Wood,  R.E.,  653,  658 
Wood,  T.  J„  495, 506, 563 
Wood,  W„  806 
Woodley,  A.,  261, 268 
Woodruff,  D .,422 
Woodruff,  E„  434, 438 
Woodruff,  S.L,  229, 231 
Woods,  D„  760,  657 
Woods,  D.  R„  487, 491 , 492, 493, 
506,  776,  805 


Woods,  J.H.,  115,  116,724 
Woodward,  C.  A.,  504 
Woodward,  M.  M„  721,  761 
Woodworth,  R.  S.,  445, 456 
Woolf,  B„  287,  288,  292, 294 
Woolf,  H„  784,  806 
Woolley,  D.R.,  14,20 
Worchel.S.,  783,  806 
Working  Party  of  the  Royal  College 
of  Physicians,  560, 566 
Workman,  D.,  502, 505 
Worthen,  B.  L.,  610,  617 
Wright,  G .,536 
Wright,  I„  285, 294 
Wright,  J.,  343 
Wright,  K.C.,  707,  713 
Wright,  K.  E„  660,  662,  663,  665, 
670 

Wright,  S.M.,  566 
Wu,  C.,  183, 421 
Wubbels,  T.,  562 
Wyatt,  J.C.,  560, 566 
Wylie,  D.  G.,  8,  10,20 
Wylie,  M„  505 

X 

Xing,  T„  224 

Y 

Yackel,E„  98,  709, 563 
Yager,  S„  412 ,423 
Yamada,S„  498, 506 
Yamaguchi,  E.,  438 
Yamazaki,  A.,  777 
Yamazaki,  K.,  777 
Yan,  J.,  227 
Yanchar,  S.  C.,  796 
Yanes,M.  J.,  274, 276 
Yang,  M„  280, 294 
Yang,  S„  204, 270 
Yang,  Y„  599,  600,  604 
Yao,K„  388, 399 
Yarbrough,  D.,  224 
Yaverbaum,  G.,  409, 422 
Yeasin,M.,280 ,293,294 
Yee,  L.  W.,  52 
Yee,N„  218, 224 
Yekovich,  C.  W.,  396 
Yekovich,  F.  R.,  396 
Yelland,  N.,  433, 438 
Yellon,  S.  L.,  389, 399 
Yeo.G.B.,  785,  806 


863 


Author  Index 


Yerasimou,  T.,  419 
Yerushalmy,  M.,  155,  156 ,160 
Yeuh,  J.,  409, 412, 423 
Yin,  Y.,  600,  604 
Yokoyama,  T.,  46,  51 
Yost,  G„  643 
You,  S.,  294 

You,  Y„  31, 32,660,670 
Young,  A.  M.,  431 ,439 
Young,  J.R.,  270, 276 
Young,  K.R.,  193 ,197 
Young,  M.F.,  70, 75, 87, 82, 98, 708, 
770,  725,  734,  759 
Young,  R.  M.,  44, 54 
Younger,  M.,  503 
Ypinazar,  V.,  564 
Ysewijn,  P.,  189, 197 
Yueh,  H.-P.,  507 
Yukl,  G.,  621, 657 


z 

Zacharia,  Z.  C.,  467 
Zachos,  P.,  465, 468 
Zamarro,  J.-M.,  342 
Zander,  U„  242, 248 
Zanottie,  M.,  483 
Zapata-Rivera,  D.,  281, 284, 294 
Zarraonandia,  T.,  360, 365 
Zawacki-Richter,  O.,  266 
Zech,  L.,  708, 534,  656 
Zeitz,  C.  M„  583 ,593 
Zelinsky,  A.,  286, 294 
Zembal-Saul,  C„  434,  438 
Zemke ,  R.,  296, 303,  660,  669,  670, 
685 ,690 
Zeni,  J.,  230, 257 
Zhang,  J.,  338, 344 
Zhang,  M.,  482 


Zickar,  M.  J.,  435, 439 
Ziegler,  A.,  134,747 
Zimitat,  C.,  495, 506 
Zimmerman,  B.  J.,  1 12,  1 14,  121 , 
724,206,207,270,478,484 
Zimmerman,  C.,  322, 327 
Zimmerman,  E.,  105,  106,770,218, 
224 

Zisman,  S.  B.,  19 
Zittle,F„  264, 266 
Zittle,F.J„  41 1,420 
Ziv,  A .,326 
Zlotolow,  S.,  27 
Zsambok,  C.  E.,  308, 315 
Zuber-Skerritt,  O.,  264, 268 
Zumbach ,  J . , 20 1 , 202 , 203 , 204 , 27 0 , 
489,499,506 
Zwahr.M.,575 


864 


Subject  Index 


A 

abductive  reasoning,  308 
abstract  concepts,  310 
abstract  knowledge,  545-549 
abstract  learning  object  content 
model  (ALOCoM),  350 
abstract  mathematical  knowledge, 
defined,  539 

abstraction,  347,  540,  554,  583,  634 
absurdity,  reduction  to,  45 
academic  play  spaces,  105-106 
academic  risk  taking,  219 
Academic  Talk,  414 
access,  271 
accommodation,  78 
accommodators,  36,  310,311 
accountability,  407, 418 
accreditation  of  prior  learning,  813 
accretion,  76 
achievement 

adaptive  technologies,  and,  280 
assessment,  264 
attitudes,  and,  192 
audio- video  redundancy,  and,  43 
computer-supported  cooperative 
learning ,  and ,  402 , 408-4 1 0 , 
411,417 

cooperative  learning,  and, 
408-410, 555 
concept  mapping,  and,  121 
electronic  performance  support 
systems,  and,  522 
feedback,  and,  134 
first  principles,  and,  177 
group  composition,  412 
individual  accountability,  and 
407 

learner,  measurement  of  with 
paper  and  pencil,  12 
learner  variables,  and,  190 


library  media  programs,  and, 
233-239 

macro-instructional  strategy 
research,  and,  722-723 
matched/mismatched  instruction , 
and, 307 

mentoring,  and,  430 
personalized  system  of 
instruction,  and  192 
prior  knowledge,  and,  306 
problem-based  learning,  and, 
489,490,496,498 
programmed  materials,  and,  190 
public  recognition  of,  324 
self-regulation,  and.  121 
social  interdependence  theory, 
and, 406, 407 

successful  working,  and,  58,  59 
technology,  and,  612 
acquired  capabilities,  387 
acquisition  noise,  94 
act  in  context,  as  root  metaphor,  59 
ACT  Pro,  788 

ACT-R  cognitive  model  of  learning, 
584,785 

activation  principle,  defined,  174 
active  cognitive  engagement,  206-208 
active  knowledge  construction,  206 
active  learning,  defined,  33,  34 
active  modeling,  291 
active/reflective,  310,  31 1 
activist,  311 
activities  analysis,  774 
activity-centered  instruction,  385 
activity  theory,  25 
adaptability,  474, 475, 476,  477 
adaptable  systems,  475 
adaptation 

content-level,  476 
lethal,  624 
link-level,  476 


adaptive/adaptable  hybrid  system, 
475,476,479 

adaptive  computer-based  training, 
213-214 

adaptive  control,  410 
adaptive  cycle,  four-process, 
280-282 

adaptive  educational  hypermedia 
system  (AEHS),286 
adaptive  feedback,  138-139 
adaptive  hypermedia  system  (AHS), 
286, 471, 475— 477 
defined,  470 

limitations  and  challenges  of, 
476-477 
taxonomies,  476 
adaptive  instruction,  810 
adaptive  instructional  systems, 
469^179 

approaches  to,  471 
defined,  470 
history  of,  470^17 1 
adaptive  learning  environments, 
286-287 

Adaptive  Learning  Environments 
Model  (ALEM),  471 
adaptive  learning  principles,  358 
adaptive  navigation  support,  476 
adaptive  open  resources,  477 
adaptive  presentation,  476 
adaptive  systems 

metacognition-based,  478M-79 
motivation-based,  478 
vs.  adaptable  systems,  475 
adaptive  technologies,  277-291 
current,  282-287 
distributed,  281 
experts’  thoughts  on,  287 
future  of,  287-29 1 
rationale  for,  279-280 
stand-alone,  281 


865 


Subject  Index 


adaptive  Web-based  hypermedia,  475 
adaptive  Web  recommendation,  477 
adaptivity,  475, 476 
defined,  278 

ADDIE,  see  analysis,  design, 

development,  implementation, 
and  evaluation 
AdeLE, 285 

adult  learning,  442, 443, 444-445 
Advanced  Distributed  Learning,  220 
Advanced  Instructional  Design 
Advisor,  709 

Advanced  Instructional  Design 
Advisor  (AIDA),  393 
adventure  learning  programs,  416 
adversarial  reasoning,  554-555 
affective  computing,  285,  809 
affective  states,  differences  in,  280 
affordances  ,254,259,261-262,265, 
270,274,338,811 
after-school  mentoring  program,  430 
agents ,  2 1 7-2 1 8 , 243 , 247 , 28 1 , 29 1 , 
312,334,336,462,478,722, 
734 

artificial,  290 
emotional,  285 
mining  by,  291 
pedagogical,  285, 417 
personal,  290 
teachable,  285 
aggregation,  of  data,  778 
AIM  (Authoring  Instruction 
Materials),  678 

algoheuristic  theory  of  instruction, 
393 

Alien  Rescue,  735 
Alphie’s  Alley,  655 
alternate  forms  method  of  test 
reliability,  769 

Alternate  Reality  Kit  (ARK),  459 
America’s  Army,  218 
American  Association  of  School 
Librarians  (AASL),  235 
American  Educational  Research 
Association,  239 

American  Evaluation  Association, 
615 

American  Psychological  Association 
(APA),  15 

learner-centered  principles  of, 
610 

American  Society  for  Training  and 
Development,  523 
analog-to-digital  boards,  797 


analysis  constructed  dataset  creation, 
778 

analysis-constructed  shared  mental 
model  (ACSMM),  781 
analysis  of  interaction 
communication,  780 
analysis  of  variance  (ANOVA),  721, 
722, 723 

analysis,  design,  development, 

implementation,  and  evaluation 
(ADDIE),  158,  589,  677,  740, 
741,754 

analyst,  instructional  design,  570 
analytical  science,  defined,  648 
anchored  instruction,  15,79, 
100-101, 149,319-320 
anchors,  scaffolding,  434 
andragogy,  442, 443, 444-445 
animated  pedagogical  agents,  213, 
217-218 
defined,  212 

animation,  178,  338,  214-215,  809 
design  guidelines,  94 
narration,  and,  43 
anonymity 

electronic  communication,  and, 
411 

of  social  mediator,  227 
answer-until-correct  feedback,  132, 
138 

anthropomorphized  interfaces ,  809 
anticipation  ratio,  780 
antididactical  inversion,  547 
anxiety,  working  memory  and,  49 
apathetic  users,  202-203 
Apple,  Inc.,  design  and  development 
and, 650 

application  principle,  defined, 
174-175 

application  problems,  100 
application  program  interfaces 
(APIs),  794 

apprentices,  medical,  559,  560 
apprenticeship,  106, 425-437,  810 
cognitive,  101-102 
defined,  33-34, 426,  539 
medical,  559,  560 
appropriation,  78 
aptitude,  defined,  472 
aptitude  variables,  474 
aptitude-treatment  interaction,  214, 
307,471,472-473,479,731 
courseware,  eight-step  model  for 
designing,  472 


defined,  470 
limitations  of,  472-473 
Architecting  Business  Change,  453 
areas  of  interest  (Aols),  788 
argument,  defined,  554 
argumentation,  413-414,  686,  688 
ARIADNE  Educational  Metadata 
Recommendation,  349 
ARIS,  334 
Aristotle,  5 
ARK,  459 
art  worlds,  168 
Articulate  authoring  tool,  674 
articulation,  101, 427, 434 
defined,  426 

artifacts,  255,  356,  361, 461, 541 , 
552,637,640,654,694,771, 
773,774,781,812 
defined,  685 
design,  685-690 
examples  of,  774 
process,  686 
product,  686,  687-689 
artificial  agents,  290 
artificial  intelligence,  102,  169,  207, 
242-243, 246, 285, 395, 474, 
641 

-based  knowledge  representation, 
599 

artificial  science,  665 
assessment,  811 
authentic,  784 

blended  learning  environments 
and, 271 

competencies,  and,  682-683 
complex  performance,  783-789 
criteria,  784-785 
defined,  765 
defined,  596 
learning,  770,  811 
defined,  765 
individual  processes, 
767-776 

vs.  performance,  766-767 
learning  processes 
group, 776-783 
individual,  767-776 
learning  vs.  performance, 
766-767 

medical  education,  562 
outcomes-based  education,  and, 
386 

performance,  see  performance 
assessment 
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problem-based  learning,  495-496 
self-directed  learning,  and,  447 
standards,  765,  784-785 
vs.  evaluation,  562 
Assessment  Design  and  Delivery 
System  (ADDS),  600 
assimilation,  78 
assimilators,  36,  310,  31 1 
Association  for  Educational 
Communications  and 
Technology  (AECT),  5,9,11, 
234,655,740,808 
associationism,  530 
associationists,  445, 451 
associative  learning  skills,  309 
astronomy,  learning,  550 
asymmetric  warfare,  244 
asynchronous  communication,  653 
asynchronous  discussion,  226,  228, 
434 

asynchronous  learning,  259,  260, 
261,264,274,409-410,415, 
416 

defined,  672 
Athens,  5 
Atlas  .ti,  615 

atomistic  models,  442, 443, 446,  812 
attention 

Baddeley’s  memory  model,  and, 
88-89 

cooperative  learning,  and,  409 
defined,  112 
attitudes,  809 
measuring,  771 
medical  student,  557 
of  potential  adopters,  621 
attribute  isolation  elaborated 
feedback,  132 
ATutor,  358 

audio- video  redundancy,  40, 42-43 
audio-visual  instruction,  7,9,  11 
auditory  design  guidelines,  92-94 
Auditory  Discrimination  in  Depth 
Program,  544 
authentic  assessment,  784 
authentic  problems,  581 
authentic  tasks,  450, 453, 454, 488, 
784 

authenticity,  99-100,  102,  104,  556 
defined,  98 
ecological,  104 
factual,  100 

learning  experience,  322 
participation  model  for,  98,  105 


procedural/process,  100 
task,  100,  784 

Authoring  Instruction  Materials 
(AIM),  678 

authoring  tools,  673-675, 679,  680 
defined,  672 
templates  in,  674 
autobiographical  memory,  48 
automated  instructional  design 
(AID),  672,  676,  678,  681 
defined,  672 

automated  knowledge,  defined,  578 
automated,  unconscious  expert 
knowledge,  590 
automaticity,  584 
automation,  785 
AutoTutor,  216,  218 
avatars,  336,  340,  736 

B 

Baby  Boom  Generation,  296,  297, 
299,300,301,302 
cuspers,  297 
backward  chaining  with 
snowballing,  451 
Baddeley’s  memory  model,  87, 
88-89,  94,  95 
defined,  86 

Balloon  Car  Challenge,  103 
bandwidth,  of  learner  interactions, 
290 

Bayesian  networks,  284,  312, 473, 
475,601 

bazaar  model,  for  open  source 
software,  357,  361 
BBC,  10,  17 
behavior  change,  629 
behavioral  engineering,  510,518 
models,  511-515 

features  of,  5 1 4-5 1 5 
interventions  in,  514 
behavioral  learning  theory,  15,  189, 
405,406,720 

behavioral  psychology,  4,  62,  63 
behavioral  technologies,  12 
behaviorism,  16,  24,  69,  72,  73,  76, 
77,98,112,122,165,189,203, 
510,511,530,718 
defined,  68 

dualism  perspective,  73 
epistemology  of,  73 
ontology,  73,  77 
pragmatic  social,  69 


radical,  11-12 
unit  of  analysis,  73,  77 
behaviorists,  74,  530,  636,  708 
Belvedere,  409, 414, 415 
Betty’s  Brain,  285 
bias,  learned,  338 
bio-instrumentation.  794 
biologically  based  devices,  285 
black  boxes,  331, 334,  746 
Blackboard,  18,226,359 
blackboards,  use  of,  6 
blended  learning,  252, 269-274 
access,  and,  271 
cost  effectiveness  of,  271-272 
defined,  270 

learning  effectiveness,  and, 
270-271 

organizational  considerations, 
273 

research  directions,  273-274 
satisfaction,  and,  272-273 
blogs,  16,  227,  228,  229,  264,  415, 
416,527,529,606,736,809, 
810 

defined,  226 
Bodington,  358 
boredom,  49,  50,  51, 280 
borrowing  principle,  371-372,  374, 
375,376,377,379 
boundary  interdependence,  406-407 
branching  programs,  programmed 
instruction,  190 
Brewster,  David,  6 
brick-and-mortar  metaphor  for 
learning  objects,  348 
British  Open  University,  17 
bug-related  elaborated  feedback,  132 
bus,  digital  data,  797 
business  process  redesign/ 

reengineering,  696,  699-700 

c 

CAI/CBI-W,  413 

Canada,  educational  television  in,  10 
Canadian  National  Railways  (CNR), 
8 

CASCADE-SEA,  739 
case-based  reasoning,  101, 102-103, 
553 

case  studies,  740, 742, 75 1 , 752, 774, 
775 

CASES,  156-157 
cassette  audio  recorders,  9 
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cathedral  model,  for  open  source 
software,  357 
cathexis,  positive,  406 
causal  explanations,  775 
Causal  Mapper,  152,  158 
CBC,  10 

CDT,  see  Component  Display  Theory 
Center  for  International  Scholarship 
in  School  Libraries  (CISSL), 
237, 238-239 
chaining,  451-452 
change 

agentry,  see  change  agentry 
agents,  619-630 
behavior,  629 
continuous,  620,  621 
defined,  620 
developmental,  621 
episodic,  620,  621 
facilitators,  623 
first-  vs.  second-order,  621 
individual,  620 
organizational,  620 
teacher,  624 
theories,  620-622 
change  agentry 

corporate,  628-629 
defined,  620 

empirical  studies  of,  622-629 
government,  627 
healthcare,  629 
K-12, 622-627 
theoretical  foundations  of, 
620-622 

developmental  change,  621 
diffusion,  621-622 
individual/ organizational 
change,  620-621 
chaos  theory,  624 
character/word  decoding,  200 
chat  rooms,  15,226,  227,  229,  410, 
415 

cheating,  300 
checklists,  768,  770 
Chelsea  Bank  program,  101 
chess 

expertise  and,  371 
long-term  memory  and,  371 
Chronicles  of  America  Photoplays ,  7 
chunking,  88, 433,  785 
CIAO ! ,  609 

CISSL  Virtual  Research  Community, 
239 

civil  law,  552,  553 


classroom  learning  communities, 
104 

clinical  education,  defined,  539 
closed-ended  questionnaires, 
780-781 

closed  source  software,  357,  361 
CNR,  see  Canadian  National 
Railways 

coaching,  181,427,430,431,433 
defined,  426 
cocktail  party  effect,  94 
coding  data,  775 
cognition 

embodied,  730 
human,  see  human  cognitive 
architecture 
in  the  wild,  734 
situated,  69,  734 
sketching,  and,  686-687 
technologies  linking  with 

learning  and  instruction, 
305-313 

cognitive  abilities ,  307-309 ,321;  see 
also  cognitive  styles,  cognitive 
traits 

defined,  306 

cognitive  apprenticeship,  101-102, 
425-437,810 
defined,  426,  427-429 
enactment  of,  429-430 
future  research  on,  436 
higher  education,  and,  429M-30 
K-12  education,  and,  430 
model,  709 

newcomer  adjustment,  435 
research  on,  429^-30, 436 
cognitive  architecture,  human. 
369-380 

cognitive  congruence,  494 
cognitive  constructivism,  69,  72,  74, 
78 

defined,  68 

dualism  perspective,  74 
epistemology,  74,  78 
ontology,  74,78 
unit  of  analysis,  74 
cognitive  cycles,  44 
cognitive  demands,  598 
cognitive-developmental  theory  of 
cooperative  learning,  405-406 
cognitive  disequilibrium,  49,  219, 
405 

cognitive  engagement,  48 
cognitive  feedback  functions,  133 


cognitive  flexibility  theory,  199,200, 
206 

cognitive  load,  201, 204,  214,  215, 
307,308,337,339,785-786, 
787 

extraneous,  42,  339,  374,  375, 
376,377,378 
extrinsic,  339 

germane,  339,  374,  378, 447 
intrinsic,  339,  374,  378,  379 
task  complexity,  and,  783 
types  of,  88 

cognitive  load  theory,  39, 40, 42, 43, 
44,48,49,50,87,88,95,370, 
374-380,522,720,721 
borrowing  principle,  and,  372 
defined,  86,  370 

message  design  research,  and,  723 
cognitive  modeling,  41, 283-284 
sound,  instructional,  93 
text  comprehension,  91 
visual  design,  92 
cognitive  models,  defined,  86 
cognitive-motivational  performance 
improvement,  510,  511, 
515-519 

features,  518-519 
feedback,  and,  516-517 
goal  theory,  and,  515-516 
incentives,  and,  517-518,  522 
cognitive  multimedia  learning,  41—44 
cognitive  overload,  88,217, 308, 453 
cognitive  processing,  automatic  vs. 
controlled,  445 

cognitive  psychology,  40,  41 , 530 
learning,  and,  44-48 
Cognitive  Revolution,  10 
cognitive  scaffolds,  459 
cognitive  schemas,  445, 446 
cognitive  semantics,  641 
cognitive  skills,  cognitive  task 
analysis  and,  585 

cognitive  strategies,  for  reading,  545 
cognitive  structuring,  427, 432 
cognitive  styles 

matched  vs.  mismatched,  307 
of  learners,  204,  205-206,  272, 
280, 284 

cognitive  systems,  549 
cognitive  task  analysis,  136,  245, 
577-591 

4C/ID, and, 587, 588-589 
cost-benefit  studies  of,  586-587, 
590-591 
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declarative  knowledge,  and, 
583-584 
defined,  578 
families,  579 

first  principles  of  instruction, 
and, 589-590 
methods,  579-583 

analyze  and  verify  data,  582 
collect  preliminary 
knowledge,  580 
format  results,  582-583 
identify  knowledge 
representations, 
580-582 

procedural  knowledge,  and,  584 
research,  583-587,  589-591 
training  design,  and,  587-588 
types  of,  579 

cognitive  tasks,  performance  on, 
783-784 

cognitive  tools,  733,  734 
cognitive  training  model,  178-179 
defined,  174 

cognitive  trait  model,  311-312 
cognitive  traits,  307-309 
identifying  ,311-312 
learning  styles,  and,  312-313 
cognitive  underload,  88 
cognitivism,  14-15,  16,  24,  35, 69, 
72, 73-74, 77-78, 165 
defined,  68 

dualism  perspective,  74 
epistemology,  73-74,  77,  78 
ontology,  73-74,  77 
unit  of  analysis,  74 
coherence 

as  truth  criterion,  58 
hypertext  and,  204 
multimedia  learning,  and,  214 
cohesion  analysis,  780 
Co-Lab,  335, 462, 463^164, 465 
collaboration,  164,  323,404 
defined,  164 

Collaboration  Management  Cycle, 
479 

Collaborative  and  Multimedia 
Interactive  Learning 
Environment  (CaMILE), 
434^135 

Collaborative  e-Learning  Structures 
(CeLS),  154 

collaborative  filtering,  247 
collaborative  group  processing,  494 
collaborative  inquiry  (Cl),  627 


collaborative  knowledge 
construction,  213 

collaborative  learning,  15,  163-170, 
176,286,335,404,479,575, 
779 

defined,  402 

participatory  design,  and,  663-664 
problem-based  learning,  and, 
492,498,500 

collective  data  collection,  defined, 
765 

Colorado  Study,  235,  237 
Comenius,  6 
commitments,  106 
common  law,  552,  553 
communication,  406 
analysis,  780 

asynchronous,  410, 415, 434, 
529, 653 

electronic ,  407 , 4 1 0 , 4 1 1 
face-to-face,  407,  411,414, 464, 
653 

group,  four  modes  of,  653 
multi-  vs.  single-channel,  43 
nature  of,  229 
online,  464 
synchronous,  529 
technology,  12 
tools,  529 
verbal,  41 

communications  theory,  24,  25 
communities  of  learners,  104 
communities  of  practice,  106-107, 
245,323,429,640 
clinical,  560 
defined,  426 
design  of,  436 
elements  of,  428 
identity  development,  and, 
435-436 

membership  in,  427, 428-429 
types  of,  428 
research  on.  435^136 
community  of  inquiry  model,  274 
Community  of  Teachers,  106 
comparison  of  generative  and 

supplantive  strategies  (COGSS) 
model,  392 

compartmentalization.  442 
compensatory  education,  543 
competence-based  education,  453, 
742 

competence  development,  446-447 
defined,  442 


competences,  446, 447, 448 
medical,  557,  559,  560 
competencies,  813 

assessment,  and,  682-683 
team,  653 
competency 

defined,  569,  571 
standards,  570,  572-573 
validation,  572 

competitive  learning,  404, 406, 407, 
408,409,413 
defined,  402 
completion  rates,  271 
complex,  defined,  29 
complex  learning,  81 1 
complex  performance 
assessment,  783-789 
defined,  765 

laboratory  for  measuring,  789-800 
complex  phenomena,  30-3 1 
five  attributes  of,  30 
complex  problems ,  308 ,  3 1 1 , 49 1 , 
492,499 

complex  skills,  443, 446, 454 
complex  tasks,  442, 443, 444, 446, 
448,449,451,452,453,454, 
516,518,579,584,768 
defined,  578 

complexity,  26, 46,  59,  88,  101 
cognitive  load,  and, 42, 339, 374, 
522 

concept  map,  206 
constructivist,  60 
content,  86,  88,  99,  132, 203 
dynamic  modeling,  336 
dynamic/static,  43 
epistemological,  340 
feedback,  132 
granularity,  and,  337 
hyperspace,  308 
memory,  and,  44 
modeling  tasks,  and,  336-339 
task,  130, 139, 336-339,427,522 
theory,  29-32 
complicated,  defined,  29 
component  display  theory  (CDT), 
25,390,391,393 
component  investigation,  743 
defined,  717 
comprehension,  115 
reading,  544,  545 
Comprehensive  Application  of 

Behavior  Analysis  to  Schooling 
(CABAS),  63 
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computational  ability,  of  humans,  44 
computer-aided  personalized  system 
of  instruction  (CAPSI), 
431—432 

Computer  as  Learning  Partner,  147 
computer-assisted  instruction,  8,  12, 
14,77,731,732,735 
Computer- Assisted  Qualitative  Data 
Analysis  (CAQDAS),615 
computer-based  assessment,  599, 600 
computer-based  instruction  (CB1), 
728,734 

computer-based  scoring,  600-601 
computer-based  training,  213-214, 
409,673,676 
costs  of,  220 

metacognition,  and,  433^-34 
computer  conferencing,  416 
computer-mediated  communication, 
225-230,431,728 
blended  learning,  and,  274 
defined,  226 

guiding  discussion  with,  229 
research  goals,  228-230 
text  handling,  226-227 
computer-mediated  technologies, 
211-220 

communications,  225-230, 
770-771 

computer  models,  458-463 
computer-supported  collaborative 
learning  (CSCL),  166-167, 
219,404,414-417,779 
ability,  and,  412 
achievement,  and,  408-410 
attitudes  toward,  411-412 
defined,  402 
future  of,  417-418 
gender,  and,  413 
group  composition,  and,  412 
individual  differences,  and, 
412-413 

relationships,  and,  410-41 1 
research  on,  408-414 
scaffolding,  and,  434-435 
social  competencies,  and,  41 1 
computer-supported  cooperative 
learning,  see  computer- 
supported  collaborative 
learning 

Computer-Supported  Intentional 
Learning  Environment 
(CSILE),  166,  168,229,409, 
415,434 


computers ,  see  also  digital  age 
adoption  of  by  schools,  16 
as  instrumentation,  799-800 
cognitive  modeling,  and,  41 
concept  maps,  1 15,  121, 201, 202, 
203,204,461,600,781 
learning  law,  and,  556 
concepts,  processes,  and  principles 
(CPPs),  581,582, 588 
conceptual  landscapes,  200,  205 
conceptual  model,  defined,  539 
Concerns-Based  Adoption  Model 
(CBAM),  621 

conclusion,  inquiry  learning,  459, 
465 

concrete/abstract,  310 
concurrent  reporting,  787 
conditions-based  instruction, 
386-389, 395 
defined,  384 
examples  of,  389-394 
Gagne, and,  389-390 
conditions-of-learning  paradigm  of 
instructional  design,  384 
Cone  of  Experience,  9 
Conference  on  the  Web  (COW),  431 
conferencing,  416 
confidence,  280 

problem-based  learning,  and,  492 
confusion,  learning  and,  49,  219 
CONNECT,  414 
connectionist  models,  40, 41 
Connexions,  346,  352 
connoisseurship.  608 
consecutive  models,  Gagne’s,  710 
consistency,  internal,  769 
constant  comparative  method,  772 
constraint  networks ,  45 
construct-related  validity,  769 
construction-integration  model,  91 , 
199,200,201 
defined,  86 

constructive  controversy,  404, 405, 
413-414 
defined,  402 

constructivism,  15-16,  24, 49, 
60-62,69,72, 112, 122,392, 
429,431,471,477,488,531, 
733,809 

contextualism,  and,  60-61 
defined,  5,  539 
radical,  15,60,61,69,72,74 
social,  15,60,61,69,74 
sociocultural,  72 


constructivist  adaptive  systems, 
477^178 

constructivist  learning  principles, 
810 

constructivists,  530,  708,  709,  730, 
756 

science  learning,  and,  549 
consultants,  external,  628 
content,  808 

analysis,  774,  780 
-centered  instruction,  385 
experts,  812 
fuzzy,  573 
knowledge,  449 
layer,  636-637 
-level  adaptation,  476 
management  systems,  289 
open, 810 

-related  validity,  769 
retention,  problem-based 
learning  and,  490-491 
space,  166 

context,  31, 59,  70,  72,  74,  75,  107, 
427,528-529 
CIAO ! ,  609 
external,  528,  533 
individual,  528 
macro-  vs.  micro-level,  386 
negotiated,  528 
team,  653-654 

user-centered  design,  and,  661 
context,  input,  process,  and  product 
(CIPP) ,  607 

context,  interactions,  attitudes,  and 
outcomes  (CIAO!),  609 
context-sensitive  design,  665 
context-specific  models,  547 
contextual  cues,  word  meanings,  and, 
544 

contextual  design,  662 
contextualism,  55-64,  69 
analytic  goal  of,  59 
constructivism,  and,  60-61 
defined,  55 
descriptive,  59 
defined,  56 

instructional  design,  and, 
60-62 

functional,  59-60 
defined,  56 

instructional  design,  and, 
62-64 

root  metaphor,  59 
varieties  of,  59-60 
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Continental  Classroom,  10 
contingency  management,  427, 432, 
510 

contingent  teaching  systems,  478 
continuous  change,  620,  621 
continuous  events,  697 
control  element,  129 
controlled  attention,  44 
controlled  process,  128,  129,  130, 
132 

controlled  testing,  743 
controlled  variables,  128,  129,  130, 
131 

controller,  system,  126,  128, 129, 
130, 131 

controversy,  constructive,  402, 404, 
405, 4 1 3 — 414 
convergers,  36,  310,311 
conversational  threads,  219 
CoolModes,  335 
cooperation 

asynchronous  vs.  face-to-face, 
409-410 

basic  elements  of,  406-408 
computers,  and,  415-417 
programs  developed  to  promote, 
415 

cooperative  base  groups,  404, 405 
cooperative  inquiry,  662 
cooperative  learning,  404,  721, 
722-723,724 

achievement,  and,  408-410,  555 
attitudes  toward,  411-412 
computer-supported,  404, 
414-417;  see  also 
computer-supported 
collaborative  learning 
(CSCL) 

future  of,  417-418 
research  on,  408-414 
cost  effectiveness  of,  417 
defined,  402 
formal,  404-405 
informal,  404, 405 
nature  of,  403M-05 
single-user  programs  reapplied 
to,  415 

technology-supported,  40 1  — 4 1 8 
theoretical  foundations  of, 
405-408 

types  of,  402, 404-405 
Web-based,  416 
copyrights,  357 

learning  objects  and,  351 


core  reference,  defined,  808 
core  values,  697 
correspondence  education,  17, 
260-261 

correspondence,  as  truth  criterion, 
57,58 

COSMO,  478 

cost  effectiveness,  blended  learning, 
271-272 

course  management  system  (CMS), 
18 

CourseBuilder,  674 
cover  story,  goal-based  scenarios 
and,  449 

covert  responses,  190 
CoVis,  409, 415 

Creative  Commons  licenses,  352 
CRIME,  430 

criss-crossing  the  landscape,  710 
criterion-related  validity,  769 
critical  decision  method  (CDM),  582 
CS383, 312 

CSILE,  see  Computer-Supported 
Intentional  Learning 
Environment 

cue-summation  theory,  40, 43 
cues,  cognitive  task  analysis  and, 
584-585,589-590 
culture,  systems  design  and,  695 
cumulative/persistent  student  model, 
284-285 
curiosity,  49 
curriculum  design 
defined,  486 
medical,  561 
problem-based  learning, 

496-498 

curriculum,  embodied,  97-108;  see 
also  embodied  curriculum 
cuspers,  297 
cybernetics,  126,  127 
defined,  126 

cycle  of  instruction,  defined,  174 
cycles,  community  of  learners,  104 

D 

Darwin  Information  Typing 
Architecture  (DITA),  350 

data 

analysis,  see  data  analysis 
coding,  775 

collection,  see  data  collection 
-driven  scoring,  600,  601 


interpretation,  inquiry  learning, 
459 

management,  775 
neuroscientific,  788 
performance  process,  785-788 
performance,  collecting, 
785-788 

product,  collection  of,  785 
reduction,  774-775 
sources,  613-615 
verbal  protocol,  787,  788 
data  acquisition  device  (DAQ),  793, 
797-799 

data  analysis,  613-615,  763-800 
conceptual  methods  for,  781 
direct  process,  778-780 
indirect  process,  780 
qualitative,  774-775 
techniques,  778-782 
data  collection,  763-800 

conceptual  methods  for,  781 
direct  process,  778-779 
indirect  process,  780 
techniques,  778-782 
Decatur  Township  school  district 
GSTE,  and,  697 
participatory  design  and,  664 
decision-making  evaluation,  607 
decision  points,  cognitive  task 
analysis  and,  585 
declarative  knowledge,  242,  246, 
387,390,393,583-584,785 
defined,  578 
declarative  learning,  446 
decoding,  word-level,  543,  544.  545 
deduction,  defined,  308 
deductive  reasoning,  308 
DEEP,  781 

default  stereotyping,  284 
del.icio.us,  349,  350 
delay  retention  effect,  137 
Delphi  techniques,  740 
demonstration,  181 
experiments,  727 
principle,  defined,  174 
DEMOS,  662 

dental  education,  problem-based 
learning  and,  491 
Department  of  Audio-Visual 

Instruction  (DA VI)  ,7,9,11,12, 
234 

Department  of  Visual  Instruction,  7 
depression,  computer-mediated 
communication  and,  229 
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descriptions,  of  collaboration,  164, 
165, 167, 168-170 
descriptive  accounts,  defined,  164 
descriptive  contextualism,  59 
defined,  56 

instructional  design,  and,  60-62 
descriptive  theories,  708-709 
descriptive  vs.  prescriptive,  26,  61 
desiderata,  695 
design  and  development 

online  instruction,  671-683 
process-oriented,  660,  661 
product-oriented,  660 
research,  see  design  and 
development  research 
teams,  see  design  and 
development  teams 
user-centered,  659-668 
design  and  development  research. 
718,748-757 
defined,  717 

dual  roles  of  participant  and 
researcher,  754 
methodologies,  750-752 
model  use,  754 
model  validation,  and,  753 
problems,  749-750 
recall  data,  and,  754-755 
trends  in,  755 
types  of,  749 

vs.  teaching-learning  research, 
752 

design  and  development  teams;  see 
also  teams 

challenges  for,  654-655 
defined,  648 
evaluation  of,  654 
examples  of,  650 
purpose/goals  of,  654-655 
social  consequences  of,  647-656 
stabilizing,  652 

design-based  research,  167,394,434, 
572-573,727, 734-735, 739, 
745,747 

design  by  repair,  665 
design,  idealized,  695 
design  in  the  large  (DIL),  649,  655, 
656 

defined,  648 

design  in  the  small  (DIS),  649, 655, 
656 

defined,  648 

design  instrumentalities,  687 
design,  iterative,  see  iterative  design 


design  jury,  688 
design  languages,  633-643 
benefits  of,  643 
cognitive  semantics,  and,  641 
computer  science,  and,  641 
content  layer,  636-637 
defined,  634 
history  of,  641-642 
innovation,  and,  638 
instructional,  636-638 
linguistics,  and,  641 
natural  languages,  and,  635-636, 
642 

range  of,  638-639 
research,  642-643 
social  learning  theory,  and, 
640-641 

sociolinguistics,  and,  640-641 
specialized,  639 
study  of,  639-641 
variation,  dimensions  of,  639 
design  layers,  635 
defined,  634 
instructional,  636-638 
design  paradigms,  742 
design  patterns,  instrumentation, 
790-797 

design  principles,  defined,  146 
Design  Principles  Database,  147-158 
pragmatic  principles,  147-157 
design  principles,  media,  86-89 
design  research,  734,  739,  745-746, 
747 

characteristics  of,  746 
defined,  746 

design  rules  of  thumb,  151 
design  science,  defined,  648 
design  space,  662 
design  theory,  692,  693,  694-695, 
696, 700 
defined,  692 
design  way,  693 
Designer’s  Edge,  677-678 
designers-by-assignment,  812,  813, 
814 

Designing  Instructional  Systems,  13 
Developing  Design  Documents  (3D), 
637-638 

development  model,  defined,  570 
development  studies,  746,  747-748 
developmental  change ,  62 1 
diagrammatic  techniques,  245 
Dienes  blocks,  546 
diffusion,  theory  of  change,  621-622 


digital  age,  16-18,  526 
digital  natives,  272 
vs.  immigrants,  571 
dilemma,  problem-based  learning 
tutors,  493 

Direct  Instruction,  12,  63, 194,  586 
direct  process  data  collection/ 
analysis,  778-779 
direct  process  measures,  777-778 
defined,  765 

direct  transfer  models  of  instruction, 
403 

directed  distance  learning 
environments,  532 
disabilities,  students  with,  271, 280, 
286,543 

disaggregator,  350 
disciplined  perception,  166 
discourse  analysis,  780 
discovery  learning,  10,  78, 176,  340, 
539-540, 586 
environment,  defined,  539 
discovery  methods,  810 
discrete  events,  697 
discussions 

asynchronous,  226, 228, 434 
computer-mediated 

communication,  and,  229 
focused,  405 

threaded,  15,  226,  227, 246, 435 
turn-to-your-partner,  405 
disequilibrium,  49 
dissemination,  of  evaluation 
findings,  615-616 

distance  education,  12,  17, 252, 261, 
411,532,722,729 
collaborative  learning,  and,  167 
defined,  5 

first  principles,  and,  177 
problem-based  learning,  and,  498 
distance  ratio  formula,  781 
distributed  evaluation,  609 
distributed  learning,  269,  281 
distributed  perspective 
defined,  164 

on  collaborative  learning,  163-170 
distributed  problem-based  learning 
(dPBL),  498 

distributed  scaffolding,  432, 436 
distributed  teams,  651 , 653 
divergers,  36,  310,  311 
Division  of  Visual  Aids  for  Military 
Training,  8 

document  analysis,  771, 773,  774 
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Dokeos,  358 

domain  knowledge,  463, 489-490 
domain  modeling,  458,  600,  601 
domain  representation,  598-599 
domain-specific  instruction,  537-562 
domain-specific  problems,  540-541 
domain-specific  theories,  defined, 
539 

doodling,  687 

dual-code  theory,  39, 40, 42, 43, 
307-308 

dual-coding  theory,  87,  88,  95,  205 
defined,  86 

dual-process  memory,  40, 44 
dual  roles,  of  researcher  and 
participant,  754 
dualism,  70 

behaviorism,  and,  73 
cognitive  constructivism,  and,  74 
cognitivism,  and,  74 
defined,  68 

situativity  theory,  and,  75-76 
sociocultural/historicism,  and,  75 
Dublin  Core  metadata  standard,  349, 
359 

durability,  447 

dynamic  information  structures,  201 
dynamic  modeling  tools,  330, 
331-336,338 
classification  of,  331-336 
dynamic  modeling,  see  modeling: 
dynamic 

dynamic  pretest,  682 
defined,  672 

Dynamical  Modelling  System  (DMS), 
461 

dyslexia,  543 

E 

echoic  memory,  94 
eclecticism,  57 
Eclipse,  358 
Ecolab,  478 

ecological  authenticity,  104 
education 

compensatory,  543 
open  source,  and,  358 
outcome-based,  560-561 
systems  design  for  change  in, 
691-701 

educational  communications,  18 
defined,  21 

theoretical  foundations  of,  21-26 


educational  communications  and 
technology,  5,  18, 21, 22, 23, 
24,25,26,29,31,386,442, 
443,451,619,648 
foundations  for  future,  807-815 
research  and  development,  30, 
811-812 

whole-task  models,  and, 
446-447,448,454 
educational  connoisseurship,  608 
educational  criticism,  608 
educational  design,  739,  744,  747 
educational  engineering,  746-747 
educational  hypertext,  199-208 
educational  media,  8-9 
educational  radio,  7-8 
educational  science,  814 
educational  technology,  12 
artifacts,  774 

complexity  theory,  and,  29 
defined,  21, 31, 812 
degree  in,  17 
history  of,  3-18 
research,  715-757 
defined,  295 

intervention.  723-726,  727 
theoretical  foundations  of,  21-26 
educational  television,  9-11,  17, 
264-265 

EDUCAUSE,  299 
effect  size,  defined,  606 
effectiveness  testing,  372,  373, 375 
elaborated  feedback,  128,  130,  132, 
133, 134, 135, 136, 138 
components  of,  135-136 
types  of,  134-135 
elaboration,  instructional  software 
design,  637 

elaboration  theory,  76,  390-391 , 
442, 443, 448^149, 450 
evidence  for  effectiveness  of,  452 
e-learning,  263,  356-363,  570,  609, 
613,674,707,709,808,810 
4C/ID,  and,  453 
heuristics  for,  721 
principles,  174,  178 
specialist,  instructional  design, 
570 

electroencephalography  (EEG),  788 
electronic  performance  support 
system  (EPSS),  51 1, 519-523 
applications  of,  520-521 
defined,  508 
embedded,  519 


external,  519,  521 
extrinsic,  519,  520,  521 
intrinsic,  519,  520,  521 
linked,  519 
research  on,  521-522 
returns  on  investment,  52 1 
types  of,  519-520 
element  interactivity,  42,  378-379 
elements  of,  660-662 
e-mail,  15, 226,  227,  229,  245,  358, 
410,415,532 

emancipatory  design,  662,  663 
embedded  hyperlinks,  203 
embodied  curriculum,  97-108 
designing,  99-107 
situationally,  99 
embodiments,  324 
defined,  539 

emergent  approach,  leveraged, 
693-694 

emergent  modeling,  540,  547-548 
defined,  539 

simulation,  102-104,  107 
emotional  agents,  285 
emotions,  48, 49 
learning  and,  219 
empiricism,  71 
defined,  68 

employee  selection  practices,  514 
encapsulation,  347 
encoding  knowledge,  246,  387 
encoding  memories,  48,  78 
engagement,  49-50,  323-324 
engineering,  behavioral,  510 
models,  511-515 

engineering  students,  problem-based 
learning  and,  489 
enterprises,  as  whole  tasks,  446 
environment  organizing  and  linking 
principle,  373 
episodic  buffer,  89 
episodic  memory,  47-48,  687,  688 
epistemological  knowledge,  488 
epistemology  ,5, 6, 15, 57, 58, 60, 70, 
71,228,340,430,528 
behaviorism,  73 
cognitive  constructivism,  74,  78 
cognitivism,  73-74,  77,  78 
constructivist,  531 
defined,  68 
objectivist,  530 
resource-based  learning,  530 
situativity  theory,  75 
sociocultural/historicism ,  7 4-7 5 
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epitome,  of  elaboration  theory,  391 , 
448 

ergonomics,  of  learning 

environments,  252,  254 
error  analysis,  136 
error  rate,  in  programmed 
instruction,  191 
e-solutions,  810 
eStep,  154-155 

ethics,  design  and  development  and, 
655 

ethnographic  evaluation,  608,  768 
ETV,  see  educational  television 
Europe,  educational  television  in,  10 
evaluation 

defined,  606 
distributed,  609 
ethnographic,  608,  768 
formative ,  606 
inquiry  learning,  459, 460 
modeling,  462, 465, 605-616 
naturalistic,  607-608,  768 
of  technology,  608-616 
program  theory,  607 
program,  models  for,  606-608 
qualitative,  607 
responsive,  608 
summative,  606 
team,  654 
theory-based,  607 
theory-driven,  607 
utilization-focused,  607 
vs.  assessment,  562 
evaluative  feedback  component,  135 
evaluator,  instructional  design,  570 
event,  experimental,  797,  798,  799 
event-related  potential,  794 
evidence-centered  design,  600 
evidentiary  reasoning,  499 
evolution,  372 

Excel,  first  principles  and,  177 
executable  models,  461 
executive  function,  of  working 
memory,  88-89 
exerted  mental  effort,  88 
expectancy  theory,  510 
expectations,  of  learners,  322 
experience-centered  instruction,  385 
experience,  concrete,  310 
experiential  learning,  33-37, 98,  99, 
319 

key  elements  of,  35 
theory,  310 
villages,  36 


Experimental  Advanced 

Instructional  Design  Advisor 
(XAIDA),  393 

experimental  method,  718,  719 
experimentation,  inquiry  learning, 
459,465 
experiments 
cost  of,  725 
defined,  717 
demonstration,  727 
design,  727 

instructional  strategies,  719 
model  validation,  740 
purpose  of,  720 
quasi,  see  quasi-experimental 
studies 

randomized,  718,  719,  720, 
727-728,736 
replication,  727-728 
rigor  of,  719,  726-728,  730 
true,  720,  721, 722,  723, 724, 
729,736,738 
usage  of,  719,  730 
expert,  see  also  experts 
-based  scoring,  600,  601 
chess  players,  371 
designers,  750 
game  players,  169 
instructional  designers,  756 
judgments,  modeling,  600-601 
knowledge,  495, 590 
performance's  scoring  criterion, 
600 
push, 431 
review,  743,  747 
systems,  242-243,  246 
expertise,  242,  319,  374,  377,  378, 
446, 522, 783 
authenticity,  and,  556 
development,  defined,  539 
domain-specific,  554 
medical,  560,  561 
reversal  effect,  377,  378,  810 
teams,  and,  652 

experts,  241, 377, 812;  see  also  expert 
automaticity,  and,  584,  585 
decision  points,  and,  585 
design  problems,  and,  743 
heuristics,  and,  242 
model  use  research,  and,  741-742 
explicit  knowledge,  242 
exploration,  101,427 
defined,  426 

space  control  elements,  308,  309 


exploratory  ad  hoc  modeling,  338 
external  conditions  of  learning,  384, 
388-389, 392 
external  consultants,  628 
external  correction  variable,  129 
external  feedback,  127,  129,  130, 
131, 132, 134, 139, 140 
external  representations,  46 
external  validity,  732, 734,  743 
defined,  717 

extraneous  cognitive  load,  42,  88, 
339,374,375,376,377,378 
defined,  86 

extroversion/introversion,  310 
eye  tracking,  285, 786,  788, 789, 793 

F 

face-to-face  communication,  407, 
411,414,464,653 
face-to-face  learning,  269,  270,  271, 
272,273,407 
face  validity,  769 
faculty  satisfaction,  272-273 
fading,  181,212,377-378,427,432 
falling  edge,  798 
familiarity,  40 
feature  explorers,  202-203 
feedback,  335,  348, 478,  51 1,514, 
515,516,517,518,523,721, 
722 

adaptive  vs.  nonadaptive, 
138-139 
cognitive,  588 
cognitive  functions  of,  133 
collaborative  learning,  and,  410 
complexity,  132 
computer-based  training,  and, 
214 

conceptual  framework  for, 
127-132 

content  of,  135-136 
defined,  126-127,508 
delayed  vs.  immediate,  137 
design  theory,  and,  694 
effects  on  performance,  516-517 
elaborated,  128,  130,  132, 133, 
134, 135, 136, 138 
components  of,  135-136 
types  of,  132,  134-135 
external 

functions  of,  132-135 
vs.  internal,  127, 129, 130-132 
form  of,  137-139 
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goal-based  scenario,  449 
internal,  127,  129,  130,  131 
just-in-time,  449 
Kluger  and  DeNisi  model,  523 
loops,  126,  128,  129,  130-132 
mentoring,  431^132 
metacognitive  functions  of,  1 34 
motivational  function  of,  134-135 
scaffolding,  432 
simultaneous  vs.  sequential,  138 
single-try  vs.  multiple-try,  138 
situated,  449 
strategies,  125-140 
timing  of,  137 
tutoring,  designing  and 
evaluating,  132-140 
types  of,  132,  137 
unimodal  vs.  multimodal,  139 
Felder-Silverman  learning  style 
model,  309,  31 1,312 
field  evaluation,  743 
Fifth  Dimension,  166,  167 
film/film  projectors,  6-7,  9 
silent,  6,  7,  8 

World  War  I  training,  and,  7 
financial  incentives,  517-518,  522 
Firefox,  358 

first  principles  of  instruction,  16, 
174-177,447,707,708,810 
application  of,  177 
cognitive  task  analysis,  and, 
589-590 
defined,  174 

five  pillars,  of  Sloan  Consortium,  270 
five-stage  model  of  mindful  feedback 
processing,  127,  131 
fixation  cross,  791 
fixed  stereotyping,  284 
flaming,  411 

fleshing  out,  during  modeling,  45 
flexible  learning,  251-255 
defined,  252 
flickr,  349,  350 

flight  simulators,  216,  678,  679 
flow,  49-50,  219,  324 
fluency-based  process,  40 
focused  discussions,  405 
foreperiod,  791 
Forest  Watch,  105 
formal  models,  579 
formalisms,  99,  108 
defined,  98 

design  languages  as,  638 
formalization,  637-638 


formative  evaluations,  606 
formative  research,  727 
formism,  57-58 

immanent  vs.  transcendent,  57 
four-component  instructional  design 
(4C/ID),25, 176, 179-181,182, 
374,386,442,443,448, 
450-451,452,587,707,774 
cognitive  task  analysis,  and, 
588-589 
defined,  174 

evidence  for  effectiveness  of,  453 
four-level  model,  Kirkpatrick’s,  608 
4-MAT,  175 

four-phase  cycle  of  instruction,  175 

four-process  adaptive  cycle,  280-282 

fractals ,  696 

fragmentation,  442 

frame  problem,  285 

Free  Software  Foundation  (FSF),  357 

FreeMind,  358 

free-operant  conditioning,  193 
frequency,  of  behaviors,  193 
frustration,  280 

function  and  derivative  model,  156 
functional  contextualism,  59-60 
defined,  56 

instructional  design,  and,  62-64 
functional  magnetic  resonance 
imaging  (fMRI),  788 

G 

Gagne’s  conditions-based  theory, 
389-390, 395 

Gagne’s  consecutive  models,  710 
Gagne’s  theory  of  instruction,  77, 678 
GAIDA  (Guided  Approach  to 

Instructional  Design  Advising), 
678 

GALILEO  database,  529 
gambits,  687 

games,  16,  105,  169,  216,  318,  320, 
321,322,324,336,338,539, 
680 

cooperative  learning,  and,  416 
essential  characteristics  of,  318 
instructional  design,  and, 
678-680 

log  files,  and,  736 
online,  323 

PICCO,  550-551, 552 
serious,  212,  218-219 
virtual  world  based,  320 


gaming,  see  also  games 
defined,  539 
tools,  680 
gender  differences 

achievement  recognition,  and, 
324 

computer-supported  cooperative 
learning,  and,  413 
design  models,  and,  756 
generational,  302 
Internet  usage,  and,  5 1 
general  aptitude  hypothesis,  of 
reading,  545 

general  knowledge  hypothesis,  of 
reading,  545 

General  Public  License  (GPL),  357 
General  Semantics,  1 1 
general  vs.  specific  knowledge,  242 
generalizability,  775 
generalizations,  310, 427 
Generation  Me  (GenMe),  300,  302 
Generation  X,  296, 297,  299,  300 
Generation  Y,  296, 297,  298,  299 
generation,  in  Wittrock’s  model,  1 12 
generational  differences,  295-302 
defined,  295 
race/gender,  and,  302 
generations,  see  also  specific  types 
attributes  of,  297 
defined,  296 
differences  in,  295-302 
generative  events  of  instruction,  392 
generative  learning,  1 1 1-123 
applied  research,  1 14-122 
comprehension,  and,  115 
foundations  of,  112 
higher  order  thinking,  and, 
115-121 

learning  outcomes,  and,  114-121 
recall,  and,  1 15 
GenScope,  461 

Gentner’s  syntactical  theory  of 
analogy,  322 
Geo3D,  147, 153 
Geometry  Cognitive  Tutor,  478 
Geometry  Explanation  Tutor  (GET), 
478 

germane  cognitive  load,  42,  88,  339, 
374,378,447 
defined,  86 

Gestalt  psychology,  442, 443, 
445^146,451 

Gilbert’s  human  competence  model, 
511-512 
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GIMP,  358 
glass  boxes,  331 , 334 
Global  Learning  and  Observations  to 
Benefit  the  Environment 
(GLOBE),  105 
Global  Rivers  Environmental 

Education  Network  (GREEN), 
105 

GNU  Free  Documentation  License, 
352,357 

goal,  see  also  goals 

-based  scenarios,  442, 443, 448, 
449-450 

evidence  for  effectiveness  of, 
452-453 

orientation,  of  learners,  321 
setting,  201-202, 206,  323,  510, 
511 

feedback,  and,  516 
theory,  219,  510,  515,  515-516 
defined,  508 
goals,  523;  see  also  goal 
categories  of,  449 
flexible,  253 

instructional,  706,  709,  711 
prescriptive  theories,  and,  709 
resource-based  learning,  and, 
528,531,533,534 
specific,  515,516 
Goals,  Operators,  Methods,  and 
Selection  Rules  model,  215 
GoNorth,  416 
GPEP  report,  487 
grammar,  337,  338 
Graphica,  433 

graphical  user  interfaces,  16 
graphics,  91-92 

great  books  of  the  Western  world 
curriculum,  385 

grocery  truck  analogy,  728,  732 

grounded  theory,  772 

group 

-based  learning,  259, 260,  264, 
264-265 

defined,  765,  776 
characteristic  data,  780 
composition,  fluctuations  in,  782 
learning  processes,  776-783 
data  types,  780 
defined,  765 
performance,  776-777 
potential  performance  of,  406 
processing,  406, 407^-08, 417, 
418 


problem-based  learning  and, 
494-495,496,497,498, 
500 

product  data,  780,781 
settings,  782 
shared  knowledge,  781 
groupware,  415, 417, 429,  672 
authoring,  defined,  672 
guidance  fading  effect,  377-378 
Guidance  System  for  Transforming 
Education  (GSTE),  624, 
696-698 
GUIDE,  678 

Guided  Approach  to  Instructional 
Design  Advising  (GAIDA),  393 
Guided  Experiential  Learning 
(GEL),  587 

guided  participation,  427,  428 
guided  reinvention,  547,  549 
guidelines  for  leadership  in  school 
reform,  Schlechty’s,  696, 
698-699 
GUIDON,  243 

H 

Hanging  with  Friends,  Velocity 
Style!,  150 

hard- skills  simulation  tools,  679 
hard  support  structures,  335 
hard  technologies,  279, 282, 285-286 
defined,  278 
Head  Start,  543 

Headsprout  Reading  Basics,  63 
hearing-impaired  students,  271 
heart-rate  variability,  785-786 
Help-Seeking  Tutor  Agent,  478 
Herbart’s  five-step  teaching  method, 
175 

heuristic  knowledge,  242,  243 
heuristics,  720-721, 730,  770 
hierarchical  hypertext ,  20 1,203, 204, 
205,206 
defined,  200 

high  performance  cycle  model, 
515-516 

higher  order  thinking,  115-121 
problem-based  learning  and, 
491-492 

holistic  approach  to  education, 
441-454, 542 

holistic  data  collection,  defined,  765 
Honey  and  Mumford’s  learning  style 
model,  309,  310-31 1,312 


hotwords ,  20 1 

human  cognitive  architecture, 
369-380 
defined,  370 

human  competence  model,  Gilbert’s, 
511-512 

human-computer  interactions, 

24-25,212,215,216, 220, 254, 
336,417,664,736 
human  performance  technology,  63, 
509,813 

humility,  22-23,  26 
hyperlinks,  475 
embedded,  203 

hypermedia,  215, 220,  286, 410, 475 
adaptive,  see  adaptive 
hypermedia  system 
defined,  212 

hyperspace  complexity,  308 
hypertext,  199-208,  215,  220,  346, 
475 

defined,  200,  212 
hierarchical,  201, 203,  204,  205 
defined,  200 
ill-structured,  205, 206 
linear,  200,202,205,206 
navigation  patterns,  and,  202-204 
reader  goals,  201-202 
relational,  200, 201 , 202, 203,206 
self-regulated  learning,  and, 
206-208 

hypothesis  generation,  inquiry 
learning,  459, 465 

I 

ibstpi®,  571-573, 574 
icon.  24 

ID  Expert,  392,  393,  709 
ID2,  392-393 

transaction  shells,  662,  665 
idealism,  71-72 
defined,  68 

idealized  design,  693, 695,  696,  700 
identical  elements  theory,  445 
identity  development,  435-436 
IDExpert,  676-677 
IEEE  Learning  Object  Metadata 
standard,  349,  350,  359,  361 
IEEE  Technical  Committee  on 

Learning  Technology,  813,  814 
ill-structured  hypertexts,  205,  206 
ill-structured  problems,  336-339, 
488,491,497,499,709,812 
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imagination  effect,  378 
imitation,  190 

human  learning  and,  372 
immersive  dynamic  modeling,  341 
immersive  environments,  287 
importance,  sense  of,  300 
IMS  Learning  Design,  356,  361-362 
IMS  Learning  Resource  Meta-Data 
Specification,  349 
IMS  QTI.361 
IMTS,  459 
INCENSE,  478 
incentives,  511,516,  522,  523 
defined,  508 

effect  on  performance,  517-518 
monetary,  516,  517,  518,  522 
indeterminacy,  687 
indirect  group  process  data,  780 
indirect  process  measures,  777-778 
defined,  765 

individual  accountability,  407 
individual-ecosystem,  75 
individual  learning  processes, 
776-777 

individualistic  learning,  404, 407, 
408,409,411,412,413 
individualized  instruction,  403 
Individually  Guided  Education,  471 
Individually  Prescribed  Instructional 
System  (IPI),  471 
inducibility,  406 
inductive  reasoning,  308 
inference  engine,  243 
information 

elaborated,  132 
extra-instructional  elaborated, 
132 

instruction-based  elaborated,  132 
just-in-time,  587,  588,  589 
multiple  sources  of,  375, 376 
novel,  374 

pushed  vs.  sought,  226 
redundant,  376 
schema-inconsistent,  46 
supportive,  587-588 
task-specific  elaborated,  132 
types  of,  3 1 

information  and  communication 
technology  (ICT),  254-255, 
270,356,359,470,478,533, 
572, 745 
study  of,  773 

Information  and  Library  Science 
(ILS),  238 


information  literacy,  533 
information  overload,  42 
Information  Power:  Guidelines  for 
School  Library  Media 
Programs ,  234 
Information  Power:  Building 

Partnerships  for  Learning,  234 
information  processing,  131,  133, 
137;  see  also  natural 
information  processing  system 
defined,  40 

dual-code,  see  dual-code  theory 
models  of,  40 
principles,  370-373,  379 
stage  model  of,  41 
symbolic,  73 
speed,  309 

theory,  87,  88,  95,  114,530 
information  retrieval,  inquiry-based, 
216-217 

information  store  principle,  371 
information  systems,  sophistication 
of,  527-528 

informational  feedback  component, 
135 

informative  feedback,  127 
informative  tutoring  feedback  (ITF), 
137 

defined,  126 
inheritance,  347 
innovation 

design  languages  and,  638 
design  of,  649 
rate  of  adoption  of,  622 
in-process  coordination,  780 
in-program  learner  behaviors, 
defined,  717 

in-program  learning  behaviors,  732 
inquiry 

-based  information  retrieval, 
216-217 

-based  learning,  53 1 
defined,  146,  459 
learning  ,213,45 9^16 1,531,810 
defined,  212, 457, 458 
model-based,  463-464 
technology-enhanced  support 
strategies  for,  145-159 
map,  WISE,  148-149 
transformative,  460 
Inquiry  Island,  217 
inside-out  story,  of  school  change, 
622 

INSPIRE,  312 


Institute  of  Electrical  and  Electronics 
Engineers  (IEEE),  359,  360 
instruction 

activity-centered,  385 
anchored,  15,  149,319-320 
audio-visual,  7,9,  11 
computer-assisted,  8,  12 
content-centered,  385 
direct,  586 

direct  transfer  models  of,  403 
direct/indirect,  544,  545 
events,  generative  vs. 

supplantive,  392 
experience-centered,  385 
four-phase  cycle  of,  175 
individual  assumption  of,  403 
learner-centered,  385 
learner-controlled,  475,476 
problem-centered,  175-176 
program-controlled,  475, 476 
programmed,  4,  8,  9,  12,510 
programmed  technologies  for, 

"  187-194 

radical  behaviorism,  and,  1 1-12 
reading,  542-545 
subfunctions  of,  133 
task-centered,  181-182 
technologies  linking  with 
learning  and  cognition, 
305-313 

technology-based,  37 1 , 372, 373, 
374-379 
visual,  6-7 

whole-task  modeling  and,  see 
whole-task  modeling 
instructional  design,  25-26,  174, 
295,589,686,706,707,708, 
709,711 

authoring  tools,  and,  674-675 
automated,  708,  709 
cognitive  apprenticeship,  and, 
429 

cognitive  load  theory,  and,  42 
cognitive  task  analysis,  and, 
582-583,588 
cognitive  traits,  and,  308 
cognitivism,  and,  756 
competencies,  569-575 
conditions-of-learning  paradigm, 
384 

constructivism,  756 
constructivist  models,  and,  61 
content  layer,  636-637 
defined,  174,296,570,740 
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descriptive  contextualism,  and, 
60-62 

descriptive/prescriptive  models 
for,  707 

design  and  development  teams, 
and, 649 

developments  in,  808-810 
education,  813 

environments,  characteristics  of, 
689 

expert  systems,  393 
functional  contextualism,  and, 
62-64 

graduate  programs,  813 
law,  and,  555 

learning  content  management 
system  (LCMS),  and,  683 
learning  objectives  as  basis  for, 
447 

models,  706, 709,  710,  717,718, 
739-748,750 
defined,  706 
nature  of,  707,  710 
objects,  812 

outcome-referenced,  conditions- 
based,  384 

prescriptive  principles  for, 
173-183 

realistic  mathematics  education, 
and,  548 

research,  24, 708, 718,  739-748, 
810-811 
roles,  570,  571 
second-generation  (ID2), 
392-393 

study  of,  812-813 
systems  approach  to,  12-14 
task-centered,  181-182 
theoretical  statements,  and,  706, 
707 

theory,  686 
tools,  676-678 

Instructional  Design  Advanced 
Workbench,  393 

instructional  design  and  technology 
(IDT),  187,  188,  194 
instructional  designers,  634,  640, 
710,742,812 
argumentation,  and,  686 
authoring  tools,  and,  674-675 
competencies  for,  569-575 
defined,  570 

domain-specific,  539,  540 
expert  vs.  novice,  756 


graduate  programs,  814 
learning  objects,  and,  347,  351 
preparing,  813-814 
Instructional  Development  Institute 
(IDI),  13 

instructional  episodes,  31 
instructional  events,  31 
instructional  models,  537-562,  810 
instructional  object,  347 
instructional  principles,  learning 
principles,  and,  179 
instructional  research.  539 
domain-specific,  540-541 
instructional  strategies 
defined,  717,  720 
levels  of,  176 
research  on,  719-730 
rigor  of,  726-728 

instructional  systems  design  (ISD), 
453,708,660,664,665,667, 
686, 689-690 
model  validation,  and,  753 
precedent,  and,  689 
Instructional  Systems  Development 
(ISD),  13-14 

instructional  technologists,  12,  814 
instructional  technology,  12,  188, 
718 

cooperative  learning,  and,  404 
defined,  740,  812 
research,  727 

instructional  theories,  76-79 
instructional  transaction  theory,  348, 
390 

instructional  variables,  287 
instrumentalist  hypothesis,  of 
reading,  545 
instrumentation 

complex  performance 

measurement,  790-797 
computers  as,  799-800 
defined,  765 

integrated  curriculum,  defined,  442 
Integrated  Task  Analysis  Model 
(IT  AM),  587 

integration  principle,  92, 175 
defined,  174 
integration,  of  text,  91 
integrative  teaching,  431 
integrativity,  447, 448 
intellectual  property  rights,  357 
Intelligent  Essay  Assessor,  216 
intelligent  learning  environments, 
477 


intelligent  tutoring  systems,  127, 
138, 191-192,216,219,243, 
286,312,434,470,471,474, 
475,479,641,810 
cost  of,  220 
defined,  212 

intentional  learning  space,  31 
interactions,  418 
CIAO ! ,  609 

oppositional  vs.  promotive,  406 
promotive,  407 
interactive  architect,  574 
interactive  learning  tasks,  125-140 
interactive  multimedia,  defined,  86 
interactive  simulation,  215-216 
defined,  212 

interactive ,  two-feedback-loop  model , 
128, 129, 130, 131, 133, 139 
interactivity,  dynamic  modeling, 
and,  338,  339 
interactivity  effect,  339 
interdependence,  406-407, 417,418 
interference  perseveration 
hypothesis,  137 

intermediate  concepts,  554,  556 
internal  conditions  of  learning,  384, 
387,395 

internal  consistency,  769 
internal  feedback,  127, 129, 130, 131 
internal  validity,  726, 729, 730, 731, 
732,  734,  743 
defined,  717 

International  Association  of  School 
Librarians  (IASL),  235 
International  Board  of  Standards  for 
Training,  Performance,  and 
Instruction  (IBSTPI),  360, 
571-573,574 

International  Electrotechnical 
Commission  (IEC),  359 
International  Organization  or 

Standardization  (ISO),  359, 360 
International  Society  for 

Performance  Improvement, 
510,511,514,523 
International  Society  for  Technology 
in  Education  (ISTE),  572 
International  Telecommunication 
Union  (ITU),  359 
Internet,  7,  16-17,  167,  199,  203, 
226, 227, 229, 244, 246, 260, 
263,272,348,415,431,498, 
532,  770 

use,  gender  and,  51 
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Internet  Engineering  Task  Force 
(IETF),  360 

interpersonal  skills,  407 
interpretation  network,  44 
inter-relay  chat  (IRC),  265 
Interservice  Procedures  for 
Instructional  Systems 
Development  (IPISD),  13 
inter-trial  interval  (ITI),  791, 792 
intervention  research,  723-726 
interventions,  740 

interviews,  580,  581-582,  589,  740, 
747,751,753,754,771,772, 
773-774,775,777,780 
intrinsic  cognitive  load,  42,  86,  88, 
339,374,378,379 
intrinsic  load,  86,  88 
Invisible  Learning?  The 

Contribution  of  School 
Libraries  to  Teaching  and 
Learning,  237 

Iowa  Tests  of  Basic  Skills,  235 
ISIS/Draw,  153-154 
ISO/IEC  JTC  1,359-360 
isolated  interacting  elements  effect, 
379 

iSTART,  218 
item  tryouts,  770 

iterative  design,  662,  665,  693,  694, 
709, 746 

J 

Japan,  educational  radio  in,  7-8 
Jasper  Woodbury  Problem  Solving 
Series,  10,79,  100,  149-150, 
460,529,534,650,733-734 
jESSE,  227 

job  performance  aids,  514 
John  Henry  effect,  728 
joint  enterprise,  428 
judging/perceiving,  310 
just-in-time  information,  587,  588, 
589 

just-in-time  learning,  263 

K 

KBS  Hyperbook/Hyperlink,  477, 478 
Keller  Plan,  188,  191 
key  feature  problems,  490 
keyword,  defined,  808 
Kids  Network,  105 
kinesics,  772 


Kirkpatrick’s  four-level  model,  608, 
610 

know-how,  136 
knowledge 

about  concepts,  136 
about  mistakes,  136 
abstract,  545-549 
acquisition,  491 
automated,  584 
-based  systems,  245 
defined,  242 
expert,  242-243 
recommender  systems,  and, 
247 

communities,  244-245 
defined,  242 
content  vs.  process,  449 
creation,  1 12,  122,  228,  242 
randomness  as  genesis 

principle,  and,  372-373 
declarative,  242, 246,  387,  390, 
393,578,583-584,785 
deep  vs.  shallow,  214 
domain,  463, 489-490 
elicitation,  245,580, 58 1 , 586, 589 
encoding  of,  246 
epistemological,  488 
expert,  495,  590 
generation.  Ill,  113,  114,  121, 
122 

defined,  112 
incoming,  280 
integration,  434 
defined,  146 

management,  242,  243-244.  511 
defined,  242 
maps,  600 
object,  347,  348 
of  performance,  132 
of  result/response,  132 
of  the  correct  response,  132 
on  how  to  proceed,  136 
on  metacognition,  136 
on  task  constraints,  135 
personal,  638 
phenomenological,  488 
prerequisite,  588,  589 
procedural,  246,  387,  393,488, 
584,785 
quality,  246-247 
representation,  152, 246, 580-582 
artificial-intelligence-based , 
599 

rule-based  approach  to,  243 


retention,  491 
seekers,  202-203 
structure  density,  203 
structure,  learning  law  and, 
553-554 
systems 

cross-cutting  issues,  245-247 
object-oriented,  246 
technology-based,  241-247 
types  of,  241 
tacit,  638 
to  workplace,  491 
tracing,  478 
tracking,  216 

transfer,  371-372,373,374,491; 

see  also  learning:  transfer  of 
types  of,  242,393,687 
model  for,  463 

units,  of  generative  learning,  113 
via  borrowing  principle,  373,374 
vs.  skills,  393 

workers,  242, 243-244,  245, 693 
Knowledge  Forum,  166,  219, 434 
Knowledge  Integration  Environment 
(KIE),  434 

knowledge,  skills,  and  abilities 
(KSAs),  team,  648,  653 
Kolb’s  learning  style  model,  309, 

310 

L 

Lab  VIEW,  794 

LAN-based  client-server  systems, 
415 

Lance  Studies,  235-236 
Landamatics,  393 
language,  41 
games,  24 

signs  and  rules  for,  24 
proficiencies,  differential,  280 
latent  semantic  analysis,  91, 246 
Latin  America,  educational 
television  in,  11 
law  schools,  problem-based 
education  and,  499 
law,  learning,  540,  541, 552-556 
leadership 

participatory,  695-696 
team,  651, 652 
leading  edge,  798 
.Learn,  358 
learned  bias,  338 
learned  capabilities,  387 
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learner,  see  also  learners,  learning 
styles 

-centered  instruction,  385 
-centered  principles,  APA,  610 
-controlled  instruction,  475, 476 
control ,  290 , 4 1 0 , 449 , 45 1 , 475 , 
476,731-732 
data,  valid.  289 
-generated  learning,  392 
-initiated  learning,  392 
model  instance,  291 
models,  279, 280,  282,  284, 286, 
289,290,291 
defined,  278 

ecological  approach  to,  291 ; 
see  also  student 
modeling 
personality,  310 
preconceptions,  339 
responsibility,  122 
variables,  287 

learner/learning  design,  574 
learners,  see  also  learner,  learning 
styles 

adapting  content  for,  280 
characteristics  of,  204-206,  213, 
280 

synthetic  learning 

environments,  and, 
321-322 

cognitive  style  of,  205-206 
expectations  of,  322 
experienced,  271, 308,  374-375, 
522 

vs.  novice,  377,  378 
expert,  88 

field-dependent,  308 
global,  307,  311 

high  vs.  low  associative  learning 
skills,  and,  309 

high  vs.  low  inductive  reasoning 
abilities,  and,  308 
high  vs.  low  information 

processing  speed,  and,  309 
high  vs.  low  working  memory 
capacity ,  and ,  308 ,  3 1 2-3 1 3 
metacognition,  and,  309 
novice,  43,  88,  202,  271, 308, 
374-375,377,378,395, 
453,522,553,583,733, 
810 

sequential,  307,  311 
spatial  ability  of,  321 
types  of,  310-311 


learning 

adult,  442, 443, 444-445 
as  four-staged  cycle,  310 
assessment,  see  assessment 
astronomy,  550 
asynchronous,  259, 260,  261 , 
264,409-410,415,416, 
672 

autonomous  life-long,  155-157 
behaviors,  in-program,  732 
blended,  see  blended  learning 
brain  research,  and,  812 
capabilities,  five  types  of,  77 
categories  of,  306 
cognitive  multimedia,  41—44 
cognitive  psychology,  and, 
44-48 

collaborative,  163-170,323,335, 
575 

companion,  312 
computer-mediated 

communication,  and, 
228-229 

computer-mediated  technologies , 
and, 211-220 
cycle,  175 
declarative,  446 
defined,  31,  164,  166,766 
discovery,  78,  176,  308, 
539-540, 586 
distributed,  269 
effectiveness,  270-271 
emotions,  and,  219 
environments,  see  learning 
environments 
episodes,  449, 452 
experiences,  255 
experiential,  16,  33-37,  319 
exploratory,  308 
external  conditions  of,  384,  392 
face-to-face,  269, 270,  27 1 , 272, 
273,407 

flexible,  251-255 
from  models ,  457,458,459, 464 , 
465 

generative,  111-123 
goal-directed,  201 
goals,  5 

group-based,  259,  260,  264,  777 
groups,  assessment  of,  776-783 
hierarchies,  388,  581 
defined,  384 
imitation,  and,  372 
in  Wittrock’s  model,  1 12 


information-processing  theory, 
and, 530 

inquiry,  145-159,213,531 
defined,  212 
intentional,  31 

interactive,  feedback  strategies 
for,  125-140 

internal  conditions  of,  384,  395 
just-in-time,  263 
law,  552-556 

learner-generated  vs.  learner- 
initiated,  392 

life-long,  488, 492,  811, 813 
management  systems,  see 
learning  management 
systems 

mathematics,  545-549 
meaningful  reception,  77 
media,  and,  40 
medicine,  556-562;  see  also 
medical  education 
model-based,  429 
model-facilitated,  457-465 
model-oriented,  46-47 
motivation,  and,  48-51 
multimedia,  41-44, 46, 48,  50 
motivation,  and,  40,  49 
natural,  404 
objectives,  306, 447 

taxonomy  (Bloom’s),  130, 136 
objects,  see  learning  objects 
observational,  73 
offline,  259,  260, 262-263 
online,  259,  260,  263-264, 270, 
271,272,273,409-410, 
411,431,434,498,500, 
573,574,610,671-683 
open  source,  and,  358 
operant  conditioning,  73 
organizations,  694,  696 
outcomes,  see  learning  outcomes 
paradox,  546,  547 
places,  251-255;  see  also  place 
independence 
defined,  252 

planning,  components  of,  744—745 
principles 

collaborative  knowledge,  166 
instructional  principles  and, 
179 

Mayer’s,  92 
procedural,  446 

psychology  of,  23-24. 445-446 
reception,  78 
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regulation  of,  460 
respondent,  73 
rote,  34,  680 
science,  549-552 
self-directed,  133,427,447,454, 
488,489,492,493,495, 
496,497,499, 559, 560;  see 
also  learning:  self-regulated 
assessment,  and,  447 
problem-based  learning,  and 
492,493 
workplace,  560 
self-paced,  259,  260, 262-264 
self-reflective,  489 
self-regulated,  121,206-208, 
433,768 
defined,  200 

simulation-based  discovery,  308 
situated,  319-320 ,427,488,531, 
730, 734 
defined,  252 
skills,  associative,  309 
social,  264,  323 
strategies,  114 
styles,  see  learning  styles 
synchronous,  259,  260,  261 , 
264-265,415 

task-centered  approach  to,  174 
tasks,  125-140,587 
4C/ID  and,  450 
team,  696 

technology-enhanced  (TEL),  36 
technologies,  see  learning 
technologies 
theoretical,  34 
theories,  41-48,  87-89 
to  read,  542-545 
transfer  of,  445-446, 447, 448, 
450,451 
increasing,  447 
transformation,  270-271 
vs.  performance,  777 
workplace,  561, 562 
Learning  by  Design,  151 
learning-by-doing  paradigm,  449 
learning  by  modeling  ,457,458, 462 , 
463,464,465 
computer,  458-463 
learning  content  management 

system,  5 1 1 , 672, 68 1 , 682, 683 
defined,  672 

learning  environments,  212,  213, 
2f4,  215, 217,  219,  220,  307, 
370, 809 


active,  271 
adaptive,  286-287 
architect,  574 

adaptive  instruction,  and,  473 
blended,  269-274 
Co-Lab,  464 
collaborative,  286 
competence-based,  742 
computer-based,  see  computer- 
based  instruction 
defined,  146 
directed  distance,  532 
enriched,  661 
ergonomics  of,  252,  254 
gaming,  323 
hybrid 

defined,  270 
hypermedia,  475 
intelligent,  477 
multimedia,  429 
open, 733 

open-ended,  528,  717,  733 
powerful,  709 
rich,  9 

socioeconomic  status,  and,  552 
synthetic,  see  synthetic  learning 
environments 

technology-enhanced,  718,  733, 
734^735,736 
technology-enhanced,  see 
technology-enhanced 
learning  environments 
virtual,  810 
Web-based,  429, 430 
learning  first,  understanding  later, 
547 

learning  management  systems, 
531-532 
defined,  226 
online,  264 

learning  objects,  220,  247,  263,  290, 
291,681,682,707 
as  LEGOs,  347 

brick-and-mortar  metaphor  for, 
348 

buying/selling  of,  351 
cataloging  and  finding,  349-350 
criticisms  of,  350-35 1 
defined,  346,  347 
literature,  345-352 
metadata  standards  for,  349 
metaphors  for,  347-348,  351 
02  model  for,  348 
specification  of,  348-35 1 


learning  organizations 

five  disciplines  of,  694,  696 
systems  thinking,  and,  694,  696 
learning  outcomes,  24,  114-121, 
383-395,771 
categorizing,  387-388 
external  conditions  of,  388-389, 
392 

hierarchies  of,  388 
internal  conditions  of,  387,  392 
medical  education,  557-558,  561 
problem-based  learning,  and, 
486,489,489-492,495, 
497 

learning  styles,  36, 306, 307, 309-311 
cognitive  traits,  and,  312-313, 
321 

defined,  34,  306,  309 
Lelder-Silverman  model,  309, 
311,312 

Honey  and  Mumford’s  model, 
309,310-311,312 
identifying  ,311-312 
index  of,  312 
Kolb’s  model,  309,  310 
Myers-Briggs  Type  Indicator, 
309,310 

learning  technologies 
classes  of,  213-219 
developments  in,  808-810 
linking  with  cognition  and 
instruction,  305-313 
research,  810-811 
Learning  Technologies  Standards 
Committee  (LTSC),  IEEE,  359 
Learning  Through  Collaborative 
Visualization  Project,  see 
CoVis 

learnplaces,  see  learning:  places 
legal  arguments,  555 
legal  case,  seven  stops  for  solving, 
553 

legal  concepts,  intermediate,  554 
legitimate  peripheral  participation, 
427,428 

LEGO  Mindstorms,  334 
legoLOGO,  415 
LEGOs,  347 
lethal  adaptation,  624 
defined,  620 

levels  of  processing,  40,41 
leverage  points,  696 
leveraged  emergent  approach, 
Reigeluth’s,  693-694 
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Levin’s  stage  model,  727 , 728 
liberating  systems  theory  (LST),  694 
library  media  centers,  233-239 
current  issues,  238-239 
defined,  234 
research,  234—238 
library  media  specialists,  234 
Library  Power  Project,  235 
life-long  learning ,  488 , 492 ,811,813 
Likert-type  scales,  771 
LiNC  (Learning  in  Networked 
Communities),  663-664 
Lindsley,  Ogden,  192-194 
linear  control,  410 
linear  hypertext,  202,  205, 206 
defined,  200 

linear  programs,  programmed 
instruction,  190 
linguistics,  641 
link-level  adaptation,  476 
Linux,  358,  363 
LionShare,  361 
LISTEN  Project,  286 
literacy,  resource-based  learning, 
and, 533 

literate,  defining,  533 
literature  reviews,  740 
log  files,  735,  736-738,  739 
defined,  717 
logic  model,  607 

LOGO  computer  environment,  410, 
413,415 

Logo  programming  language,  336 
long-term  memory,  87,  88,  89,  92, 
371-375,377,378,379,380, 
530,583 
audio,  93 
chess,  and,  371 
defined,  86,  370 
loose  coupling 
defined,  620 
low  floor  approach,  337 

M 

MACH-III,  459 
machine,  as  root  metaphor,  58 
machine  learning,  284 
macro-adaptive  systems,  471-472, 
479 

defined,  470 
macrocontexts,  100 
macro-instructional  strategies,  721 , 
722-723,724,730 


magic  lantern,  6 

magnetic  resonance  imaging  (MRI), 
"  788 

magnetoencephalography  (MEG) , 

"  788 

management  styles,  generational 
differences  and,  300 
Manhattan  Project,  650 
manifests  of  traits  (MOTs),  312 
manipulation,  of  environment,  44 
maps,  relational,  201 
Maslow’s  theory  of  needs,  510 
MAS-PLANG,  312 
Mastery  Learning,  262, 471 
mathemagenic  methods,  447 
defined,  442 

mathematics,  learning,  540,  545-549 
frameworks  of  reference,  546 
models,  546 
mathematizing,  547 
meaning  making,  111-123,  200,  547 
defined,  112 

meaningful  reception  learning,  77 
means  interdependence,  406-407 
means  planning,  700 
mechanism,  57,  58,  69 
M-Ecolab,  478 
media 

as  instructional  strategies, 
728-729 

-based  design  principles,  86-89 
centers,  library,  233-239 
comparison  studies,  226,  728 
complex,  31 
costs  of,  226 
educational,  8-9 
effects,  728 

learning,  and,  40, 48-5 1 
metamorphosis  of,  527 
replications,  729 
roles  of,  16 
visual,  6 

Media  Programs:  District  and 
School,  234 

medical  education,  540,  542, 
556-562 

cognitive  task  analysis,  and,  586 
computer-based  assessment,  and, 
600-601 

pressures  for  change  in,  559-560 
problem-based  learning,  and, 
486-487,490,491.496, 
497,499,540,561 
undergraduate,  556-562 


medical  students,  virtual 
environments  and,  321 
mega-universities,  17 
member  checking,  754 
memes,  696 
memorization,  rote,  319 
memory,  39-51, 87,  88 
auditory,  89 
autobiographical,  48 
cognitive  views  of,  445 
defined,  40, 48 
echoic,  94 

encoding,  storage,  retrieval,  48 
episodic,  47-48,  687,  688 
interpretation  network  of,  44 
long-term,  87,  88,  89,  92, 
371-375,377,378,379, 
380,530,583 
audio,  93 
chess,  and,  371 
defined,  86,  370 
model  of  the  world ,  44 
semantic,  47,  48 

short-term,  87,88, 89,92, 94, 530 
audio,  94 
defined,  86 
spatial,  89 
theories,  41-48 
visual,  89 

working,  41. 42, 43, 44,  88,  92, 
307-308,312-313,337, 
373,374,375,376,378, 
379,380,530 
anxiety,  and,  49 
defined,  370 

executive  function,  88-89 
expanding,  376 
long-term,  44 

narrow  limits  of  change,  and, 
373 

memory  research,  physical  vs. 

functional,  41 
mental  efficiency,  789 
mental  effort,  785 
mental  models ,41,45,46,47,458, 
550,695,696,781 
defined,  40 
shared,  653 

mental  representation,  45 
mentoring,  429 

formal  programs  for,  430^-3 1 
online,  431 
research  on,  430^-32 
strategies,  431-432 
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Mentoring  for  Effective  Primary 
Science  Teaching,  431 
message  design  principles,  9 
message  design  research,  723 
metacognition,  112,  114,  130,  133, 
137, 138, 139,204,207,208, 
215,410,433-434,471 
-based  adaptive  systems, 
478-479 
defined,  309 

problem-based  learning,  and, 
491,492,493,494 
metacognitive 

feedback  functions ,  1 34 
guides,  tutors  as,  494 
scaffolds,  529 

metadata,  244,  289,  349,  359,  528, 
681 

vs.  tagging,  349 
meta-modeling,  341 
meta-principles,  of  Design  Principles 
Database,  147-157 
metatags,  527 

micro-adaptive  systems,  471, 
473-475,479 
defined,  470 

micro-instructional  strategies, 
721-722,724,730 
micropayments,  for  learning  objects, 
351 

micro  worlds,  15,  16 
MicroWorlds,  338 
Midwest  Program  of  Airborne 
Television  Instruction,  10 
Millennial  Generation,  296,  297, 
298,299-300,301 
cuspers,  297 
survey  of,  297 
mind-body,  70 
mindfulness,  561 
mindlines,  245 

mindset,  systems  design  and,  695 
Minerva  project,  477 
minimalist  instruction,  principles  for, 
178 

defined,  174 
Mission  Rehearsal,  218 
mission,  of  goal-based  scenario,  449 
MIToCaR  (Model  Inspection  Trace 
of  Concepts  and  Relations),  47 
mixed-methods  research,  748,  751 , 
753,757,768,771,773,775, 
781 

defined,  717,  765 


mnemonics,  visual,  721 
mobile  phones,  262,  263,  809 
mobile  technologies,  262 
modality  effect,  375-376 
modality,  multimedia  learning  and, 
214 

model,  see  also  models 
argument,  555 

-based  inquiry  learning ,  34 1 , 457 , 
458,463-464 
-based  reasoning,  322 
-building  tools,  330,  331-334, 
335,337,341 
unguided,  339,  340 
connoisseurship,  608 
defined,  330,  457 
development,  740 
research  on,  741 
-exploration  tools,  331-334 
-facilitated  learning,  457-465 
granularity,  337 
level  of  abstraction,  337 
logic,  607 

of  domain  learning,  554 
of  the  world,  44 
research,  see  model  research 
scoring,  600-601 
sequence,  589 
tracing,  138,  340, 478 
use,  research  on,  718,  741 . 742 
validation,  717,  718,  739,  740, 
742,743,745,746,747, 
748,749,750,751.753 
defined,  717 

Model-Based  Assessment  (MBA), 
598 

Model-Enhanced  Thinker  Tools,  341 
Model  Inspection  Trace  of  Concepts 
of  Relations  (MITOCAR),  781 
model  research,  739,  757 
defined,  717 

methods  and  strategies,  753-754 
studies,  740 
trends,  756 
modelers,  novice,  341 
modeling,  44, 427, 43 1 , 432, 433 
agent/individual-based,  334 
building  blocks,  335-336 
cognitive,  283-284 
collaborative  vs.  group,  335 
defined,  426,  457 
Design  Principles  Database,  and, 
155-156 

dynamic, 330,  331-342 


design  and  research  issues, 
340-342 
immersive,  341 
output  of,  339,  340 
place-based,  341 
software,  338 
emergent,  see  emergent 
modeling 

expert  judgments,  600-601 
exploratory  ad  hoc  modeling, 
338 

instructional  processes,  708 
languages,  335-336 
learning  by,  see  learning  by 
modeling 

motivation,  and,  337 
qualitative,  282 
quantitative,  282 
technologies,  329-342 
tools 

dynamic, 331-336 
classification  of,  331-336 
unit,  334 

variable-based,  334 
modeling-scaffolding-fading , 
212-213 

Model-It,  147,  152,  158,334,462, 
734 

ModellingSpace,  335 
models,  see  also  model 

as  mathematical  entities,  462 
construction  of,  462 
defined,  350, 457 
domain,  458 
domain-specific 

problems,  540-541 
requirements  for,  541-542 
evaluation,  605-616 
executable,  461 
formal,  579 

instructional,  537-562,  810 
instructional  design,  706,  709, 
710,717,718,739-748, 
750 

kinds  of,  331 

learning  from,  457, 458,  459, 
464, 465 

mathematical,  546 
mental,  458,  550,  695,  696 
problems  with,  541 
qualitative,  462 
shared  mental,  653 
transparent,  546 
validity  of,  331, 607,  710 
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modularity,  of  software,  665 
molecular  modeling  software, 
153-154 

Molecular  Workbench,  156,  158 
money ,  as  incentive  for  performance, 
516,517,518,522 
Moodle,  18,226,358,361 
MOOs,  see  object-oriented  MUDs 
Morningside  Academy,  194 
Morningside  Model  of  Generative 
Instruction.  63 
morphogenesis,  621 
morphostasis,  621 
motion  pictures,  6 
motivation,  39-51, 280,  323,  324, 
511,590,809 

-based  adaptive  systems,  478 
cooperative  learning,  and,  409, 
410 

games,  and,  680 
human  cognitive  architecture , 
and, 380 
intrinsic,  51 
model,  478 
modeling,  and,  337 
multimedia  learning,  and,  40, 49 
processes,  defined,  112 
task-oriented,  464 
theory,  471 
types  of,  48 

Wittrock’s  model,  and,  112,114, 
121 

workplace,  510 

motivational  function  of  feedback, 
134-135 

motor  learning,  442, 443^-44 
Ms.  Lindquist,  138 
MUDs,  see  multi-user  domains 
multidimensional  scaling,  781 
multimedia,  40, 41-44, 46, 48,  50, 
213,214-215,220,247,707 
cognitive  apprenticeships,  and, 
429 

costs  of  producing,  725-726, 730 
guidelines  for,  89-94 
animation/video,  94 
auditory,  92-94 
text,  89-91 

visuals/graphics,  91-92 
learning,  principles  for,  178 
defined,  174 

mixed-methods  study  of,  773 
problem-based  learning,  and, 
498-499 


multiplayer  online  role-playing 
games  (MMORPGs),  336 
multiple  analysis  of  variance 
(MANOVA),  722 

multiple-choice  questions,  101,  132, 
202,203,213 
deep  vs.  shallow,  214 
Multiple  Episode  Protocol  Analysis 
(MEPA),  788 
multiple  intelligences,  742 
multiple  regression  models,  473 
multiple-try  feedback,  132,  138 
multi-user  domains  (MUDs),  216, 
265 

multi-user  virtual  environments 
(MUVEs),  735 

multiverse  environments,  736 
mutual  adaptation,  624,  625 
defined,  620 
mutual  engagement,  428 
My  Online  Teacher  (MOT),  476 
MYCIN,  243 

Myers-Briggs  Type  Indicator,  309, 
310,311 


N 

narration,  animation  and,  43 
narrative  context,  99 
NASA  Task  Load  Index  (TLX),  785 
National  Defense  Education  Act,  9 
National  Education  Association 
Department  of  Audio-Visual 
Instruction,  7 

National  Institute  of  Child  and 
Human  Development 
(NICHD),  543 

National  Reading  Panel  (NRP),  543, 
544, 545 

National  Society  for  Programmed 
Instruction,  510 
natural  information  processing 
system,  370-371 
natural  learning,  404 
naturalistic  evaluation,  607-608, 768 
defined,  606 

navigation  patterns,  202-204 
active  vs.  passive,  203-204 
needs  assessment,  573 
negative  interdependence,  406 
Net  Gen,  297,  299 
networks,  cooperative  learning  and, 
415 

neuroscientific  data,  788 


new-age  instructional  designer 
competencies,  569-575 
newcomers,  to  communities  of 
practice,  435 

No  Child  Left  Behind,  235 , 542, 543, 
729 
noise,  99 

acquisition  vs.  processing,  94 
nonadaptive  feedback,  138-139 
nonparticipant  observation, 
772-773,774 

notation  system,  634, 635,  642 
for  developing  e-learning 
technologies,  360 
note  taking,  1 14,  115,  121,  157 
novice-expert  difference  model,  284 
novices,  204,  241 , 43 1 , 434, 452, 
555,556,579,582,583,584 
chess  players,  37 1 
designers,  750 
developers,  672 
instructional  designers,  756 
modelers,  341 
complex  tasks,  and,  783 
design  problems,  and,  743 
game  players,  169 
heuristics,  and,  242 
law,  554 
NVivo,  615,  788 

o 

02  model  for  learning  objects,  348 
Object  Management  Group,  360 
object-oriented  MUDs  (MOOs),  2 16, 
218 

object-oriented  programming,  246, 
347 

object  standards,  247 
objectivism,  71 
defined,  68 

objectivists,  530,  533,  756 
objects,  instructional  design,  812 
objects,  learning,  see  learning  objects 
objects,  model,  462 
observational  learning,  73 
observations,  310,  771, 772-773, 
774,779,786,788 
knowledge  elicitation  and,  580 
Observer  software,  788 
Ohio  study,  237-238 
online  gaming,  323,  336 
online  instruction,  671-683, 
770-771,774 
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online  learning,  see  learning:  online 
online/offline  measures 
defined,  765 

on-task  adaptive  learner-control,  476 
on- task  performance,  473, 474 
ontology,  70,71. 246,  289, 488, 
552-556 

-based  cataloging  of  learning 
objects,  350 
-based  searches,  641 
behaviorism,  73,  77 
cognitive  constructivism,  74,  78 
cognitivism,  73-74,  77 
defined,  68,  539 
situativity  theory,  75 
sociocultural/historicism,  74-75 
Open  Archives  Initiative,  350 
open  corpus  Web  sources,  475, 477 
open  educational  resources,  351-352 
defined,  346 

open-ended  learning  environment, 
528,733 
defined,  717 

open  learning  environments  (OLEs), 
733 

Open  Source  Initiative  (OSI),  357 
open  source  learning  management 
systems  (LMSs),  358 
open  source  software,  355-363 
defined,  356 
development  model,  357 
open  standards,  355-363 
defined,  356,  359 
Open  Systems  Connect,  360 
openness,  22-23,  26 
operant  behavioral  theory,  530 
operant  conditioning,  73 , 76,77,189, 
193 

operations  research,  12 
oppositional  interaction,  406 
organic  development,  as  root 
metaphor,  58 
organicism,  57,  58,  69 
organism-environment,  70 
organization,  as  learning  principle, 
92 

organizational  redesign,  514 
orientation,  inquiry  learning,  459, 
465 

orientation,  modeling,  462, 465 
OSS,  see  open  source  software 
outcome-based  education,  386, 446, 
560-561 

outcome  interdependence,  406^-07 


outcome-referenced,  conditions- 
based  models,  383-395 
alternatives  to,  385-386 
alternatives  within,  386 
defined,  384 
utility  of,  386 
overlay  model,  284 
overt  responses,  190 

P 

pace  independence,  259-265, 311 
Pandora,  247 

parallel  distributed  processing 
models,  40, 41 , 45, 47 
part-task 

backward-chaining  training,  451 
models,  443-446, 450-454 
defined,  442 
practice,  588 

participant  observation,  772 
participation  models  of  embodiment, 
104-107 

participatory  design,  662-663,  665 
application  of,  663-664 
collaborative  learning  example, 
663-664 
defined,  660 

in  Decatur  Township  schools, 
664 

projects,  review  of,  667 
strong  vs.  weak,  663 
participatory  leadership.  695-696 
participatory  simulations,  105 
partitioning,  181 
Pathfinder,  781 
pattern  matching,  44 
pattern  note,  113 
Pebble-in-the-Pond  instructional 
design,  177, 181,  182 
defined,  174 

pedagogical  agents,  285, 417,  722 
animated,  212 

as  learning  companions  (PALs), 
722 

peer  apprenticeship,  43 1 
peer  review,  246,  581 
perception-action,  70 
perception,  disciplined,  166 
performance 

assessment,  see  performance 
assessments 
criteria,  770 

data,  collection  of,  785-788 


defined,  596,  766 
expert,  600 
focus  on,  542 

improvement,  see  performance 
improvement 
levels  of,  512 
measuring,  509-5 10 
process  data,  785-788 
vs.  learning,  777 

performance  assessments,  766-767 , 
770,784,811 
design  of 

cognitive  demands,  598 
domain  representation, 

598- 599 

task  representation,  599 
templates  and  objects, 

599- 600 

technology-based,  595-602 
design  of,  598-600 
validity  of,  596-598 
performance  improvement,  507-523 
behavioral  engineering,  510,  518 
models,  511-515 
certification  programs,  523 
cognitive-motivational,  510, 
515-519 
defined,  508 
future  of,  523 
model,  513-514 
research,  522-523 
technological,  510,5 10-5 1 1 , 
519-522 

performance  measurement,  see 
performance  assessments 
performance  objectives 

cognitive  task  analysis,  588 
performances,  as  outcome  category, 
387 

Perry  Preschool  Project,  543 
Pershing’s  performance 

improvement  process  model, 
513-514 

personal  agents,  290 
personal  digital  assistants  (PDAs), 
89,  262, 263 

personal  information,  sharing  of,  227 
personal  mastery,  696 
personal  responsibility,  407 
personality  tests,  310 
personalization,  324,  722 
personalized  instruction,  see 
Personalized  System  of 
Instruction 
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Personalized  System  of  Instruction 
(PSI),  12,63,  187,  191-192, 
262,471,810 

historical  origins  of,  191-192 
model,  elements  of,  191 
psychological  principles  of,  192 
research  on,  192 
persuasive  technologies,  809 
perturbances,  defined,  620 
Pew  Internet  survey,  527 
phenomenological  knowledge,  488 
philosophical  perspectives,  67-80 
categories  of,  71-72 
phonemes,  543, 544 
phonograph  record,  7 
phonological  awareness,  543-544 
phonological  loop.  89,  94 
defined,  86 

Piaget,  Jean,  14,  15,  23,  24,  36, 45, 
49,58,69,74,78,405-406, 
549, 550 

PICCO,  550-551, 552 
Pillans,  James,  6 
place  independence,  259-265 
plan  construction,  284 
plan  library,  284 
plan  recognition,  284 
planned  change  theories,  620 
planner  systems,  EPSS,  519 
plasticity,  665 
Plato,  5, 24,  58,  71 
PLATO,  14,  189,471-472 
pluralistic  walkthrough,  662 
podcasts,  264 
polymorphism,  347 
positive  cathexis,  406 
positive  interdependence,  406-407 
defined,  402 

positive  interdependence  theory,  418 
positron-emission  tomography 
(PET),  788 

potential  group  performance ,  406 
power  law  of  practice,  785 
PowerSim,  334, 464 
practice,  defined,  175 
practice  fields,  100 
pragmatism,  58,  60,  69,  72 
defined,  68 
pragmatist,  3 1 1 
precedent,  688, 689,  690 

collecting/disseminating,  689 
defined,  685 

Precision  Teaching,  63,  187,  188, 
192-194 


historical  origins  of,  192-193 
psychological  principles  of,  193 
research  on,  193-194 
precision,  of  theory,  56,  57 
preconceptions,  57,  113,  322,  339 
prediction,  inquiry  learning,  459 
prescriptions,  for  collaboration,  164, 
165, 167, 168-170 
prescriptive  accounts,  defined,  164 
prescriptive  principle,  defined,  175 
prescriptive  principles  for 

instructional  design,  173-183 
prescriptive  theories,  708 
prescriptive  vs.  descriptive,  26,  61 
presentation  forms,  in  component 
display  theory,  390 
presentation  layer,  791 
primedia,  346,  348 
priming,  189,  190,  585 
primitives,  336 

Principle  Maker,  150-151,  158 
Principled  Assessment  Designs  for 
Inquiry  (PADI),  600 
prior  knowledge  ,130, 200 , 20 1 , 202 , 
203,204-205,207,208, 
306-307,321,372,373,387, 
410,464,474,582,589,721, 
732,810 

private  study,  learning  places  and, 
255 

Probe  Method,  491 
problem-based  learning,  15, 16, 100, 
101,102,103,176,213,386, 
485-500,531,540,541,553, 
562,707,721,722-723,809, 
810 

achievement,  and.  489, 490, 496, 

4989 

assessment  issues,  495-496 
assumptions,  488 
characteristics  of,  488-489 
collaborative  learning,  and,  492, 

498. 500 

confidence,  and,  492 
content  retention,  490-491 
curriculum  design,  496-498 
defined,  486,  539 
dental  education,  and,  491 
dilemma,  493 

distance  education,  and,  498 
distributed,  498 
engineering  students,  and,  489 
group  processing,  494-495, 496, 

497.498.500 


higher  order  thinking,  and, 
491-492 

history  of,  486 — 488 
implementation,  492-499 
learning  outcomes,  and,  486, 
489-492,495,497 
life-long  learning,  and,  488, 492 
medical  education,  and, 

486-487,490,491,496, 
497,499,540,561 
metacognition,  and,  491 , 492, 
493,494 

model.  3C3R,  497-498 
multimedia,  and,  498^199 
problem  design,  496, 497-498 
problems,  effectiveness  of, 
496-497 

process,  steps  of,  489 
reflection,  and,  492, 498 
research  on,  489-500 
roles,  493^194 
scaffolding,  and,  433 
self-directed  learning,  and,  492, 
493 

self-directed  learning,  and,  499 
self-perception,  and,  492 
study  of,  774 
technology,  and,  498^-99 
virtual,  498 

problem-centered  instruction,  175-176 
problem  solving,  555 
defined,  486 
self-directed,  413 
problem-solving  skills,  413, 430, 
489,491,497,561,811 
assessing,  495 
expert  vs.  novice,  371 
novices,  377 
transfer  of,  49 1 
problem  structuredness,  496 
procedural  information,  4C/ID  and, 
450 

procedural  knowledge,  246,  387, 
393,488,584,785 
procedural  learning,  446 
process  artifacts,  686 
process  entity,  337 
process  knowledge,  449 
process  measures,  777-778 
process  model,  696 
Process  of  Communication  model,  11 
processing  noise,  94 
process-oriented  design  and 
development,  660,  661 
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process-oriented  worked-out 
examples,  555 

process-tracing  techniques,  786,  789 
defined,  765 
proctors,  192 

product  artifacts,  686,  687-689 
product,  inquiry  learning,  459 
product-oriented  design  and 
development,  660 
product-oriented  worked-out 
examples,  555 

product  research,  739,  752-753, 
755-756,  757 
production  memory,  41 
production  rules,  243 
professional  development,  626 
program  control,  410 
program-controlled  instruction,  475, 
"  476 

program,  defined,  175 
program  evaluation  models,  606-608 
Program  for  Learning  in  Accordance 
with  Needs  (PLAN),  471 
program  sequences,  in  programmed 
instruction,  191 

program  theory  evaluation,  607 
programmed  instruction,  4,8,9,  12, 
63,77,187,188-191,510,710 
branching  vs.  linear,  190 
historical  origins  of,  188-189 
model,  elements  of,  190 
psychological  principles  of, 
189-190 

research  on,  190-191 
Programmed  Logic  for  Automated 
Teaching  Operations,  see 
PLATC)" 

programmed  technologies,  187-194 
defined,  188 

programming  languages,  336 
progress  testing,  562 
project-based  learning,  103-104, 532 
Project  Head  Start,  543 
project  manager,  instructional 
design,  570 

promotive  interaction,  406, 407, 418 
prompting,  189,  190 
prompts,  fading  of,  190 
prompts,  scaffolding,  434 
pronunciation,  543 
propositional  representation,  90,  91 
proteges,  431 
Protege,  246 

protocol  analysis,  245,  780 


prototyping,  734;  see  also  rapid 
prototyping 
proxemics,  772 

psychological  constructivism,  60 
psychological  perspectives,  72-76 
psychological  phenomena, 
knowledge  as,  387 
psychology  of  learning,  23-24 
punctuated  equilibrium,  620 
Purple  system,  346 

Q 

QMIND,  682 
QUADRATIC  tutor,  478 
qualitative  data  analysis,  615 
qualitative  evaluation,  607 
qualitative  modeling,  282 
qualitative  process  theory,  337 
qualitative  research,  defined,  717, 766 
qualitative  research  methods,  22, 
717,718,719,720,723,724, 
725,729,730,734,735,738, 
748,749,750,751,753,757, 
768,771-775 
quantitative  modeling,  282 
quantitative  research,  defined,  766 
quantitative  research  methods,  22, 
717,718,734,735,748,749, 
750,751,757,768 
quasi-experiment,  defined,  717 
quasi-experimental  studies,  22,  50, 
514,518,522,717,720,722, 
723,726,729,730,732,740, 
747 

query-based  retrieval  systems, 
216-217 
QUEST,  459 
Quest  Atlantis,  106 
questioning,  427,  431, 432 
questionnaires,  771, 777,  780-781 

R 

R2D2  model,  662 
radical  behaviorism,  1 1-12 
radical  constructivism ,15,60,61,69, 
72,74 

radio,  educational,  7-8 
RAND  change  agent  studies,  624, 
625 

random  generation,  372-373,  375 
randomized  field  trials  (RFTs),  720, 
727-728,729 


randomized  intervention  studies, 
719-720 

randomness  as  genesis  principle, 
372-373,374,375 
rapid  prototyping,  662,  664-666, 
667,709,740,753 
defined,  660 

rationalism,  69, 71-72, 73,74, 77, 78 
defined,  68 

read-think-aloud  protocols,  773 
reading  comprehension,  544,  545 
Reading  First,  543 
reading,  learning  how  to,  542-545 
Reading  Roots,  655 
realism,  71 
defined,  68 

realistic  mathematics  education 
(RME),  547,  548 

Really  Simple  Syndication  (RSS), 
350 

reasoning,  552-556 
ability,  308-309 
adversarial,  554-555 
case-based,  553 
defined,  539 
rule-based,  553 
recall,  115 
recall  data,  754-755 
recall-like  process,  40 
reception  learning,  78 
reciprocal  teaching,  101-102,  228 
recognition,  40 
recognition-based  long-term 
memory,  44 

recommender  systems,  246-247 
reconstructive  studies,  739 
recontextualized  resources,  528, 533, 
534 

reculturing,  625, 626 
defined,  620 
recurrent  tasks,  25 
reduction  to  absurdity,  45 
reductive  bias,  655 
redundancy 
defined,  86 
effect,  376-377 
multimedia  learning  and,  214 
reengineering,  see  business  process 
redesign/reengineering 
reference  values,  feedback  and,  126, 
128,  129, 130, 131 
reflection,  37,  101,  156-157,310, 
427,433,434 
defined,  34, 426 
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in  action,  665 
inquiry  learning,  460 
problem-based  learning,  and, 
492,498 

self-directed  learning,  and,  447 
reflector,  311 

regulation  of  learning,  460 
Relational  Frame  Theory  (RFT),  64 
relational  hypertext,  201 , 202,  203, 
205,206 
defined,  200 
relational  maps,  201 
relational  space,  166 
relationships,  asymmetric  vs. 

symmetric,  433 
relativism,  72 
defined,  68 
relativists,  530 
relaxation  networks,  45 
reliability,  775, 781 
test,  769-770 
remembering,  44-48 
replication  experiments,  727-728, 
729 

representation  strategies,  85-95 
research 

-based  educational  engineering, 
746-747,  748 

design  and  development,  see 
design  and  development 
research 

design-based,  see  design-based 
research 

designs,  715-757 
developmental,  727 
educational  technologies, 
730-739 
formative,  739 
instructional  design,  718, 
739-748 

instructional  strategies,  718, 
719-730 

model,  see  model  research 
nature  of,  22-23 
product,  see  product  research 
qualitative  vs.  quantitative,  22, 
61 

technology-  vs.  theory-oriented, 
356 

tool,  see  tool  research 
research  report,  writing,  775 
Research,  Education,  Information, 
and  School  Libraries  Special 
Information  Group,  239 


resource-based  learning,  444, 
525-534,810 
components,  528-530,  533 
defined,  526 
emergence  of,  526-528 
epistemology,  530 
foundations  and  assumptions, 
530-531 

issues  and  implications,  533-534 
research,  531-533 
resource  interdependence,  407 
resource  use,  526 
differences  in,  533 
issues  related  to,  534 
scaffolding  for,  532 
resources 

as  learning  objects,  347 
attention,  88 
changing  nature  of,  528 
defined,  526 

in  goal-based  scenarios,  449 
recontextualized,  528,  533,  534 
socially  constructed,  527 
user-centered  design  and 
development,  667 
respondent  learning,  73 
response  certitude  model,  127,  130 
response-contingent  elaborated 
feedback,  132 
responsive  evaluation,  608 
restructuring,  76,  625, 627 
defined,  620 

retrospective  reporting,  787 
reusability,  477 

reuse,  learning  objects  and,  345 , 346, 
347,348,349,350,351,352 
reward  systems,  514 
rewards,  324,  517 
rich  learning  environments,  9 
rising  edge,  798 
risk  taking,  academic,  219 
River  City,  461, 735 
robot-controlled  devices,  334-335 
role,  in  goal-based  scenarios,  449 
role  interdependence,  407 
root  metaphor,  56-62 

contextualism,  59,  60,  61, 62 
defined,  57 
formism,  57 
mechanism,  58 
organicism,  58 
rubrics,  768,  770 
rule-based  programming,  336 
rule-space  methodology,  598 


rules  of  thumb,  design,  151 
Rummler-Brache  three-level  model, 
512-513 

s 

Sakai,  18,226,358,361 

salient  features,  of  animations,  43 , 46 

sample  size,  612 

satisfaction,  272-273 

SAVER,  312 

scaffolding,  79,  91,  100,  101,  103, 
104, 150, 153-154, 165, 167, 
212,229,340,429,532,533, 
735 

4C/ID,  451 
by  mothers,  433 
cognitive,  459 
computer-supported 

collaborative  learning 
(CSCL),  and,  434-435 
conceptual,  529 
defined,  426,  526 
discourse-based,  433 
distributed,  432, 436 
fading,  and,  432 
for  performance,  432 
hard,  335 

hypertext  navigational,  203 
inquiry  learning,  459,  460 
just-in-time,  432 
metacognitive,  529 
online,  434 
procedural,  529,  532 
prompts,  434 
research  on,  432-435 
resource-based  learning,  and, 
529-530, 532 
soft,  335 

software-based,  433^134 
strategic,  529,  530 
teacher-provided,  433 
types  of,  427 

scaled  instructional  strategies 
defined,  174 

scenario  operations,  goal-based,  449 
scenarios,  design  of,  323 
schema,  45, 46, 49, 76, 91, 113, 181, 
347,373,387,445,446,583, 
587 

schemata,  41, 42, 44-47, 49,  1 13 
defined,  40 

Schlechty  Center,  698-699 
School  Libraries  Worldwide,  235 
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School  Library  Guidelines,  234 
School  Library  Journal,  238 
School  Library  Media  Research,  235 
school  library  resource  center 
(SLRC),  236-237,  238 
school-community  relationships, 

623 

science,  artificial  vs.  natural,  665 
science  learning,  549-552 
defined,  539 

science,  technology,  engineering, 
and  mathematics  (STEM),  650 
scientific  management,  510 
scientific  method,  217,434, 458, 465 
SCI- WISE,  478 
scope,  of  theory,  56,  57 
scoring,  600-601 
Scottish  Council  for  Museums, 
Archives,  and  Libraries,  236 
search  engines,  529 

as  electronic  performance 

support  system  (EPSS),  519 
Second  Life,  336 

secondary  task  paradigm,  795-796 
SECTION  model,  609 
selection,  as  learning  principle,  92 
self,  433 

self-assessment  skills,  131 
self-directed  learning ,133,447,454, 
488,489,492,493,495,496, 
497, 499,  559,  560;  see  also 
self-regulated  learning 
self-efficacy,  49-50,  121,  134,  139, 
321,491,510 
self-evaluations,  768 
self-generation,  113 
self-paced  learning,  259, 260, 
262-264 

self-perception,  problem-based 
learning,  and,  492 
self-reflective  learning,  489 
self-regulated  learning,  121,  127, 
131, 133,206-208,215,433, 
768 

defined,  200 

self-regulation,  114,  121,  122,274, 
309,433,709 
defined,  112 
self-representation,  309 
self-study,  253 
semantic  memory,  47, 48 
semantic  networks ,  246 
Semantic  Web,  244,  246,  289,  358, 
477 


semantics,  1 1 , 44, 46, 91 
cognitive,  641 
design  languages,  and,  635 
natural  language,  642 
SenseMaker,  149,  158,413 
sense-making,  78 
sensing/intuitive,  310,  311, 313 
sensors,  feedback  loops  and,  129 
sensory  memory,  defined,  86 
sequence  model,  589 
sequence,  programmed  instruction, 
191 

sequence  training,  443 
sequencing,  88, 433, 446, 448, 449, 
452 

elaboration  theory,  391 
sequential  analysis  of  group 
interactions,  780 
sequential  feedback,  138 
sequential/global,  307,  311 
serious  games,  212,  218-219 
7  ±  2  elements  of  information,  41 , 44 
Seven  Principles  for  Good  Practice 
in  Undergraduate  Education, 
609-610 
shaping,  189 

Sharable  Content  Object  Reference 
Model  (SCORM),  220, 288, 
348,349 

metadata  specification,  349 
sharable  content  objects,  347 
shared 

knowledge,  781 
meanings,  166 
mental  models,  653 
defined,  648 
repertoire,  428 
vision,  696,  700 
SHERLOCK,  478 
short-term  memory,  87,  88,  89,  92, 
94,  307,  530 
audio,  94 
defined,  86 

Sickle  Cell  Counselor,  452-453 
sidekick  systems,  EPSS,  519 
sigmas,  213 

signal-detection  theory,  44 
signs  and  rules  for  language,  24 
silent  films,  6,7,8 
SimCity,  529 
SimEarth,  529 

similarity,  as  root  metaphor,  57 
simplifying  conditions  method,  76 
SimQuest,  308,  337,  340, 463 


simulation,  318, 458;  see  also 
computer  modeling,  model- 
facilitated  learning,  simulations 
-based  discovery  learning,  308 
-based  training,  318 
defined,  318 
interactive  ,215-216 
defined,  212 

models ,  see  simulation  models 
software,  tools  for,  679 
simulation  models,  107,  329-342 
authenticity,  100-102 
defined,  98 
emergent,  102-104 
simulations,  15,  16,  34,  36, 46,  102, 
287,322,331,341,457,458, 
459,460,461,462,463,465; 
see  also  simulation 
agents,  and,  334 
as  black  boxes,  331 
as  learner-controlled 
environments,  410 
borrowing  principle,  and,  375 
Co-Lab,  and.  335 
cooperative  learning,  and,  416 
defined,  330, 457 
Design  Principles  Database,  and, 
155-156,  158 
fidelity  of,  322 

gaming-  vs.  modeling-oriented, 
331 

hard-skills,  679 
human  circulatory  system,  340 
instructional  design,  and, 
678-680 

instructional  transaction  theory, 
and, 348 

interacting  elements,  and,  379 
management,  321 
medical,  561 
participatory,  105 
physics,  339 
redundancy,  and,  377 
soft-skills,  672,  678,  679 
split-attention  effect,  and,  375 
to  teach  weight/density,  340 
workplace,  101 
simultaneous  feedback,  138 
single-photon  emission  computed 
tomography  (SPECT),  788 
single-try  feedback,  138 
situated  cognition,  15, 69,  79 
situated  design,  662 
situated  feedback,  449 
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situated  learning ,  3 1 9-320 ,488,531, 
730,734 
defined,  252 

situated  science,  defined,  648 
situatedness,  254, 427 
defined,  426 
situation  model,  200 
situational  analysis,  700 
situational  embodiment,  97-108 
defined,  98 
situational  model,  91 
situationally  embodied  curriculum, 
99,  108 

situative  theory,  98,  99 
situativity  theory,  69,  72,  75-76,  79 
defined,  68 

dualism  perspective,  75-76 
epistemology,  75 
ontology,  75 
unit  of  analysis,  75,79 
sketching,  cognition  and,  686-687 
Skinner,  B.  F„  12, 62, 63,69, 73, 77, 
98, 188-189, 190, 191, 192, 
193,406,510,511,530,707 
Skolelinux,  358 
slide  projectors,  6,  7 
Sloan  Consortium’s  five  pillars,  270 
small  group  skills,  407 
Smithtown,  287, 459, 460 
Soar  architecture,  41 
social  cognitive  theory,  510 
social  constructivism,  15, 60, 61, 69, 
74 

social  influence  theory  of  change, 
621 

social  interdependence  theory,  402, 
405,406,417,418 
social  learning,  323,427, 436 
theory,  47 1 , 640-641 
social  network  analysis,  781 
social  presence  ,411,416 
lack  of,  264 

social  recognition,  517,  518 
social  skills ,  404 , 405 , 406 , 407 , 4 1 7 , 
418,557,561 
social  status,  324 
social  systems  design,  693 
social  worlds,  168 
socialization,  cognitive 

apprenticeship  and,  435 
socioconstructivists,  708 
sociocultural  constructivism,  72 
sociocultural/historicism,  69,  72, 
74-75,78-79 


defined,  68 

dualism  perspective,  75 
epistemology,  74-75 
ontology,  74-75 
unit  of  analysis,  75 
socioeconomic  status,  31, 235,  295, 
297,300,301,302,533,540, 
543,551,552 
sociolinguistics,  640-641 
Socrates,  5, 6 

soft-skills  simulations,  678,  679 
defined,  672 

soft  technologies,  279,  282-285 
defined,  278 

software,  672,  676,  678,  683 
-based  scaffolding,  433-434 
closed  source,  357 
collaborative  learning,  and,  342 
dynamic  modeling,  338 
features,  defined,  146 
free,  357 

modeling  technologies,  and,  330 
modularity,  665 
open  source,  355-363 
simulation  tools,  679 
SOPHIE,  459 
Sophists,  5 

sound,  instructional,  92-94 
source  code,  357 
SourceForge,  357,  361 
space 

academic  play,  105-106 
design,  662 
content,  166 
intentional  learning,  31 
relational,  166 
spatial  ability,  321 
spatial  priming,  337 
Special  Needs  Opportunity  Window 
(SNOW),  286 

specification,  modeling,  462, 465 
specificity,  447 
speech-capture  devices,  286 
speed,  of  task  performance,  785 
split-attention  effect,  43,  375,  376, 
723 

split  halves  method,  769 
spoken  language  processes,  in 
groups,  779 
sports,  442, 443-444 
stage  model,  of  information 
processing,  41, 48 
Stagecast  Creator,  338 
stages  of  concern,  621 


Standards  for  School  Library 
Programs,  234 
Standards  for  School  Media 
Programs,  234 

Stanford  Microarray  Database  (SMD), 
781 

StarLogo,  336 
STEAMER,  459 

STELLA,  334,  335,  339,  340, 462, 
464, 733 

step  size,  in  programmed  instruction, 
191 

Step-Up-To-Excellence,  696,  698 
Steps  of  Practical  Self-Cultivation,  6 
stereotyping,  adaptive  methods  and, 
284 

stimulus,  control,  and  response  layer 
(SCRL),  791-797,  798 
stimulus  control,  transfer  of,  189, 190 
stimulus  presentation  and  control 
(SPC)  model,  791-797 
with  external  hardware  (SPCxh), 
793-797 

stimulus-response  pairs,  445, 446 
storyboard,  682 
defined,  672 
strange  attractors,  696 
strategic  leadership  team,  698 
stratification,  637 
stream-of-behavior  chronicles,  772 
street-level  bureaucrats,  621 
stretching  the  mold,  273 
structured  interviews,  773-774,  780 
structure-guidance-coaching- 
reflection  cycle,  176 
defined,  174 

student-expert  partnerships,  105 
student  model,  279;  see  also  learner 
model 

cumulative/persistent,  284-285 
temporary,  285 

student  modeling,  defined,  306 
student  satisfaction,  272 
study  design,  612-613 
subject-matter  experts  (SMEs),  580, 
581,582,583,588,589,672 
defined,  578 

subject  matter,  five  operations 
conducted  on,  390 
subjective-objective,  70 
subjectivity,  remembering  and,  40, 48 
substitutability,  406 
successful  working,  as  truth  criterion, 
58 
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summarizers,  elaboration  theory,  391 
Summary  Street,  216 
summative  evaluations,  606 
Supercharged!,  320 
supplantive  events  of  instruction,  392 
Supporting  Multiuser  Interactive 
Learning  Environment 
(SMILE),  151 

supportive  information,  587-588 
supportive  information,  4C/ID  and, 
450 

Surface,  Matching,  Deep  Structure 
(SMD),  47 
surveys,  740,  777 
symbolizing,  547 
symbol-matter,  70 
synchronization  of  SCRL 
applications,  795 

synchronous  learning,  259, 260, 261 , 
264-265,415 

synchrony,  in  computer- mediated 
communication,  226 
syntax,  642 

synthesizers,  elaboration  theory,  391 
synthetic  experience,  318 
synthetic  learning  environments, 
317-324 

advantages  of,  320 
authenticity/fidelity  of,  322 
collaborative/social  learning, 
and, 323 
defined,  318 
design  of,  321-324 
effectiveness ,  empirical  literature 
regarding,  320-321 
embodiment,  and,  324 
engagement/emotional  context 
of,  323-324 

justification  for  use  of,  319-320 
model-based  reasoning,  and,  322 
reward,  and,  324 
scenario/case  design,  and,  323 
social  status,  and,  324 
spatial  ability,  and,  321 
technological  fluency,  and,  321 
system 

adaptive,  279 
defined,  29,  279,  694 
dynamics,  461-462, 463 
systematic  educational  engineering, 
740 

systemic  change,  692,  693 
defined,  692 
design  model,  699 


design  theory,  and,  693 
ecological,  692 

learning  organizations,  and,  696 
mindsets,  and,  695 
need  for,  693 

participatory  leadership,  and, 
695-696 

shared  vision,  and,  696 
stakeholders,  and,  695 
systems  design,  691-701, 709 
defined,  692 
foundations  of,  693-694 
models,  696-700 
need  for,  693 
principles  of,  694-696 
stakeholder  involvement  in,  695 
systems  theory,  693,  694,  696,  700 
defined,  692 
liberating,  694 

systems  theory,  feedback  and, 
128-129 

systems  thinking ,  692 , 693 , 694 , 696 , 
809 

hard  vs.  soft,  694 

T 

T’oegye,  Yi,  6 
Tactical  Iraqi,  218 
tagging,  349 

task,  see  also  learning  tasks 
analysis,  cognitive,  577-591 
authentic,  784 

-centered  instruction  principle , 
defined,  174 

-centered  instructional  strategy, 
176, 178, 181-182 
defined,  174 

classes,  450-451, 587,  588 
complex,  see  complex  tasks 
complexity,  130,  139,  336-339, 
427,522,783 
defined,  766 
interdependence,  407 
recurrent  vs.  nonrecurrent,  25 
representation,  599 
structuring,  433 

Task  Load  Index  (TLX),  NASA,  785 
teachable  agents,  285 
teacher  change ,  624 
teacher  education,  428, 429^-30, 
431,487 

teaching-learning  vs.  design  and 
development  research,  752 


teaching  machines,  12,  77, 189,  193 
teaching,  model-oriented,  46-47 
types  of,  189 

team  development  publishing  tools, 
680-683 

Team  Evolution  and  Maturation 
(TEAM)  model,  651 
defined,  648 

team-leader  behavior,  types  of,  652 

team  learning,  696,  777 

teams 

competencies  of,  653 
distributed,  651 
leadership  of,  65 1 , 652 
model  for  development  of,  65 1 
optimal  context  for,  653-654 
research  on,  650-655 
self-managed,  652 
training  requirements  for,  653 
vs.  working  groups,  650-651 
teamwork,  defining,  650-65 1 
technological  knowledge,  open 
source  software  and,  361 
technological  performance 

improvement,  510,  510-511, 
519-522 
technology,  11 

adaptive,  277-291 
-based  instruction,  371 , 372,373, 
374-379,  749 
attitudes  toward,  411 
-based  knowledge  systems, 
241-247 

-based  learning,  436,  605,  606, 
608,612,771 

-based  performance  assessments, 
595-602 

cooperative  learning,  and, 
401-418;  see  also 
computer-supported 
cooperative  learning 
defined,  5,  279 

-enhanced  learning,  36, 306, 307, 
394,  572,  574;  see  also 
computer-supported 
cooperative  learning, 
synthetic  learning 
environments,  technology- 
supported  cooperative 
learning 

-enhanced  learning 

environments,  733,  734, 
735,736,739 
defined,  718 
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-enhanced  support  strategies, 
145-159 

evaluation,  608-616 
hard,  278,  279,  282,  285-286 
integration,  811 
learning,  cognition,  and 

instruction,  and,  305-313 
learning  how  to  use,  410 
-mediated  instruction,  269,  270 
of  teaching,  12 

-oriented  research,  defined,  356 
soft,  278,279,282-285 
-supported  cooperative  learning, 
402,403,404,412, 
414-418 

time,  pace,  place  independence, 
and, 259-265 

Technology-Enhanced  Learning  in 
Science  (TELS),  150,461 
TELCERT,  359 

television,  see  educational  television 
templates,  authoring  tool,  674 
templates,  design,  681-682 
templates,  for  students,  150 
temporal  flexibility,  253 
Ten  District  Standards,  699 
tenstrips,  548 

test  items,  categories  of,  463 
test-retest,  769 
testing  implementations,  310 
Tests  of  Achievement  and 
Proficiency,  235 
tests,  selecting,  768-771 
text  design  guidelines,  89-91 
textbase,  91, 200 
Textbook  Revolution,  352 
theoretical  foundation,  defined,  21 
theoretical  statements,  706-708,  710 
defined,  706 

instructional  design,  and,  706 
justification  of,  708 
object  of,  706-707 
origins  of,  707 
sophistication  of,  707-708 
theories,  see  also  theoretical 
statements 

descriptive,  708-709 
instruction,  710 
instructional  design 
developing,  709-710 
justification,  7 10-7 1 1 
sophistication,  710-711 
prescriptive,  708,  709 
theorist,  3 1 1 


theory  development,  705-711 
defined,  706 
theory 

-based  evaluation,  607 
-driven  evaluation,  607 
meaning  of,  56,  706 
nature  of,  22-23 
of  change,  699 
of  engagement,  699 
-oriented  research 
defined,  356 

Thermodynamics :  Probing  Your 
Surroundings,  151 
think-aloud  protocols,  41, 579,  589, 
740,747,773,786 
ThinkerTools,  460-461 , 462,  465 
thinking 

abstract  vs.  concrete,  23 
aloud,  see  think-aloud  protocols 
descriptional  vs.  propositional,  686 
language  games,  and,  24 
nonverbal,  687 
tags,  341 

thinking/feeling,  310 
Thompson/NETg  study,  453 
Thorndike’s  law  of  effect,  126 
threaded  discussions,  15,  226,  227, 
246,435 

threads,  conversational,  219 
3C3R  problem-based  learning 
model,  497-498 

three-dimensional  manipulation, 
152-154 

thresholds,  group  learning  process 
assessment,  782 
Thunderbird,  358 
time  and  motion  studies,  510 
time  independence,  259-265 
time  on  task,  785 
Title  VII  audio-visual  research.  9 
TMGL  landscape,  see  computer- 
mediated  technologies 
tool  research,  739,  752-753, 
755-756, 757 
defined,  718 

toolbox,  of  media  design  principles, 
86-89,95 

tools,  see  also  tool  research 
communication,  529 
defined,  526 
manipulation,  529 
processing,  529 
resource-based  learning,  529 
searching,  529 


topic-centered  instructional  strategy, 
182 

defined,  174 

topic-contingent  elaborated 
feedback,  132 

Traditionalist  Generation,  297 

trailing  edge,  798 

training 

design,  cognitive  task  analysis, 
and, 587-588 

electronic  performance  support 
systems,  and,  522 
films,  7,  8 

open  source,  and,  358 
requirements,  team,  653 
systems  design  for  change  in, 
691-701 

vs.  natural  learning,  404 
transactional  leaders,  defined,  648 
transactions,  393 
transclusion,  346 

transfer  of  learning,  445-446, 447, 
448,450,451 
increasing,  447 

transfer  of  stimulus  control,  189,  190 
transfer  paradox,  447 
transferring  knowledge,  371-372, 
387,491 

transformational  leaders,  defined, 
648 

transformative  inquiry  processes, 
460 

transistor-transistor  logic  (TTL),  798 
treatment,  defined,  472 
triangulation,  776 
trigger,  797,  798,  799 
Triumph  of  the  Will ,  8 
truth  criterion,  56-60 
contextualism,  58 
defined,  57 
formism,  57 
mechanism,  58 
organicism,  58 
tuning,  76 

turn-to-your-partner  discussions,  405 
Tutor  Intervention  Profile  (TIP),  494 
tutoring,  see  also  tutors 
adaptive,  470 
feedback,  132-140 
evaluating,  139-140 
for  legal  education,  555 
micro-adaptive  instruction,  and, 
473 

systems,  contingent,  478 
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tutors,  131,  166,207,217,218,219, 
285,318,320,473,474,478, 
479, 489, 492, 493^94, 495, 
768;  see  also  tutoring 
as  metacognitive  guides,  494 
TVOntario,  10 

u 

uncertainty,  defined,  29 
unconscious,  automated  knowledge, 
590 

unfreeze-change-refreeze  cycle,  620 
Unified  Modeling  Language  (UML), 
360-361,362,363 
unit  of  analysis,  69,  70 
behaviorism,  73,  77 
cognitive  constructivism,  74 
cognitivism,  74 
defined,  68 

situativity  theory,  75,79 
sociocultural/historicism,  75 
United  Nations  Educational, 
Scientific,  and  Cultural 
Organization  (UNESCO),  572 
units  of  analysis,  612 
UNIX,  357 

unstructured  interviews,  580,780 
usability 

defined,  660 
documentation,  743 
USD,  312 

user-centered  design,  520,  662,  665, 
808 

defined,  660 
user-centered  design  and 
development,  659-668 
user  control,  200 
user  embodiment,  324 
user-friendly  design,  662 
user  modeling,  216 
user  participation,  in  design  and 
development,  663,  666-667; 
see  also  participatory  design 
uses  and  gratification,  50-51 
utilization-focused  evaluation,  607 
UTOPIA,  663 

V 

validation,  defined,  596 
validation  studies,  717,  718,  739, 
740,742,743,745,746,747, 
748,749,750,751,753 


validity,  741, 775 

construct-related,  769 
content-related,  769 
criterion-related,  769 
defined,  596 
experimental,  threats  to, 
726-727,729,730 
external,  727,  732,  734,  743 
face,  769 

internal  ,726,727,730,731,732, 
734,  743 

model,  607,710;  see  also  model: 
validation 

performance  assessment,  596-598 
criteria,  597 
test,  769 
true,  729 
Vensim,  334 

verbal  communication,  41 
verbal  information,  learning,  306 
verbal  protocols,  787,  788 
verbal  reports,  786-787,  788,  789 
verbal/visual,  311.313 
verification  feedback  component, 

135 

veterinary  students,  simulation  and, 
321 

vicarious  overt  trials,  687 
video  design  guidelines,  94 
video  games,  169;  see  also  games 
Video  Traces,  169-170 
villages,  experiential  learning,  36 
Virtual  Collaborative  Research 
Institute  (VCRI),  414 
virtual  communities,  245 
virtual  environments,  218,  810 
virtual  humans,  809 
virtual  problem-based  learning,  498 
virtual  reality,  287 
virtual  schools,  17-18 
Virtual  Usability  Lab  (VULab), 
614-615 

virtual  worlds,  318,  322,  324 
persistent,  319,  320 
Visual  Instruction  movement,  6-7 
Visual  Math,  156 
visual  media,  early,  6 
visual  sketchpad,  86 
visualization,  24 

Design  Principles  Database,  and, 
155-156 
visuals,  86 

categories  of,  91 
Vmodel,  335, 337 


VMware,  363 

voice  over  IP  (VoIP),  260,  265 
VPBL,  498 

Vygotsky, Lev,  14, 15,60,69,78,91, 
165,405,406,478,549 

w 

warfare,  asymmetric,  244 
warrants,  737 

weak  intelligent  design  support 
systems,  348 

Web-based  adaptive  hypermedia,  475 
Web-Based  Inquiry  Science 
Environment  (WISE),  147, 
148-149, 150-151,  157,  158, 
533 

Web  Knowledge  Sea,  477 
Web  log  file,  613-614 
defined,  606 
WebCSILE,  434 
WebCT,  226 
WebLab,  153-154 
WebQuests,  16,  528 
whole-task  models,  441^-54,  588, 
809,810 

competence  development,  and, 
446-447 
defined,  442 

educational  communications  and 
technology,  and,  446-447 
evidence  for  effectiveness  of, 
451-453 

examples  of,  447-45 1 
history  of,  443^146 
wholist-analytic  questionnaire,  205 
WHURLE,  477 
wicked  problems,  709 
Wikipedia,  228,  246,  358,  810 
wikis,  16,  228,  264,  527,  529,  606, 
736,  809 
WordMath,  430 
words,  learning,  544 
worked  example  effect,  374—375,  377 
worked-out  examples 

process-  vs.  product-oriented,  555 
for  legal  education,  555 
working  groups,  650-651 
working  memory ,  4 1 , 42 , 43 , 44 ,  8 8 , 
92,307-308,312-313,337, 
373,375,376,378,379,380, 
530 

anxiety,  and,  49 
defined,  370 
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executive  function,  88-89 
expanding,  376 
long-term,  44 

narrow  limits  of  change,  and, 
373 

Working  on  the  Work,  699 
workplace 

electronic  performance  support 
systems  in,  508,  511 
knowledge  transfer  to,  491 
learning,  561 , 562 
mentoring,  430^-31 
models  used  in,  341 
motivation  in,  510 
performance,  510,  515,  542 
performance  improvement,  and, 
386 

problem-solving  skills,  and,  491 
research  in,  755 


research  problems,  and,  749-750 
self-directed  learning  in,  560 
self-efficacy,  and,  50 
simulation,  101 
simulation  use  in,  321 
synthetic  learning  environments, 
and, 320 

world  hypothesis,  56,  58 

World  War  I  military  training  films, 
7 

World  Wide  Web,  15,  16-17,51, 
203,212,244,246,247,263, 
346,360,415 

World  Wide  Web  Consortium 
(W3C),  360 

worldviews,  56-58,  59,  61, 62,  64, 
69,  75,  80;  see  also 
philosophical  perspectives 
types  of,  57 


written  language  processes,  in 
groups,  779 
WUSOR,  284 

X 

Xanadu,  346, 348 

XML  Schema  Definition,  362 

XSLT,  349 

z 

zone  of  learnability,  91 
defined,  86 

zone  of  proximal  development 
(ZPD),  78,  91, 428, 432, 434, 
436 

defined,  426 
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